\documentclass[10pt,UKenglish]{article}

\RequirePackage{amsthm,amsmath,amsfonts}
\RequirePackage{bm}

\usepackage{enumerate}
\usepackage{paralist}
\usepackage{graphicx}
\usepackage{hyperref}

\usepackage{color}
\newcommand{\E}{\mbox{E}}
\newcommand{\sd}{\mbox{sd}}
\newcommand{\V}{\mbox{V}}

\usepackage{url}
\usepackage{parskip}

\newcommand{\B}{\boldsymbol}
\newcommand{\Bb}{\mathbf}

\begin{document}

\begin{center}
\section*{STK2100 -- Machine Learning and Statistical Methods for Prediction and Classificatio}
\subsection*{Mandatory assignment 2 of 2}
\end{center}

\subsubsection*{Submission deadline}

Thursday April 24 2025, 14:30 in Canvas (\url{canvas.uio.no}).

\subsubsection*{Instructions}

Note that you have \textbf{one attempt} to pass the assignment. This means that 
there are no second attempts. You can choose between scanning handwritten notes 
or typing the solution directly on a computer (for instance with Latex). The assignment 
must be submitted as \textbf{a single PDF file}. Scanned pages must be clearly legible. 
The submission must contain your name, course and assignment number.

It is expected that you give a \textbf{clear presentation with all necessary explanations}. 
Remember to include all relevant plots and figures. All aids, including collaboration, are allowed, 
but the submission must be written by you and reflect your understanding of the subject. If we 
doubt that you have understood the content you have handed in, we may request that you 
give an oral account.

In exercises where you are asked to write a computer program, you need to hand in the code 
along with the rest of the assignment. It is important that the submitted program contains a 
trial run, so that it is easy to see the result of the code.

\subsubsection*{Application for postponed delivery}

If you need to apply for a postponement of the submission deadline due to illness or other 
reasons, you have to contact the person responsible for the course, Ingrid Hobæk Haff 
(e-mail: \href{mailto:ingrihaf@math.uio.no}{ingrihaf@math.uio.no}), no later than the 
same day as the deadline. All mandatory assignments in this course must be approved in the 
same semester, before you are allowed to take the final examination.

\subsubsection*{Specifically about this assignment}
Include the code that you have used in the report.

\subsubsection*{Complete guidelines about delivery of mandatory assignments:}
\url{www.uio.no/english/studies/admin/compulsary-activities/mn-math-mandatory.html}

\begin{center}
GOOD LUCK!
\end{center}  

\pagebreak

\subsubsection*{Problem 1}
Consider the following figures:\\
\begin{tabular}{cc}
\includegraphics[scale=0.3]{figure-A.png}&\includegraphics[scale=0.3]{figure-B.png}\\
\includegraphics[scale=0.3]{figure-C.png}&\includegraphics[scale=0.3]{figure-D.png}
\end{tabular}

\begin{enumerate}[(a)]
\item Which of the above divisions of the covariate space into regions $R_{m}$ can have been 
produced by a regression tree? Explain why the one/ones you did not select cannot have been 
produced by a regression tree.
\item For the divisions that could have been produced by a regression tree, draw the corresponding
tree, including the formulas in the nodes (of the type $x_{j} \leq t_{k}$) and the values $c_{m}$ in the 
leaves.
\item Using the tree(s) selected in (b), provide an estimate for the following values:
\begin{enumerate}[i)]
\item $f(x_{1} = 0.6, x_{2} = 0.6)$
\item $f(x_{1} = 0.1, x_{2} = 0.6)$
\item $f(x_{1} = 0.6, x_{2} = 0.1)$
\end{enumerate}
\end{enumerate}

\subsubsection*{Problem 2}
In this problem, we will consider a slightly modified version of the the data set \texttt{Wage} from 
the book ISLR (see the weekly exercises), where the aim is to predict the wage of a group of men 
from the Atlantic Region in the U.S., based on a set of covariates, using different models. The data
are available in the file \texttt{wage.csv} under 
\url{/studier/emner/matnat/math/STK2100/v25/oblig/}, and 
can be be obtained using the following R commands:\\
\begin{verbatim}
wage <- 
read.csv("/studier/emner/matnat/math/STK2100/v25/oblig/wage.csv",
header=TRUE)
\end{verbatim} 
The data set contains the following columns:
\begin{itemize}
\item \texttt{year}: the year when the wage was registered
\item \texttt{age}: age
\item \texttt{maritl}: marital status (Married/Unmarried)     
\item \texttt{race}: race (White/Black/Other)       
\item \texttt{education}: highest degree of finished education 
(< High scool/High school grad/Some college/College grad/Advanced Degree)  
\item \texttt{jobclass}: type of job (Industrial/Information)  
\item \texttt{health}: state of health ($\leq$ Good/$\geq$ Very good)     
\item \texttt{health$\_$ins}:  has health insurance (Yes/No)
\item \texttt{wage}
\end{itemize}
Note that the two covariates \texttt{year} and \texttt{age} are quantitative, and the remaining ones 
are qualitative. R-code that is particularly relevant are the ones from the lectures on January 28 and
March 25.

\begin{enumerate}[(a)]
\item (i) Start by randomly dividing the data set into a training and a test set. Make sure that both 
the training and the test set contain all the categories of qualitative covariates. Use the same 
training and test sets in all the sub-problems of Problem 2.\\
(ii) Fit a linear regression model to the training set, and comment on the fit. Do all covariates seem
to have an effect on the wage? Justify your answer.\\
(iii) Evaluate the prediction performance on the resulting model on the test set, using the mean squared error loss $\frac{1}{N}\sum_{i=1}^{N}(y_{i}-\hat{y}_{i})^{2}$ as the performance
measure. This will be compared to the performance of models later in the problem.
\item (i) Fit an additive model to the same data, using natural cublic splines for the two
quantitative covariates \texttt{year} and \texttt{age}.\\
(ii) Why should you not model the qualitative covariates with splines?\\
(iii) Should both quantitative covariates be modelled with splines, or can the effect of one of them or 
both be just as well captured by a simple linear term? Justify your answer.\\
(iv) Do all the covariates seem to have an effect on the wage? Justify your answer.
(iv) Evaluate your model of choice on the test data, and compare to the linear regression model
from (a).
\item (i) Fit a regression tree to the data, and plot it. Which covariates contribute to the model? 
Comment on potential differences from the models fitted in a) and b) with respect to this.\\
(ii) Prune the tree using 10-fold cross-validation. Plot the resulting tree, and comment on potential
differences from the tree fitted in (i).\\
(iii) Evaluate your regression tree of choice on the test data, and compare the results to the ones
you got in (a) and (b). Which model would you choose for predicting the wage? Justify your answer.
\end{enumerate}


\subsubsection*{Problem 3}
In this problem, we will consider a slightly modified version of the \texttt{Vertebral Column} 
dataset from (\url{https://archive.ics.uci.edu/dataset/212/vertebral+column}). It contains 
information about 310 orthopaedic patients, which belong to 2 classes healthy and unhealthy 
(these patients have either disk hernia or spondilolysthesis, but are put in the same class). Each 
patient is represented in the data set by 6 biomechanical attributes derived from the
shape and orientation of the pelvis and lumbar spine, as well as the class
\begin{itemize}
\item \texttt{pelvInc}: pelvic incidence
\item \texttt{pelvTilt}: pelvic tilt
\item \texttt{lumbLord}: lumbar lordosis angle
\item \texttt{SacrSl}: sacral slope
\item \texttt{pelvRad}: pelvic radius 
\item \texttt{degrS}: grade of spondylolisthesis
\item \texttt{class}: healthy=0 / unhealthy=1
\end{itemize}The data
are available in the file \texttt{vertebral-column.csv} under 
\url{/studier/emner/matnat/math/STK2100/v25/oblig/}, and 
can be be obtained using the following R commands:\\
\begin{verbatim}
vert <- 
read.csv("/studier/emner/matnat/math/STK2100/v25/oblig/vertebral-column.csv",
header=TRUE)
\end{verbatim} 
The aim is to predict whether a patient is healthy or not based on the 6 biomechanical attributes,
using different classification methods. R-code that is particularly relevant are the ones from the 
lectures on February 25 and April 2.

\begin{enumerate}[(a)]
\item (i) Start by randomly dividing the data set into a training (2/3) and a test set (1/3), such that
the fraction of healthy and unhealthy patients is approximately the same in the two sets. Use the 
same training and test sets in all the sub-problems of Problem 3.\\
(ii) Fit a logistic regression model to the training set, and comment on the fit.  Do all covariates seem
to have an effect on the risk of being unhealthy? Justify your answer.\\
(iii) Evaluate the prediction performance on the resulting model on the test set, using the
mean misclassification error $\frac{1}{N}\sum_{i=1}^{N}I(y_{i} \neq \hat{y}_{i})$ as performance 
measure. This will be compared to the performance of models later in the problem.
\item (i) Fit a linear discriminant analysis (LDA) model to the same data.\\
(ii) Evaluate the model on the test set, and compare to the logistic regression model.
\item (i) Fit a quadratic discriminant analysis (QDA) model to the same data.\\
(ii) Evaluate the model on the test set, and compare to the logistic regression and the LDA model.
\item (i) Fit a one-layer neural network to the data, trying different sizes (number of hidden 
neurons) and decay parameters $\lambda$ (remember to standardise the covariates).\\
(ii) Evaluate the fitted models on the test set, and compare the best performing one to the
models fitted earlier in the problem.
\end{enumerate}

\end{document}
