Assignment 3 Solution

$30.00 $24.00

Problem 1 [25%] Suppose that I collected data for a group of machine learning students from last year. For each student, I have a feature X1 = hours studied for the class every week, X2 = overall GPA, and Y = whether the student receives ˆ ˆ ˆ = 1.0. an A. We fit a…

5/5 – (2 votes)

You’ll get a: zip file solution

 

Description

5/5 – (2 votes)

Problem 1 [25%]

Suppose that I collected data for a group of machine learning students from last year. For each student, I have a feature X1 = hours studied for the class every week, X2 = overall GPA, and Y = whether the student receives

ˆ

ˆ

ˆ

= 1.0.

an A. We fit a logistic regression model and produce estimated coefficients, β0

= −6, β1

= −0.1, β2

  1. Estimate the probability of getting an A for a student who studies for 40h and has an undergrad GPA of 2.0

  2. By how much would the student in part 1 need to improve their GPA or adjust time studied to have a 90% chance of getting an A in the class? Is that likely?

Problem 2 [25%]

Consider a classification problem with two classes T (true) and F (false). Then, suppose that you have the following four prediction models:

  • T: The classifier predicts T for each instance (always)

  • F: The classifier predicts F for each instance (always)

  • C: The classifier predicts the correct label always (100% accuracy)

  • W: The classifier predicts the wrong label always (0% accuracy)

You also have a test set with 60% instances labeled T and 40% instances labeled F. Now, compute the following statistics for each one of your algorithms:

Statistic Cls. T Cls. F Cls. C Cls. W

recall

true positive rate

false positive rate

true negative rate

specificity

precision

Some of the rows above may be the same.

Problem O2 [30%]

This problem can be substituted for Problem 2 above, for up to 5 points extra credit. The better score from problems 2 and O2 will be considered.

Solve Exercise 3.4 in [Bishop, C. M. (2006). Pattern Recognition and Machine Learning].

1

Problem 3 [25%]

In this problem, you will derive the bias-variance decomposition of MSE as described in Eq. (2.7) in ISL. Let

  • be the true model, fˆ be the estimated model. Consider fixed instance x0 with the label y0 = f(x0). For simplicity, assume that Var[ ] = 0, in which case the decomposition becomes:

E h(y0fˆ(x0))2i = Var[fˆ(x0)]

+

E[f(x0) − fˆ(x0)]

2

.

|

{z

}

|

{z

}

| {z }

test MSE

Prove that this equality holds.

Hints:

Variance Bias

1. You may find the following decomposition of variance helpful:

Var[W ] = E (W − E[W ])2 = E W 2 − E[W ]2

2. This link could be useful: https://en.wikipedia.org/wiki/Variance#Basic_properties

Problem 4 [25%]

Please help me. I wrote the following code that computes the MSE, bias, and variance for a test point.

set.seed(1984)

population <- data.frame(year=seq(1790,1970,10),pop=c(uspop)) population.train <- population[1:nrow(population) 1,] population.test <- population[nrow(population),]

E <- c() # prediction errors of the different models for(i in 1:10){

pop.lm <- lm(pop ~ year, data = dplyr::sample_n(population.train, 8))

e <- predict(pop.lm, population.test) population.test$pop

E <- c(E,e)

}

cat(glue::glue(“MSE: {mean(E^2)}\n”,

“Bias^2: {mean(E)^2}\n”,

“Var: {var(E)}\n”,

“Bias^2+Var: {mean(E)^2 + var(E)}”))

## MSE:

2869.61343086216

## Bias^2:

2681.61281912074

##

Var:

208.889568601581

##

Bias^2+Var:

2890.50238772232

I expected that the MSE would be equal to Biasˆ2 + Variance, but that does not seem to be the case. The MSE is 2402.515 and Biasˆ2 + Variance is 2428.706. Was my assumption wrong or is there a bug in my code? Is it a problem that I am computing the expectation only over 10 trials?

Hint: If you are using Python and need help with this problem, please come to see me (Marek).

2

Assignment 3 Solution
$30.00 $24.00