Homework 4 Solution

$30.00 $24.00

Q1 (100 pts) In this computer project, we will use a neural network for curve fitting. Draw n = 300 real numbers uniformly at random on [0, 1], call them x1, . . . , xN. Draw n real numbers uniformly at random on [− 101 , 101 ], call them ν1, . . .…

5/5 – (2 votes)

You’ll get a: zip file solution

 

Categorys:
Tags:

Description

5/5 – (2 votes)
  • Q1 (100 pts) In this computer project, we will use a neural network for curve fitting.

    1. Draw n = 300 real numbers uniformly at random on [0, 1], call them x1, . . . , xN.

    1. Draw n real numbers uniformly at random on [ 101 , 101 ], call them ν1, . . . , νN.

    1. Let dI = sin(20xI) + 3xI + νI, i = 1, . . . , n. Plot the points (xI, dI), i = 1, . . . , n.

We will consider a 1 × N × 1 neural network with one input, N = 24 hidden neurons, and 1 output neuron. The network will thus have 3N + 1 weights including biases. Let w denote the vector of all these 3N + 1 weights. The output neuron will use the activation function φ(v) = v; all other neurons will use the activation function φ(v) = tanh v. Given input x, we use the notation f (x, w) to represent the network output.

    1. Use the backpropagation algorithm with online learning to find the optimal weights/network that minimize

the mean-squared error (MSE)

1

N (dI f (xI, w))2. Use some η of your choice. Plot the number of

N

I=1

epochs vs the MSE in the backpropagation algorithm.

Hint: As discussed in class, for a given fixed η, the algorithm may not always result in a monotonically decreasing MSE (the descent may overshoot the locally optimal point). You may have to modify the gradient descent algorithm in such a way that you decrease η (e.g. via η 0.9η) whenever you detect that the MSE has increased. Also, beginning with a very large η may result in an immediate divergence of the weights.

  1. Let us call the weights resulting from the backpropagation algorithm (when it converges) as w0. The curve (x, f (x, w0)), x [0, 1] will then be a fit to the points (xI, dI), i = 1, . . . , n. Plot the curve f (x, w0) as x ranges from 0 to 1 on top of the plot of points in (c). The fit should be a “good” fit.

  1. Your report should include a pseudocode of your training algorithm including all update equations written out explicitly (similar to what I had done in the second question of your second homework). The pseu-docode should be written in such a way that anyone would be able to implement your algorithm without knowing anything about neural networks.

1

Homework 4 Solution
$30.00 $24.00