Description
Questions
Please finish the programming part first.
1. RBF Kernel (2 points) As discussed in class, the RBF kernel is defined as
-
-
-
-
-
-
-
K(x, x′) = e−γ∥x−x′ ∥22
(1)
-
-
-
-
-
-
Hopefully, from the programming part, you have already gotten a sense about how the hyper-parameter γ impact the model performance. Based on your observation and Equation 1, please give an intuitive explanation about how γ could impact model complexity. Your answer should cover
-
-
Whether higher or lower values leads to more flexible models, and
-
-
-
Why?
-
-
Polynomial Kernels (3 points) In our lecture on kernel methods, we show that a special case of the polynomial kernels
-
-
-
-
-
-
-
K(x, x′) = ( x, x′ + c)d
(2)
-
-
-
-
-
-
with d = 2 and x, x′ ∈ R2. On our lecture slides, we show how this special case can be decomposed as a dot product with a nonlinear mapping Φ(·)
-
-
-
-
-
-
K(x, x′) = Φ(x), Φ(x′) .
(3)
-
-
-
-
-
In this problem, consider d = 3 with x, x′ ∈ R2 and show how the Φ(x) is defined in this case. Note that, before splitting the kernel function to be a dot product of two high-dimensional vectors, make sure merge the same items as much as you can, as we demonstrated in class.
1
