Quantifind Interview Question

What is overfiting?

Interview Answer

Anonymous

Jun 6, 2015

I explained it with some scattered points in a 2d graph. So we want to predict Y as a function of X. We propose the following model Y=a0+a1*x+a2*x^2+...+an*x^n where n is the degree of the polynomial. If n is very large we will fit perfectly the points in the graph but if we apply the model to a new dataset we will find large prediction errors because we had been fitting the noise. Conversely if n is small perhaps the model is not too complex to describe the trends of the data and our predictions will be biased. This tradeof is called the bias-variance tradeof. Find full explanation in the book "Introduction to Statistical Learning"