Ridge regression was originally developed to deal with the problem of having more feature than data point.
It can also be used to add bias into our estimations, giving better estimate.
Shrinkage methods allow us to throw out unimportant parameters so that we can get a better feel and understanding of the data. It can also generate better prediction value than linear regression.
def ridgeRegression(mat1,mat2,lamb=0.2):
trans1 = mat1.T*mat1
deno = mat1 + eye(shape(mat1)[1])*lamb
if linalg.det(deno) == 0.0:
print "singular matrix, cannot do inverse"
return
w=deno.I*(mat1.T*mat2)
return w
This function, implements ridge regression for any given value of lambda. Default value is 0.2.
def test (arr1,arr2):
mat1 = mat(arr1)
mat2 = mat(arr2).T
mean1 = mean(mat2,0)
mat2 = mat2-mean1
means2 = mean(mat1,0)
var1 = var(mat1,0)
mat1 = (mat1-means2)/var1
testPoints = 20
wmat = zeroes((testPoints,shape(mat1)[1]))
for i in range(testPoints):
w = ridgeRegression(mat1,mat2,exp(i-10))
wmat[i,:] = w.T
return wmat
This tests over a number of lambda values.
Hope it helps !
Keep Learning ๐