Monday, February 23, 2009

Dace in Netflix

Today is the last day of my paternity leave. Before going back to work, I guess it's time to log the fun I have had with the Netflix challenge.

I began to study the problem when I got a break between projects in late October 2008. After reading papers from the leading teams (many thanks to BellKor and Gravity for their papers), I realized it was such an interesting and well-defined problem. Since the college days, I have always wanted to work on Artificial Intelligence, but ended up living with VLSI designs everyday. Now here is a good chance to satisfy my itch and learn from experts in this area. Besides, the remote chance of one million dollars surely does not hurt.

This actually became a perfect task after my son was born. The competition is mostly about algorithms, yet needs significant amount of run time to produce results. So I can do the thinking while holding the baby in my arms and mumbling lullabies. Find a break to code something, kick off a run, only need to come back hours later to check the results. And when I am watching baby gradually learn to interact with the world, I couldn't help thinking where machine learning can lead us to.

There is a big difference though. While my son keeps growing everyday, the progress on the Netflix prize seems to reach an asymptotic stop. As of today, my score is 9.52% improvement. Starting from tomorrow, there will be less time for me to spend on the problem, but I will still work on it. There are still a few ideas to explore and it's fun!