Monday, February 23, 2009

Dace in Netflix

Today is the last day of my paternity leave. Before going back to work, I guess it's time to log the fun I have had with the Netflix challenge.

I began to study the problem when I got a break between projects in late October 2008. After reading papers from the leading teams (many thanks to BellKor and Gravity for their papers), I realized it was such an interesting and well-defined problem. Since the college days, I have always wanted to work on Artificial Intelligence, but ended up living with VLSI designs everyday. Now here is a good chance to satisfy my itch and learn from experts in this area. Besides, the remote chance of one million dollars surely does not hurt.

This actually became a perfect task after my son was born. The competition is mostly about algorithms, yet needs significant amount of run time to produce results. So I can do the thinking while holding the baby in my arms and mumbling lullabies. Find a break to code something, kick off a run, only need to come back hours later to check the results. And when I am watching baby gradually learn to interact with the world, I couldn't help thinking where machine learning can lead us to.

There is a big difference though. While my son keeps growing everyday, the progress on the Netflix prize seems to reach an asymptotic stop. As of today, my score is 9.52% improvement. Starting from tomorrow, there will be less time for me to spend on the problem, but I will still work on it. There are still a few ideas to explore and it's fun!

13 comments:

  1. Any revelations on the tyep of algorithm you're using?

    Are you combining several things together, or is there any major algorithm you're using?

    ReplyDelete
  2. I am using an ensemble of KNN, Matrix factorization/SVD, Neural networks/RBM, global effects etc.
    They are pretty much the same as described in the papers, just with some small tweaks.

    ReplyDelete
  3. Could you email me at jrkjared at aol dot com? I have a business proposal for you. I am a current MBA student that could use your skills to build a business.

    Thank you,
    Jared

    ReplyDelete
  4. One against so many experts....I'm pulling for you....Enjoy the baby, they grow up fast....

    Joe

    ReplyDelete
  5. are you by any chance from latvia?

    ReplyDelete
  6. Hi Dace,
    congrats for excellent results so far. I'd like to discuss some ideas with you, please drop a mail to rene0 at finantic dot de if you are interested. I am physicist and work heavily with data mining algorithms in the financial realm.
    ciao, René

    ReplyDelete
  7. Dace,

    Seems like you are getting a bunch of these inquiries, but would love to talk to you about a company I have founded. I'm a successful entrepreneur already, but have a great idea that you could potentially be a part of. Ping me if you'd like to chat. -Dave

    ReplyDelete
  8. Dace,

    Why are you not teaming up with the rest?

    ReplyDelete
  9. Hi Dace,

    I am a student, Can you pls tell me how we can check th error on qualifying set? I think after the completion of the Netflix prize, we do not have any support for it?

    Thank you

    ReplyDelete
  10. Dancing is the most vibrant and beautiful form of art. Besides this, it's a great way of social interaction and provides a fun workout, which increases flexibility...

    Geneva dance

    ReplyDelete
  11. Thanks for sharing this great article! I feel strongly about it and love learning more on this topic. It is extremely helpful for me. I hope you post again soon. jeremy larson deluxe marketing inc

    ReplyDelete
    Replies
    1. I like your stuff, classic right, Now... everything with digital, perdita di peso thanks for remind me for this memories

      women need need exercise specially more 40 age, because their hormone
      upper body exercises for woman for your body lower lower body exercises for women and abdominal too abdominal muscle exercises for woman this special therapy if you do not have a time to exercises testosterone replacement therapy

      Delete
  12. maticpress Review
    avatar genie pro Review
    expert fame Review
    I don't know how the crossing at Old Street ended up being a shared use crossing even though there's a seperate pedestrian crossing right next to it. And the phasing of the signals is terrible - trapping us on the island whilst we breathe in the fumes of passing motor traffic.

    ReplyDelete