Monday, November 19, 2007

The Netflix Prize: Netflix Dangles a One Million Dollar Carrot For Technology Innovation

Netflix is trying to maintain its good user experience advantage over their competition by offering a $1 million prize to improve their current recommendation system. Take a look at some of the jargon in their rules that you have to follow:

As of the start of the Contest, the RMSE of Cinematch on the quiz subset, based on training the Cinematch algorithm using the training data set alone, was 0.9514. The RMSE of Cinematch on the test subset, based on training the Cinematch algorithm using the training data set alone, was 0.9525.

To qualify for the Grand Prize the RMSE of a Participant’s submitted predictions on the test subset must be less than or equal to 90% of 0.9525, or 0.8572 (the "qualifying RMSE"). After three (3) months have elapsed from the start of the Contest, when the RMSE of a submitted prediction set on the quiz subset improves beyond the qualifying RMSE an electronic announcement will inform all registered Participants that they have thirty (30) days to submit additional candidate prediction sets to be considered for judging.

At the end of this period, qualifying submissions will be judged (see Judging below) in order of the largest improvement over the qualifying RMSE on the test subset. In the case of tied RMSE values on the test subsets, the submission received earliest by the Site will be judged first. If no qualifying submission can be verified or no Prize can be awarded, the Contest will reopen and new qualifying submissions will be considered according to the protocol described above. The decisions of the Contest judges are final.

I ask the same question of Netflix that I asked of NASA and the Department of Defense: Why can't you put this money towards an internal process? Is this cheaper? Is it impossible to innovate internally? Does this version of sequestering folks to innovate lead to a better change of discovery? I'm very curious about this.

No comments: