Magazine article AI Magazine

A Short History of the RecSys Challenge

Magazine article AI Magazine

A Short History of the RecSys Challenge

Article excerpt

The first modern research paper on recommender systems (Resnick et al. 1994) was published in 1994 by Resnick, Iacovou, Suchak, Bergstrom, and Riedl. More than two decades have passed since, and the recommender systems research field has grown rapidly, and along with it a research community which, nowadays, is centered on the ACM Conference on Recommender Systems.1 Ever since Resnick and colleagues published their paper, recommender systems have continuously adapted to current and upcoming trends and technologies. The recommendation approach proposed by Resnick and colleagues was revolutionary when it was introduced, utilizing collaborative filtering to predict scores that users would give to newsgroup articles in order to identify the most relevant articles, that is, those with the highest scores. Today, even though similar approaches are in use, they are usually just one part of complex recommendation approaches that can include large collections of algorithms and data sources.

In 2006, one initiative, the Netflix Prize,2 created a focus on recommender systems and contributed to major advancements in the field during its three-year run. Similar initiatives have led to great improvements in related fields, for example, the Text Retrieval Conference3 in information retrieval, and the KDD Cup4 in data mining.

Following the success of these, the RecSys Challenge5 is a yearly competition organized in conjunction with the ACM Conference on Recommender Systems.

A Brief History

The Netflix Prize was launched in 2006, the same year that the summer school on Recommender Systems was organized. By 2007, the Netflix Prize had attracted thousands of participating teams, and the first ACM Conference on Recommender Systems6 was held. At the 2009 ACM RecSys conference, the Netflix Prize concluded. During the duration of the prize, significant advancements had been made in the recommender systems research field, for example, establishing matrix factorization methods such as SVD as state of the art in recommendation. At the 2010 ACM RecSys conference, the seed for what would become the RecSys Challenge was organized as the Challenge on Context-aware Movie Recommendation (CAMRa). CAMRa attracted a moderate number of participants, but contributed to establishing the RecSys Challenge series.

Challenge Structure, Yearly Overviews, and Future Trends

The RecSys Challenge has followed a similar structure since its inception: (1) a data set and problem are presented, (2) teams sign up and participate, (3) participants submit their solutions in time for a deadline, (4) participants submit papers outlining their approaches, (5) during a workshop at the ACM RecSys conference participants present their approaches and winners are announced. Only a selected number of teams present their work at the workshop due to time and space restrictions. One of the main features of the challenge is to make available a new real-world data set.

The first challenge to be held in conjunction with ACM RecSys, CAMRa (Adomavicius et al. 2010), focused on recommending movies in specific contexts, that is, during the Oscars and during Christmas, for a specific mood, and based on social ties between users. The challenge was organized as a collaboration of the Technische Universität Berlin (TU Berlin) and Moviepilot GmbH. Several data sets with various contextual dimensions were released for the challenge and 40 international teams participated.

In 2011, the second iteration of CAMRa was organized again by Moviepilot and the Technische Universität Berlin, focusing on contextual movie recommendations (Said et al. 2011) in the context of group recommendation. This time Moviepilot requested that the users of their service share information about who they shared a household with in order to accurately predict recommendations for groups of users. The number of participating teams grew to a total of 45.

The 2012 challenge was the first to focus on something else than movie recommendation (Manouselis et al. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.