The Age of Recommendations

With distributed computing improvements and given the possibility of handling big inventory catalogues, the “Age of Search” presents its complement called: the “Age of Recommendations”, which we are already in and using it all the time. Think of these examples: when you are offered the next movie to watch (Netflix), when you get friends recommendations (Facebook), when you search for a page (Google), and when you are suggested what your next book should be (Amazon), among plenty of examples in e-commerce sites and social networks.

Businesses challenges for getting more sales

It’s a fact that businesses discovered the value for increasing sales in better understanding the behavior of each customer and improving the recommendations they offer in a personalized way. Amazon claims to perceive 35% of sales coming from the recommendations done after you bought something. And, knowing the behavior is not only knowing the purchased list. It’s also collecting ratings of preferences and likes. That’s a big challenge since not all users rate their purchases or have the same “baseline” when rating goods. Hence, businesses also need to extrapolate unknown ratings of their products from the known ones, in order to recommend things you’d probably have not searched for (AKA, the age of search complement) and measure the effectiveness of that whole dynamic process. Yes, it’s a dynamic process, because your users or customers are not static and the products being offered by every shop aren’t either.

Recommender Paradigms

Nowadays, there are basically three paradigms: “content or item-based”, “collaborative or user-based” recommendations and a third being a mix of both.

Content or Item-Based Recommendations

The first paradigm is to try to get the advantage of using the similarity between items or some common attribute to make predictions; for example, in movies, using the same director; in books, using the same genre or topic. If you watched Avatar, another James Cameron movie could be a good candidate for your interests.

The advantages of the first paradigm are its user independence since the ratings are on the items themselves and always provides transparency to unveil the relevant attribute for which it’s been picked as a recommendation. For new items, they can be recommended right away from the beginning even if they have zero ratings, just because of its own attributes. From a computational complexity perspective, it requires only minor effort. But, at the same time, the biggest advantage of this approach has the issue that not necessarily similar attributes of the items reflect how users decide.

Collaborative or User-Based Recommendations

In the second paradigm, the focus is to identify users whose preferences are similar to the given user (called neighbors) and recommend items those neighbors have liked. This might include also making extrapolations of other user ratings and comparing those with current user history and preferences. Here, the inventory catalog can be fully recommended since products from different categories can be recommended based on user preferences. Which, in fact, can be measured and is named Recall.

This paradigm does not come for free and encounters some known problems like:

• Many people must participate: no nearest “neighbors” are found in too-sparse matrices.
• New user problem: lots of ratings must be collected before we can produce decent recommendations for a new user.
• New item problem: many users must rate new items before they can be recommended.
• Grey sheep problem: users with unusual taste do not get accurate recommendations (I confess I belong to that set)
• Scalability problem: the computational complexity associated needs a lot of resources.

To sum up, there’s always a balance between expected recommendations accuracy and computing efforts, and that will dictate the paradigm(s) to adopt.

Is it only for websites?

I recently reread a citation in a classical paper written by Martin, Donaldson, and Ashenfelter, that I believe is still accurate: “[W]hen we evaluate the current generation of recommender systems from the point of view of the recommendee, we find that most recommender systems serve the goals of the business instead of their users’ interests. Thus we believe that the big promise of recommender systems has yet to be fulfilled.”

Since then, a lot of progress has been seen in this field like systems starting to consider the goals and activities of users in addition to their preferences and needs. It’s been conceptually englobed as the third wave of recommender systems that will be capable of learning about user’s contextual behaviors and preferences to anticipate user’s actions at any point in time and act upon them. Have you recently tried Apple’s Siri or Samsung’s Bixby personal assistants? You will easily relate that a bit of it it’s under the hood in that AI.

Final words

This blog post does not expect you after reading it to comprehend the algorithms, processes and historical evolution of these type of systems, but you should at least be able to start identifying them and learn a bit about some of the challenges and future uses they will get into.