Recommendation Engines
A Recommendation Engine is a system that provides personalized product and content recommendations to users. Such systems are driven by Machine Learning algorithms and utilize data that contain prior knowledge about users’ known interests and interactions. Three of the main approaches for Recommendation Engines are:- Content based filtering: This approach recommends products similar to what a user has viewed or liked in the past. Since the recommendations are user specific, they can easily scale up to support a large number of users. On the other hand, these recommendations do not offer the users the opportunity to discover products different to what they like so far.
- Collaborative filtering: This approach is based on the assumption that two different users who liked the same products in the past will probably like the same ones now and in the future. Such models recommend products liked or viewed by a user with similar interests. A great advantage of this method is that it does not require domain knowledge about the products and also gives users the chance to explore products that they explored before. However, such models cannot recommend products newly added to a company’s catalog and require frequent retraining in order to include them in their recommendation lists.
- Hybrid: This approach combines both content based and collaborative filtering and adapts the recommendations.
Requirements
Which approach should a company choose though? Before deciding, it is important to assess the concept of Recommendation Engines from three main perspectives:Business Objective: What shall be improved?
Identifying the business objective is the first prerequisite that should be fulfilled in order to make sure that the recommendations will ultimately achieve the desired outcome for a business. For example if the objective is to increase the revenue of a wide variety of products, then a Recommendation Engine could be optimized to suggest to users more diverse products . On the other hand, if the objective is to improve the user experience, the recommendations could be based for example on products that fit well together or products that are liked or viewed by users with similar interests.Serving: Touchpoints and Timing
Another important aspect is to understand how a Recommendation Engine is intended to be used. For example, will the recommendations be presented to users in real time as soon as they visit a website, an app or even a store in person? Or will the strategy be instead to serve recommendations periodically via an email, display and video advertising campaign? Will the users interact with it for example by giving feedback about the recommendations? Such decisions will definitely have an impact not only on the Machine Learning method but also on the overall architecture of the Recommendation Engine and thus, they should be addressed at a very early stage of the development phase.Data Assessment
Developing a Recommendation Engine that is based on Machine Learning techniques requires large volumes of historical user and product data. Such data is fed to Machine Learning models to optimize their ability to make predictions on recommendations. As a result of this, poor quality data can lead to poor recommendations (concept known in machine learning as “Garbage in, garbage out”). A good practise is to analyze the available data and assess whether it is relevant and sufficient for building a Recommendation Engine. Knowing for example if the product catalog contains many items, how often the product catalog data is updated as well as if there is data capturing users feedback, will definately simplify the decision making process. Even if the data is available, having the right capabilities to identify the business needs (Business Understanding) as well as to process the data and implement the solution (Cloud & Data Science) is the key to delivering a Recommendation Engine that will eventually create value for the company and its customers.
Google Cloud Platform Solutions for Recommendation Engines
Currently, Google Cloud Platform offers four possible ways to leverage state of the art Machine Learning techniques and deploy Recommendation Engines in the cloud.- BigQuery ML
- Recommendations AI
- AI Platform
- Self built & customized with Tensorflow or other similar programming frameworks
Using BigQuery ML for Recommendations
BigQuery ML is a great tool for low cost and fast implementations of Recommendation Engines, however currently supports only one specific recommendation type ( recommends to users only products which are liked or viewed by other users with similar interests). The models are created using SQL and can be accessed in BigQuery via a Web User Interface (Rest API, command line tools and external tools such as jupyter notebooks are also supported). When it comes to performance tests (such as A/B testing), a separate implementation needs to be integrated with this solution.Recommendations AI – Reco AI
AI Platform
AI Platform is a development platform which allows quick and cost-efficiently deployment of ML projects. It definitely requires a certain level of ML expertise and domain knowledge for preparing, building, validating and deploying ML models. When it comes to Recommendation Engines, AI platform offers the possibility to use one of its built-in algorithms (Wide & Deep) which has been used in the past for increasing app acquisitions in Google Play.Self built & customized Recommendations Engine
Self Customized Recommendation Engines on the other hand offer the most flexibility but require ML, domain expertise as well as higher development and maintenance efforts. Below you can see a quick summary of what each solution can offer:| BigQuery ML | Recommendations AI | AI Platform | Self Built & Customized | |
| Recommendation Type | “Products liked by users with similar interests” | “Other products you may like” | “Most relevant products” | Depends on the selected method, for example: “Similar products”, “Products liked by users with similar interests” |
| “Frequently bought together” | ||||
| “Recommended for you” | ||||
| “Recently viewed” | ||||
| Method | Matrix factorization | Not disclosed | Wide & Deep | Hybrid |
| Data Source | Product ids | Product catalog | Product catalog | Product catalog User events |
| User ids | User events from a wide range of data sources | User events from a wide range of data sources | User events from a wide range of data sources | |
| Implicit or explicit feedback | Optionally other relevant data sources | Optionally other relevant data sources | Optionally other relevant data sources | |
| Domain Knowledge | Low | Low | High | High |
| Development Effort | Medium | Low | Medium | High |
| Runtime Cost | Medium | High | Medium | Low |
| Configuration UI | No | Yes | No | No |
| Customization | Low | Low | Medium | High |
| Performance Measuring | Can be customized | Incorporated in the UI | Can be customized | Can be customized |
| A/B testing | Can be customized | Implicit | Can be customized | Can be customized |
Are ready made Recommendation Engines from well known suppliers the right solution?
When searching for Recommendation Engines you will find a plethora of well known vendors offering out of the box software. Even though many of them offer great solutions there are several points that one must be aware of:- Black box solutions: Most solutions are leveraging data and machine learning techniques to develop the recommendation models. Such models are considered “black box” as it is not possible to interpret exactly how variables are combined with each other to make predictions.
- Customization: Ready made solutions can still provide the possibility to customize the Recommendation Engine according to your needs. For example, to
- serve recommendations across multiple touchpoints and channels
- incorporate additional data sources (eg. weather data) that might be relevant for your business
- develop models depending on specific objectives (maximize revenue, click-through rate etc.)
- apply specific business rules (eg. stock availability)
- Compliance: Solutions have to comply according to GDPR standards.
- Cost & Return of Investment (ROI): The relatively high cost of paying for an out of the box service can easily pay off in a very short period of time if the set up is done according to the desired business needs and objectives.
