By devtodev analysts, Vera Karpova and Vasiliy Sabirov
Currently, product analytics reached a sufficiently high level of development. Many analytical systems are equipped with a variety of tools that will tell in detail how users behave in the application: when they buy, where they live, how much they cost for the company and how they leave.
These tools have become a part of daily life, regular monitoring; assistants in the decision-making process – now it is a must-have for any project.
Funnels and segments don’t surprise anybody anymore, and as in any other business, having reached the top of one reveals a will to go further and improve.
In this regard, the sphere of analytics is no exception, and in the past few years a new kind of data analysis – predictive analytics – began to develop.
You’ll also have an idea of predictive analytics, if you monitor the metrics on a daily or even hourly basis.
For example, you know that usually at 12 a.m. there are about 20,000 users in your game, and today this indicator is much lower. It equals 15,000 users. You understand that there is a trend for decline, which means that it is necessary to find the cause as soon as possible and improve the situation before the indicator falls even more.
And, looking at the chart, you assume that the difference with yesterday’s indicator will be even greater in one hour, although this has not happened yet.
Predictive Analytics is not guess-work, but it is the same forecasting process which is based on a large amount of user data collected over a long period, and more complex and accurate calculations.
How the Forecasting Process Takes Place
The first stage is data collection. This is a usual tracking of events in the application, and most often it is already implemented, as it is used to calculate and control the basic metrics, a/b tests, segmentation and product analysis. The model will be created on the basis of this data over a long period.
The next stage is to define the parameters that are more likely to affect the predicted event, and with the help of machine learning algorithms, to detect the patterns. This also eliminates the “unnecessary” events and emissions.
This is the most difficult stage, because the more different data exists in the application (i.e. speed and number of passed levels, amount of game currency, level of the player, method of registration, number of days in the game, number of purchases, time of the first purchase, etc), the more difficult it is to determine what parameters could affect the final event.
It is important not to overdo it with eliminating: the less factors are selected to predict a certain event, the lower is accuracy of the resulting model.
Then, on the basis of historical data and previously collected patterns the predictive model is created, which indicates the possibility of an event for a specific user.
The result of all steps described above (as we remember that every analytics is needed to draw conclusions) could be the following statement:
A player from the United States with the iPhone 7, who installed the game two days ago, made one purchase of $3 USD and passed four levels, is likely leave the game in eight days.
Then, it is necessary to determine accuracy of the resulting model. An indicator of accuracy is the coefficient of Prediction Accuracy, which it is calculated as the ratio of predicted values to actual values. Accuracy that’s equal to one is an indication of an ideal model, and accuracy equal to zero, alas – is a random guess.
To measure these indicators, one needs to select a group of users, make a forecast for them and observe how the actual behavior of this segment coincides with the forecast that has been made.
If the reality goes along with the forecast, or at least coincides with it by 95%, then the model can be applied to new data; if not, it is worth once again to review and adjust the factors affecting the predicted event.
When you are satisfied with the accuracy of the model, you can start interacting with the users, changing their behavior on the basis of the received data.
An important plus of forecasting is that you know the result in advance, a few days before the target event happens. This means you have time to retain the user in the game, to encourage them to make a purchase or engage more in the gameplay.
NB: When building models, do not forget about the segmentation, because in a single group of users (for example, the “newcomers – just installed the app”), there are always people with different social and demographic characteristics and different behavior.
What kind of Tasks Can Be Solved by Predictive Analytics?
The goal of any company is to keep the player happy and retain them as long as possible in the project. And predictive analytics helps achieve it: the result of usage of this model should be the increase of user’s LTV.
Thus, at the moment, the most popular metric for predicting is the churn rate. It has a direct impact on revenue – the longer the user stays in the game, the more revenue they will bring.
Churn rate forecast may show:
- How many sessions will be performed by the player before leaving the game
- How many users there are and probability of whether they’ll leave the game
- What actions are performed in the game, by those who are planning to stay and those who are likely to leave
So, for this model you need to determine which indicators may affect the player’s leaving of the application. For example:
- The number of user sessions in the game, and their frequency
- Duration of sessions
- Speed of passage of the game (the number of completed levels, missions)
- When the first purchase with respect to the first session was completed
- The number of all payments made by the player
- Social activities (eg, participation in game chat)
- Number of days until the end of subscription
- The level the player achieved from the first session
Metrics that affect the event of interest and their number can be individual for each game.
Once you have identified the factors influencing the result, built an accurate model on their basis and found out when users are going to leave the game, you may start experimenting in order to improve the predicted metric.
What can be done:
- Offer a bonus to the players
- Provide access to otherwise inaccessible features
- Send a targeted push notification or email
- Facilitate the passage of a certain level
- Change something in the user’s path, that is – influence the events that affect the players’ churn.
The next important metric-affecting revenue is conversion into purchase, especially into the first purchase.
The following factors may affect it:
- User-training or tutorial
- Passing the first level
- The moment of being introduced to the purchase
- Demographic characteristics of the player
- Technical characteristics of the device
- Game design
By creating a model, you will define:
- What makes users start paying
- Who is not ever going to buy anything
- Users with what attributes are more likely to make a purchase (with what device, OS, from which country)
And as a result, you will be able to compare people who are likely to make a purchase, and those who are not going to buy anything.
To improve the conversion rate you can:
- Offer a discount or welcome bonus for the players
- Change the time of being introduced to the purchase
- Experiment with design of the gameplay where the purchase is proposed
The degree of engagement of the user also affects LTV. In other words, this is the same conversion, only into an action which is important to “hook” the user on the game and increase the sticky-factor.
Everything is quite individual here. For one game this indicator could be the first level passed, for another it could be conversion into registration or a certain number of points. For a task managing application it could be creation of a task, for a dictionary – the found word, for a messenger – a contact added and messages sent. In this case, you can build a model that will show which users tend to be converted into the targeted action.
Accordingly, methods of exposure to the audience in order to increase the metric will be different.
If there is advertising inside your app, it is possible to create a model to determine the best moment to show ads, a model to segment users by the likelihood of their response to it. Also, the model allows us to calculate the most appropriate time to display the ad.
In general, knowing how a particular user will behave in the game, you can change their way by improving the satisfaction with the game and, consequently, affect LTV and increase revenue.
A Few Examples of the Predictive Analytics Implementation
Developers from the Innova company successfully created a model for predicting the churn of players (for two types of users).
Prediction of the churn of “newbies” was built on the basis of their actions in the first couple of days, and the fact that the user is likely to leave the game they were already playing for a few months was known to the developers 2-3 weeks in advance before it actually happened – through analyzing the various actions of users and their impact on the result.
The Electronic Arts company uses predictive analytics to find “weak” spots in the game, predict players’ churn and find out if there is a demand for this or that feature after the release, based on the daily collecting of 50 TB of data.
In Alien Child game machine learning is used in a slightly different way: with its help the main character of the game has the dialogue with the users, accumulating data and improving responses.
Predictive Analytics in devtodev
We, at devtodev, understand the importance of predictive analysis and have already launched a mechanism for LTV prediction based on payments in the first days of using the product. You will be able to predict how much money one player will bring for all time spent in the game, and plan your future activities on this basis:
- Limit the price of traffic purchase
- Calculate your return on investment (ROI) in acquisition
- Compare the quality of different traffic sources with each other
- Plan variable and fixed costs of user acquisition
- Monitor qualitative changes in your product (LTV is the convolution of many indicators of quality of the project in one metric, and LTV like no other indicator is useful in measuring the dynamics of product quality)
devtodev successfully builds patterns of user behavior. For example, you can answer the following questions about your users:
- On what day and at what level they make their first, second, and third payment
- The level where they are converted into paying users/make repeated payments
- How much they pay in their first/second/subsequent purchases
- What they buy at each level within the game for real or virtual currency
- How they move in the game from location to location, and where they have problems, etc.
Knowing these patterns, you can apply changes to your game, making it more convenient for users, and therefore increasing their loyalty. Suppose you determine that on the second day of the game at level six users make their first payment, buying a chest of stars. If the user who comes to this place is getting a push-notification with an offer of a discount for that chest of stars, this user is more likely to respond to your offer, and you are likely to increase the conversion into paying users.
We are working hard to make analytics as clear for the user as possible and to make the transition from data to making a decision as short as possible. In addition to the above, devtodev is developing the prediction mechanism of players’ churn and their payment conversion, a support system of decision-making on planned updates, and lots more interesting services.
Features of Predictive Analytics
Despite the fact that the predictive model can accurately forecast the behavior of users, this approach has a few features you should consider when using it:
Firstly, it is impossible to foresee all the external factors and events that may occur in the future and impact your users and their behavior. For example, iTunes does index prices for a particular country, and people’s purchasing activity will change, or there can be a dramatic change in political situation.
Secondly, the impact on user behavior can not only come from economic and external events, but also from your own experiments. So you need to keep in mind that if you are constantly experimenting, any tests, especially related to prices, currencies and products, can greatly affect the behavior of the players, which means the prediction created on the basis of old data is no longer relevant.
What to Expect from Predictive Analytics
Today the most important task, when creating a predictive model, is to determine the parameters that most strongly influence the event you’re predicting. And it’s best to have it done by a person with experience in the subject, and even better – in a specific product. Only a specialist who knows “their” user can adjust the preset parameters to improve prediction accuracy.
An important role in machine predicting is still played by the human factor. So the next expected step for predictive analytics should be a greater degree of automation – machine algorithms themselves should determine the parameters affecting the result and identify what event in users’ path should be changed in order to qualitatively change the resulting event.
Another expected trend of predictive analytics, according to experts, is the ability to personalize the game by choosing the most optimal parameters of the gameplay (difficulty level, awarding systems, locations, etc.) and monetization for each player. It is already possible to collect any data about user behavior in the game, and each interaction with them can either increase or decrease the revenue, either improve or impair their subsequent behavior. So, in the long run, machine learning can lead to the fact that the game itself will adapt to the particular player, adjusting its path and maximizing revenue.
Now companies are trying to focus not only on what the users were doing in the past, but what they are going to do in the future. And predictive analytics is a new tool that will allow developers to not only predict the future, but also control and adjust it.
The past and present are our means: the future alone is our goal. – Blaise Pascal
devtodev is an intelligible game analytics system that helps identify problems in an app’s performance, find the points of growth and make the most effective decisions.