A Comprehensive Analysis of the Tools that Support Mobile Game Development (Part 2)

main

Contributions Online PR & Marketing

A Comprehensive Analysis of the Tools that Support Mobile Game Development (Part 2)

September 10, 2014 — by Industry Contributions

A managing partner of Bitfold Online Games, Mike Turner knows his way around the design and development of mobile and social games. He also plays the role of analyst at times. He provides a guide to tools that can benefit mobile game developers in this two-part article series.

This article series aims to clarify what useful tools and services exist for each lifecycle step and provide a framework for evaluating their usefulness to your product. When talking about game operations tools, it’s helpful to segment them by stage in the player’s lifecycle they address:

1. System Management Tools: Keep game servers and clients healthy
2. User Acquisition Tools: Get new users into your game
3. Behavioral Analytics: Understand users and their desires
4. Engagement and Retention Tools: Keep users engaged for longer
5. Monetization Tools: Boost the number of paid conversions & spend per user

In this second of two articles, we will be looking at the remaining three of the five segments of game operations tools: behavioral analytics, engagement and retention tools, and monetization tools. If you missed the first part of this series, you can catch up here.

Behavioral Analytics

Behavioral analytics are critical. Everyone knows this. What everyone doesn’t know is what data they should be tracking, what tools they should be using to do so, and what to do with that data once they have it. This section will try to shed some light on what data is most important, how you should be thinking about using that data to manage your game into a financial success, and what tools will be the most effective in helping you do that.

Deeply Understand the Different User Cohorts

Different users will respond very differently to the content and features of your games. They will also have different lifetimes and behaviors in the game. Successful developers work hard early on to determine how users should be cohorted based upon their usage patterns, demographic, and traffic source, and then they carefully manage each of these cohorts to maximize their experience and positive behaviors in your game (such as social engagement, lifetime, and spending).

Heartbeat vs. Actionable KPIs – Actionable KPIs are More Important

Heartbeat KPIs are things such as DAU and ARPU that tell you about the general health of your game. They do not give you deep insights into behavior, however. You need to establish KPIs, specific to your game, which help you understand what your players retain and why, what features and content they engage with, and when your players monetize.

As you define what’s important, you often need to dig deeply into your data to find out what’s happening.

Experiment. A lot. Use A/B testing
Test fixes, content, and new features constantly, and test them against control groups. A lot of your guesses as to what will help improve user behavior will actually be wrong, but some will lead to significant improvements in your numbers. A/B testing will help you a lot in your experiments.

Do More than React. Model and Predict
Don’t just release and test. Predict. After a while, you will have enough data to establish trends and create mathematical models that predict user responses to specific content and changes.

Engage, Engage, Engage

What to look for in tools:

Actively engage each specific cohort of users with content that they love and offers they would like. It increases their engagement and maximizes their spending.

A Comparison of Tools

The following is a list of analytics tools that are well suited to online game development. Each of their offerings are slightly different, so we recommend first determining the needs of your game and reaching out to them to get the details of their offerings.

These tools are extremely helpful to your analysis. For most companies, it would take longer than is possible to create an in-house metrics solution that mimics their functionality. However, all games are unique, and none of these tools will measure EVERYTHING you need to measure.

When first launching your game and throughout its early lifetime, these tools will be more than sufficient. However, if your game becomes a huge success, you’ll want to supplement these tools with your own analysis tools that create custom metrics and analyses that these tools can’t. This will help you ensure you have the absolute best idea of what your players want and how to please them.

Finally, pair your behavioral analytics with good system metrics in order to avoid system downtime hurting your KPIs. DeltaDNA, one of the leading gaming analytics packages, cites technical issues as a top reason for users failing to engage with an app. This implies that although many game developers may be doing a good job understanding and serving users, they may not be managing their system problems as well as they could. And it’s hurting their revenue.

To avoid technical issues damaging your game’s numbers, you want to ensure that in addition to having excellent behavioral analytics, your operations team is equipped with proper logging and server monitoring tools. This helps ensure your system remains as error free as possible.

Engagement and Retention Tools

User engagement can (roughly) be boiled down to the following components:

Given that your game design is engaging, behavioral metrics packages are your primary tool for understanding your users and knowing how to engage them. There are, however, a few extra tools that act as supplements to your ability to engage users.

Optimized Player Segmentation and Targeting

Creating player segments and deciding what features and content suit them best is challenging. You can use simple observation of your metrics to determine this, but there are some statistical tools that can greatly improve your predictive ability. Honeylizer is one of the best tools for this and will help you determine how players should be segmented and what the best content is to serve to those segments.

Social Engagement - Integration with Established Social Networks

People like playing with their friends. In a game, if they have the option to play the game with friends, they will often do so. You can create this integration yourself with Facebook Graph’s and iOS Game Center, and if you have the resources, you should try this.

However, the Game Center and Facebook Graph API are fairly complex and change all the time. This means your app’s social integration can break constantly. If you’d prefer to outsource the management of this, you can choose third-party packages that make integration and maintenance of social functionality easy.

Multiplayer Facilitation

Adding social networking and multiplayer elements to your game can often grow your engagement. A few tools provide libraries and services to you, which help you integrate with social networks fast and provide multiplayer functionality to your game.

Both of these packages offer social network integration. For multiplayer functionality, Swarm focuses more on leaderboards and achievements, while Nextpeer focuses on facilitating peer-to-peer multiplayer functionality within your core gameplay.

Customer Experience Management and Help Desks

As your game grows to tens and hundreds of thousands of users, you will often become flooded with support issues that, if unmanaged, can damage your online and app store reviews. Having a system to manage support issues will help your users feel like they’re being taken care of and help you better understand what users are qualitatively thinking.

Your customer support system should include the following:
● A wiki or set of support pages with issue FAQs and support information
● A ticket system for customers to report issues
● A web portal to respond to customer tickets
● Optional customer support outsourcing to help you manage inquiries

Vendors that provide such systems include the following:

For most games, an overwhelming amount of customer complaints are due to operational issues. If a large number of players are complaining about something, you can use logging tools to help you identify the problem and solve it immediately.

Monetization Tools

Games today are overwhelmingly free-to-play and monetized primarily via in-game purchases. However, ads can be a strong source of secondary income for a developer that implements them well.

Ad Publishing

Today, advertising providers offer a wide variety of options for apps and games. These include native ads, rewarded installs and actions, rewarded video, moment ads, rich media ads, and ad mediation and bidding. (More information on these options can be found in the first part of this article series).

There is a lot of variety in the amount of return these ads can give you and what each advertiser pays. Before integrating ads, you should look carefully at the rates that companies pay for each type of advertising.

Maximize Payouts, Minimize Annoyed Users

You want to maximize your ad impressions clicks while minimizing the annoyance of your users.

Some good rules of thumb in this process are:
● Use native ads to imbed them directly into the UI of your game so that they are a fluid part of the game’s experience and don’t disturb the player.
● Place moment ads in areas where players can get ahead by interacting with the ads.
● Offer rewarded ads at points where extra in-game currency will benefit the user.
● If using ad tools that offer mediation, use the mediation and real-time bidding tools the ad provider offers to get the most contextual content to your users. This will maximize your user’s interaction with it and help to minimize their annoyance.

Matrix of ad publishing service each network provides

Implementing in-app purchases can be somewhat tedious. If this is tripping you up, you can use SOOMLA to help you speed this process up.

When Should You Use Third-Party Game Operation Tools?

Let’s quickly recap the strategies for choosing tools for maximizing your game’s performance at each step of the customer lifecycle.

1. System Management Tools
Online games are put under an incredible amount of stress and things fail – a lot. To keep your system at optimal uptime, you should have good logging tools to detect and solve system issues quickly.

2. User Acquisition Tools
Today, there are a variety of advertising formats beyond mobile banner ads. If you don’t have a big advertising budget, work to get lots of organic traffic via social media, app store optimization, and direct deals with other developers through direct-deal platforms like those that Chartboost offers.

If you do have a decent marketing budget, work hard to design good native and rich media ads and place them using mediation tools with ad networks that have game-centric focuses. Continually fine-tune your campaigns until you find the best ads and the best networks.

3. Behavioral Analytics
Behavioral analytics are your primary tools for understanding who your users are, what they like, and how to serve them. In focusing on your users, you want to focus on actionable KPIs and insights instead of top-level ones like simple DAU and ARPDAU.

When searching for tools, you want to look for those that provide you the rigorous ability to segment users, define your own KPIs, track where your users came from, and data mine deep into your data for granular insights.

4. Engagement and Retention Tools
Retention and engagement is primarily a function of the developer’s ability to understand who users are and cater to their desires. However, there are tools out there that help you automate the process of classifying your users, tools that help you bring social functionality to the game, and tools that help you directly support customer issues with your games.

5. Monetization Tools

Microtransactions are the primary form of making money in a free-to-play game, but ads are a great secondary form of revenue.

Microtransactions are the primary form of making money in a free-to-play game, but ads are a great secondary form of revenue. The same options for advertising (listed above) are great for monetizing. The best way to optimize monetization via ads (ad publishing) is to make ads a seamless experience in your app and place them at points where interacting with ads is beneficial for your users. Make the same rigorous use of behavioral analytics you use elsewhere in your game to maximize your ad revenue!

Using a Decision Framework to Decide on Tool Usage

These tools are meant to automate key functions of game operations. However, they do require effort to integrate and they do cost money.

So when making the decision to use third-party tools, you want to ask a few questions:

● How crucial is the functionality the tool provides to your game? Does your game REALLY require it?
● What does your team say about it?
● How time consuming is it to integrate and maintain? Some are easier, some are more complex.
● Do the tools bring a greater revenue or cost savings than the cost of the tool?
● Do these easily work with your chosen game engine and technology platform?

Once you’ve run through this checklist with your team, you can make the decision!

Contributions Development Industry Online Research

Factoring System Performance Data into Game Analytics for Better Retention & Monetization

September 2, 2014 — by Industry Contributions

A managing partner of Bitfold Online Games, Mike Turner knows his way around the design and development of mobile and social games. He also plays the role of analyst at times. He looks into how system performance data matters for game analytics in this article.

The social and mobile gaming revolution in 2009 transformed the usage of analytics in gaming from something mainly used in MMO games to a universal best practice for all games. In subsequent years, the analytical methods adopted to analyze and affect user behavior have become very advanced. Today’s game analytics focus on:

Data mining: Executing specialized queries into behavioral data to get a granular picture of user actions and preferences.
User segmentation: Segmenting users into meaningful cohorts and maximizing positive behavior of each player segment.
Predictive modeling: Using statistical models to predict which changes will increase retention, strengthen engagement, and maximize paid conversions and user spending.
Custom KPIs: Establishing actionable performance indicators custom to each game (e.g., paid user conversion at specific key events, churn rate after particular engagement goals).

Successful developers today are experts at employing these advanced methods to gain deep insight into what their users like and providing content and features that keep them engaged.

Game companies don’t often pay enough attention to how the system performance affects their numbers. Skilled usage of behavioral analytics enables developers to create targeted content and features which keep their users engaged. However, mishandled system errors can cause significant disruption to user experience and properly managing them is a weak point of many developers. Technical errors such as bugs, unexpected downtime, and latency happen a lot, and they are typically not noticed until they manifest in a game’s KPIs.

DeltaDNA, a leading gaming analytics platforms, cites technical issues as one of the top barriers preventing players engaging with games.

Optimally though, developers should be detecting and fixing system errors before they damage a game’s numbers. This article will focus on how to combine behavioral analytics with system performance analytics to avoid the negative effects of technical errors and maximize a game’s retention, engagement, and monetization numbers.

System Errors Create Noise in Behavioral Analytics Data

Once a system error begins, it’s often not noticed until it shows up as a negative trend in behavioral analytics. Sometimes, it is obvious that the negative trend is due to a technical error. But sometimes, it’s unclear whether it’s a system issue, an unbalanced gameplay feature, or content that users don’t like. To determine which possibility it is, analysts generally have to mine deeply into their data to determine the affected users and conditions correlated with the negative trend.

An example of this is portrayed below:

If it is determined that the negative trend has been caused by a technical issue, it means two things:

- The issue has been allowed to persist long enough to damage the game’s KPIs and has cost the developer money.
- Data about which individual features and content users prefer have been obfuscated by technical error, making it more difficult to establish clear conclusions on what changes should be made to the game to better engage users.

Diagnosing game error with behavioral analytics tools is not desirable. A better approach is to use log management tools specifically designed to monitor system health and error logs to diagnose and fix errors.

Diagnosing game error with behavioral analytics tools is not desirable.

Below we examine how to implement log management alongside behavioral analytics to ensure that behavioral data collected is representative of a (mostly) bug-free gameplay experience, and that technical errors are eliminated before they can cause disruptions in gameplay experience or game revenues.

Using Log Management Tools to Detect System Errors Before Users Find Them

To find errors early, you want to use tools that provide you real-time system health data and help you detect and fix technical errors within a short timeframe (optimally less than a day). Most of this data can be surfaced by logging your application and the clients and servers it is installed on. Implementing logging is very similar to how you would implement behavioral analytics. You have to decide what data is important to track, but avoid inundating yourself with too much garbage data.

You have to decide what data is important to track, but avoid inundating yourself with too much garbage data.

For system data, you want to log:

● Information about the performance and health (CPU usage, memory allocation, etc.) of the servers your game exists on
● Information about your server-side application’s performance and behavior
● Information about your database performance
● Information about your client’s behavior, the state of the device it’s being used on, client-side network conditions, and code that interacts with your server application. If your game is client-only, you should still log crashes, exceptions, and select information on the application’s behavior to help you quickly determine what’s causing bugs in your application, should they arise.

However, once you log this data, it’s unwise to simply dump it to file to be searched later. To work with this data deftly, you need a place that gives you tools for properly monitoring and analyzing it.

Centralizing Logs and Isolating Issues with Log Management Platforms

Sending logs to log management providers like Loggly provides developers with mature tools to monitor log data in real-time and quickly analyze it to find the causes of technical issues. These tools include automatic log centralization and organization, search tools that enable you to search log data with custom parameters, visualization tools which represent log data in a variety of chart types that help developers identify errors and performance trends, and system monitoring and alert tools that monitor logs for abnormal behavior and send your team email, SMS, or PagerDuty alerts when there are errors or deviations from optimal system health.

When combining these tools, you provide the capability for your live operations team to know of errors immediately via alerts and provide them tools to search logs quickly for the root cause. This allows many errors to be solved BEFORE they’re able to effect your game’s behavioral analytics. Even if some issues do escape early detection, and show up in your gameplay metrics, developers will be able to diagnose and fix them much faster with good logging tools than without.

How to Pair Log Data Management with Behavioral Analytics to Provide Better Gameplay Experiences for End-Users

Let’s re-examine the game-update example above assuming the availability of log management tools from programs such as Loggly. After the update, the live operations team notices retention suffering. With Loggly in place, the team can cross-reference both their behavioral data and log data to quickly determine if the cause is due to a technical error or to user dissatisfaction with the update’s content. If the cause is determined to be a system error, developers can use Loggly to diagnose and fix the error quickly.

In the case above, if the issue in retention was not due to system errors or bugs, the analysts would be able to use Loggly to rule out the possibility of system errors quickly and focus on re-balancing the gameplay.

Best Practices for Combining Logging and Behavioral Data

Be Predictive

You can use your behavioral analytics to tell which KPIs and user cohorts are most heavily affected by error. Once you have this data, identify all of the areas of system performance that can affect your KPIs. Send all logs surrounding those areas to your log management program, set up monitoring of those logs, and create alerts to monitor deviations from optimal performance.

For instance, our behavioral analytics might tell us that NON-hardcore gaming cohorts don’t accept much difficulty in the game or long waits at the loading screen. Therefore, we set up a saved search for SLA violations related to loading in Loggly, and set it to monitor these SLAs and alert Dev Ops teams when they happen.

Cross-Validate Issues

If an issue arises and it’s not clear whether it’s a system error or behavioral error, both pieces of data can be checked simultaneously to quickly rule out possibilities of system errors.

Use Logging to Check System State Before and After Release

At most successful game companies, releases happen almost daily. Use logging to check the system logs before and after releases to validate each release’s integrity. This will save the Live Ops team from having to spend time diagnosing and fixing technical issues down the line and prevent users from being affected by any bugs in a release.

Use Logging to Monitor Third-Party Vendor Integrations

Third-party integrations often have their own performance issues. For instance, ad impressions shown in your game will often take the form of interstitial video and rich media ads. However, if players are in areas of low connectivity or have slower devices, these ads can fail and interrupt the game experience. With logging, you’ll be able to tell when and how errors such as these are happening so that you can make corrections to your third-party integration and avoid future interruptions.

Conclusion

In game companies, most people on a Live Operations team have access to the game’s behavioral metrics and are always thinking about how their work affects them. By integrating a centralized log management into this process, the whole team will be encouraged to take the system’s health into account when analyzing user behavior and pushing new content and features. System issues have a significant and often hidden effect on your game’s key numbers. Proactively monitoring and fixing them frees up developer time and allows you to keep the money and users those issues would otherwise cost you.