Mobile Gaming UX Design
Minecraft Earth Gaming Showing In Mobile | Image credit: Mika Baumeister/Unsplash

Machine learning to improve the mobile game UI, UX design

Artificial Intelligence

Machine learning keeps on revolutionizing almost every industry and sector, from crop planning in agriculture to cancer diagnosis in the healthcare industry. These topics are often more broadly addressed because they have made an impact that is tangible and beneficial for humanity.  Machine learning in games, particularly in mobile game design, is also hitting the headlines in the same way.  To grasp the scale of the gaming industry, according to Newzoo’s Global Games Market Report, the video game industry has reached a global market value of $139 billion by the end of 2018 and is already far more significant than the Film and Music industry put together.

Across all platforms combined, the gaming industry now has over 3 billion gamers worldwide. Mobile games are also one of the most valuable forms of entertainment. This can be observed from the fact that Rockstar Games’ Grand Theft Auto V was the most profitable entertainment product at $6 billion in total revenues in terms of all movies, TV shows, and music. GTA’s success, along with other popular titles including mobile games like Angry Birds or Candy Crush, is based on how thoroughly the game can build a world, captivate the player, and provide hundreds of hours of playable content.

Machine Learning in Games

If you don’t already understand what Machine Learning is, in very simple terms, it’s a system that can learn and grow with experience, without being explicitly coded. ML is also an application or subset of AI (Artificial Intelligence) that allows machines to learn from data without being programmed explicitly. AI is a technology to create intelligent machines that can simulate human intelligence capability and behavior.

Games have a significant history in the co-evolution of AI and machine learning. AI has been applied to games since the early days – from classic games like checkers to modern-day real-time strategy games like StarCraft remastered. The reason why machine learning has exploded in the last 7 years is significant developments in GPU processing speed and the enormous amount of data available for machine learning and deep learning algorithms to feed on.

This article is about exploring how AI has been applied to turn-based and real-time games and how the game design is improved with the application of machine learning in game development. The construction of artificial worlds in-game UI design has served as a useful testbed for AI algorithms. Additionally, applying AI in games allows the development of competitive opponents to challenge players.

Use of Machine Learning in the mobile game design

Behind the success of any mobile game, there is plenty of labor-intensive work that is done by the developers of the game development company. Every little characteristic and nuance of each character and object in the game environment has to be hand-coded. 

This constant work takes up a significant part of a game’s development time. With the recent development in the Graphic Processor Units (GPU) and the large amounts of playing data, a new proposal has come up – To incorporate Machine Learning and other Artificial Intelligence technologies into the game development process. It can allow the game engine to respond to the player’s actions dramatically, and lessen the time taken by the hand scripting of little things in the environment.

In the crusade for experiencing more realistic worlds, captivating challenges, and unique content, AI development companies are increasingly adapting to machine learning as a useful tool in game development. Therefore, once machine learning evolves to a level where it can be reliably used in games, it could fundamentally improve the game UI design and experience in many ways:

1. Algorithms playing as NPCs

As you very well know that the opponents in a video game are pre-scripted NPCs (Non-Playable-Characters), but a machine learning-based NPC could allow you to play against less obvious opponents. These characters can adjust the difficulty level according to the environment and the player’s game-style. So the more well-versed you become with the game, the smarter your enemies could get and respond in ways that could counter your actions within the game. Game development companies are already working on early applications of machine learning-based NPCs. SEED by EA trains NPCs by imitating the players on the top. Its NPCs study dynamic shifts and actions. It uses human players’ actions as the training data, which means the algorithm trains four times faster than reinforcement training.

2. Map/Level generation

There are already plenty of instances where game app developers have used machine learning to auto-generate everything from dungeons to realistic terrain. It can increase the replayability of a game as players are shown new experiences every time they play. It can also help to lessen production costs and storage space and allow new types of games to be built on the unique affordances of content generation. Getting this right can offer a game with unlimited replayability, but it can be one of the most challenging ML to develop.

3. Making games more attractive 

Game developers are utilizing machine learning on this front as well. Game developers are utilizing machine learning on this front as well.  In a mobile game, often things look good from a distance, but when you move closer to the objects, they seem to render poorly and become pixelated. Microsoft is working with Nvidia to solve this problem and make prettier games that adapt to your play preferences. They’re using machine learning to significantly improve graphics and renderings dynamically. 

In real life, when you’re at a distance from an object, the details aren’t clear, but as you move forward towards it, you can see finer details. This dynamic rendering of finer details is a hurdle that computer vision algorithms can help to overcome.

4. Audio generation

Being able to create audio sound effects or music on the go is already in operation for other areas, not just games. Just imagine being able to have custom-designed sound effects for your game developed entirely by the machine learning algorithm. The term for this is procedural music, used in association with non-linear, dynamic, interactive, or adaptive music. It refers to programmed music within a game that can adapt or respond to different states or events at varying levels in real-time. 

5. Modeling Complex Systems

A machine learning algorithm’s strength is its capacity to model intricate systems. Game developers are constantly trying to make games to be more immersive and realistic. Illustrating the real world is incredibly challenging, but the development of machine learning algorithms can predict the downstream consequences of a player’s actions or even modeling things the player can’t control, like the climate.

One current example of complex modeling currently in production is EA Sports’ FIFA 19. When you choose your team of all-star soccer players, FIFA calculates a team’s chemistry score based on how much the players on your team have in common. During the match, team morale can drop if you are losing or slipping. It can also surge when the crowd starts to cheer when you are playing well. The variations in the moral impact the player’s abilities in the game. More slip-ups occur when morale is low, skill shots and lucky breaks happen more often when your team is playing well together.

6. More realistic interactions

Another significant challenge in creating an immersive virtual world is how players interact with friendly NPCs. In many games, you need to communicate with scripted characters to complete your levels. However, these conversations are limited in scope and usually follow on-screen instructions. With the help of natural language processing, you can talk out loud to in-game characters and get true responses, just like Siri, Alexa, or Google Assistant. Besides, games that include VR haptics or imaging of the player could allow computer vision algorithms to recognize body language and intentions, further intensifying the experience of interacting with NPCs.

Conclusion

Mobile games account for 50% of gaming revenue in the industry. Playing games on your phone or tablet is easy, you just have to pick up and play when you have spare time, without any need for a dedicated console. A few years ago, mobile games had limited scope because your phone didn’t have the processing power and graphics of a console or PC. However, those limitations are changing with the implementation of AI in game development and AI chips in the latest smartphones to add processing power.

Many of the advantages of machine learning in game UX design discussed above will become possible to mobile games and the hardware will continue improving to make mobile gaming more vivid, interactive, and immersive. It will make way for artificial intelligence development companies to continue using ML algorithms to make smarter and lifelike games.