A trio of students from the University of British Columbia have put their expertise to the test by building a single-player, artificial intelligence (AI)-controlled air hockey table.

This is interesting enough, but what really takes the biscuit is that the AI managed to learn to play purely in simulation.

Typically, AI-controlled robots like this are trained through trial and error in the 'real world' by failing and learning from thousands of practice runs. While effective, this is time-consuming and can damage hardware.

Screenshot of the AI air hockey system in action. Image: HudsonNock/YouTube.

The UBC team, however, decided to see if they could train the AI in the digital space, make its mistakes there first, and then be unleashed on a real air hockey table. To this end, they built a highly accurate digital twin of the table to train the AI.

Once the training was complete, they then copied the trained AI into the real robot to see how it performed against a human opponent. And the results were surprising.

Despite any real-world experience, the AI-controlled robot air hockey player was able to put up a real challenge. All this straight 'out of the box', as it were.

Tempered digitally for the real world

What is also interesting here is that AI typically struggles to learn tasks like air hockey. This is because the puck, in particular, moves very fast, can move unpredictably, and is heavily influenced by subtle changes from interactions with paddles and walls.

Any robot player also has to factor in time lags from motors, cameras, voltage fluctuations, mechanical vibrations, and imperfect puck tracking. So, even tiny errors can result in game-losing miscalculations.

To this end, the team purposefully designed the AI’s training environment to be imperfect. This is a diversion from most training scenarios, which can be 'too perfect', but the real world clearly isn’t.

So, the team factored in things like uneven rails, warped tables, inconsistent rebounds, power supply dips, and camera latency. Called 'domain randomisation', this enabled the AI to learn to expect the unexpected and react as best it can, just like a human player has to. 

 

This, the team explains, teaches the AI to predict a range of likely outcomes from a puck bounce rather than an exact outcome based on a set algorithm. So, the AI learnt to expect a puck to be in an approximate place in the future and react accordingly.

To speed up training, the team ditched the idea of using normal physics engines like Unity and Unreal and instead turned to something called 'soft actor critic' training. In short, this is a sort of carrot-and-stick-based learning environment.

Air hockey is a punishing sport

Within it, the AI takes action and gets rewards or punishments based on its performance. Over millions of simulated games, the AI gets better and better at playing the game and adapting to all the built-in chaos of the game.

When applied to the real world, the AI player was also giving a helping hand using a special overhead camera and a retroreflective tape-coated puck. This helps the AI 'see' the puck more clearly at 120 frames per second.

Beyond air hockey, the team’s innovation also has some potential applications for training other autonomous systems like drones, vehicles, robots, etc. If the training of these platforms could be sped up and made more realistic, similar sim-to-real transfers could be a huge boon.

If you want to learn more, details are available on the project’s official GitHub page.