How do you teach a robot to play foosball (table football)? In part 2 of this blog series, we introduced Foosbot: an internal development project that runs in parallel with our research to discover and develop new machine learning approaches.
The road to AI-enhanced ADAS
At Cambridge Consultants, internal projects such as this provide opportunities to explore personal areas of interest, collaborate on pushing the state-of-the-art in emerging technologies and solve complex technical challenges. Foosbot complements our expertise in deep learning by exploring the system design challenges associated with Artificial Intelligence (AI).
Since Foosbot’s introduction, our team has been hard at work on coming up with innovative solutions to complex machine learning problems. We’ve learned that there are various nuances to teaching a machine to play foosball, many of which we will detail in future posts (stay tuned!). In this post we focus on the AI strategy and how we have navigated the complexities of the Foosbot design.
The big picture
To create a foosball-playing robot, we first had to think of how a human plays the game. We identified three major components to game play by humans: tracking the ball, devising a strategy, and making strategic decisions a reality by moving the handles.
Once we grasped the complexity of human game play, we quickly realized that machine learning alone would not solve this systems-level problem. Designing the Foosbot would require expertise in control systems, systems engineering, and real-time processing to be successful against a human. To compete with human sensing, intelligence, and reaction, our Foosbot will use machine vision to track the ball, implement machine learning to inform strategic game play, and engage in real-time, real-world interaction with humans to move the handles.
AI Foosbot strategy
So what strategy should a robot employ to play foosball? There are many established AI algorithms that can be applied to this problem, both classical and cutting edge. Choosing an algorithm involves trading off development time, complexity, and performance level. Most importantly, we need to consider the big picture as we have established this is a true systems-level project. With lots of moving pieces (literally!) there will certainly be some un-anticipated nuances that arise as we progress.
All things considered, it did not seem like the wisest choice for our team to invest too much upfront development time in the latest and greatest deep reinforcement learning algorithm based on assumptions that may turn out to be incorrect. Instead, we decided to design a baseline state machine-based AI Foosbot strategy, called ‘bot0’ explained in the next section. We view this bot0 as our minimum viable strategy. The benefits of bot0 are twofold: in addition to getting up and running quickly, bot0 will also be extremely useful in evaluating other AI algorithms for foosball strategy at later phases of the project.
Our end goal is a Foosbot playing on a physical foosball table. However, at this phase, we’re working in a simulated virtual environment based on the OpenAI Gym framework. OpenAI Gym uses Box2D, an open source two-dimensional physics engine, as one of its simulation engines. By using Box2D, we were quickly able to develop a realistic looking, fast-paced two-dimensional foosball simulation.
Bot0 is designed to work in this Box2D-based foosball simulator. Because this is a two-dimensional simulation, we had to model three-dimensional actions by projecting onto two dimensions. For example, consider a foosplayer handle swinging back to prepare to kick the ball. To model this in two dimensions, it appears as if the entire handle moves backwards from an aerial view, but if the ball were to intersect the handle, it will pass underneath unobstructed. It’s important to note that the interface to the simulated environment and the real environment are the same, so we can play with bots in simulation and then simply swap out the environment configuration to run bot0 in the real world.
Bot0: the minimum viable strategic solution
Bot0 is our minimum viable solution for Foosbot strategy and is based on a finite state machine that we designed from scratch here at Cambridge Consultants. As this is bot0 and we’re keeping it simple, each handle has its own independent finite state machine. Consequently, each handle will determine its actions independently of the other handles. At later phases, we’ll explore more cooperative playing techniques.
To design bot0’s finite state machine, we first came up with a list of possible states that a particular handle could be in at any given time. The handle could be in a blocking state if the opponent shoots the ball at your handle, preparing to shoot, shooting, idling, or what we called ‘un-blocking’, which is when the foosplayers on the handle in question will lift their feet to allow the ball to pass underneath. This un-blocking state is useful if another handle on your team shot the ball towards the opponent’s goal and the foosplayers on your handle need to get out of the way (or ‘un-block’!).
The transitions between the states are based on the position and velocity of the ball, the high-level logical expressions of which are held within the labels on the arrows in the state machine diagram above.
The entrance to ‘Prep’ was a notably interesting transition. What is a good way to determine that a handle should start preparing to shoot? It turned out that to get close to how an amateur human player might go about making this decision, we had to consider two distinct situations. One is when a ball is approaching a handle at slow speed, such that the player has enough time to rotate the handle backwards to gain striking power before the arrival of the ball. The other is when the ball is “sitting” in front of the handle -- within striking distance but not actually approaching the handle (perhaps moving in parallel with the handle, which is a particularly challenging situation).
Translating the action described by the finite state machine to “target handle positions” (the input to our simulated PID controller that controls the handle position) required careful consideration and a few iterations to get right. For example, in the ‘Prep’ state, the action is for the handle to prepare for a shot. But what does this mean in terms of moving the motors that control the handle? In the virtual environment, we interpreted this action as lining up the foosplayer with the point of intersection of the ball with the handle, while tilting its feet back by a set angle. Moving to the physical table environment will surely introduce new complications that may change our approach for preparing to shoot.
How did we know when bot0 development was done? If this were basketball, we could easily define some metric – for example – ‘good enough’ is hitting a certain percentage of shots. However, foosball performance is more subjective than a single metric. Accordingly, our goals for bot0 include working states (blocking, unblocking, preparing to shoot, shooting, and idling), and game play that looks as if a decently skilled human could be playing. Specifically, we were looking for the foosplayers to not make silly mistakes and at a minimum block when the ball is coming at them, not score on their own goals, and shoot when a human might choose to shoot. This subjective metric for bot0 will become an objective metric for our future AI strategic bot implementations. The strategy chosen for the final Foosbot should beat bot0 and other competing AI-based strategies.
So, what's next?
In our simulated virtual environment, bot0 performs skilfully and looks like a human playing in a 2D game. See for yourself in the gif below! In this short clip, bot0 is playing bot0 in our Box2D-based virtual foosball environment. At any given time in this animation, the colors of the foosplayers correspond to the state from the finite state machine above.
The next step is to get bot0 working on the physical foosball table. This transition to the physical table will present unique challenges that we haven’t yet encountered. Once we’ve transitioned bot0 to the physical table, we will start exploring more complicated AI strategies to compete against bot0. For now, bot0 has set the bar high!