The game is up. Playtime is over and it’s time for virtual reality (VR) to graduate from a world of shooters and armchair sport to become a seriously influential component of breakthrough innovation. Already, engineers, architects, designers and even estate agents are exploiting the ability of VR to aid visualisation. But the potency VR has for creating immersive virtual experiences where human behaviour can be studied is much less widely utilised. Although I’m pleased to say that Cambridge Consultants is now very much on the case.
AI in the driving seat
Our bespoke VR driving simulator for human factors studies of level three (L3) autonomous driving was spawned by the desire to reveal behavioural insights. We think the value of L3 autonomy can be unlocked by learning more about human behaviour. We could determine, for example, if drivers are capable of event-free handover of control so that demonstrably safe and useful L3 systems could be created.
But first some background, starting with the reality that L3 falls in an uncomfortable middle ground. At this level, a vehicle may be put into an autonomous mode in specific operational domains, such as on a highway or in a carpark. But while it is driving autonomously, the driver is expected to be aware and ready to respond effectively to a request to retake control. This puts drivers in an awkward position. For large parts of the journey they are not expected to actively control the vehicle but remain ultimately responsible for its safety.
And that’s the crux. If the proposed advantages of autonomous driving are freeing up time for other tasks and improving safety, then how much benefit does L3 bring the driver? Potentially very little. This has led some to speculate that semi-autonomous solutions are a waste of time and may cost lives. Despite this, there is regulatory change underway in Europe to enable hands-off L3 driving.
An industry split
Over the last decade or so, a wave of tech companies has entered the automotive space, including Waymo, Baidu Apollo, Uber and Pony.ai. These players, often funded by venture capital, intend to reach L4 autonomy directly in the hope of reaping the rewards of automating taxis, haulage, and last-mile delivery. Often, they are not invested in the manufacture of vehicles themselves. They intend to retrofit production vehicles with expensive and bulky sensor suites and computers, the cost of which should be dwarfed over the lifetime of the vehicle in comparison to wages paid to a human driver.
However, from the OEM’s perspective, the economic drivers are different. Value for them lies primarily in their expertise in vehicle design and manufacture rather than in artificial intelligence and digital services. They must continue to sell vehicles in high volume to remain profitable, and the potential decline in private ownership due to the emergence of robo-taxis is a threat to their pockets.
The direct jump from L2 to L4 for a cost effective and aesthetically pleasing production vehicle is too high an economic barrier and does not allow for iterative development of the technology. For OEMs, L3 is a natural part of technological progression towards high automation and is therefore important to their businesses. They must continue to compete to encourage customers to buy private vehicles by adding technologies that offer increased safety and convenience and are ready for the road now. Among the voices advocating progress into L3 are Audi (who had to abandon their first-to-market L3 system), Tesla and BMW.
VR driving simulator for L3
Let’s now return to our VR simulator for L3. Testing in simulation is never as good as the real thing, but what if the tests you want to do are too dangerous or expensive in real life? In the case of studying human behaviour in hazardous driving scenarios, VR provides the ability to monitor driving in a simulation at no risk to the subject. Driven by the growth of the gaming industry, VR headsets can display lifelike virtual worlds in real time, and worlds can be generated more quickly and at lower cost than ever before. This makes VR testing an important resource for behavioural studies, enabling human behaviour to enter the design loop earlier in development.
We developed a proof-of-concept research platform in virtual reality to try and answer some of the questions regarding handover in L3 autonomy. In our driving simulator, the test subject travels autonomously through an urban environment where a basic traffic simulation is implemented for the other vehicles. During the drive, various scenarios can be triggered and the driver is requested to take over control of the vehicle.
The scenarios vary from an unexpected change in complexity of road layout due to traffic works, to a person stepping out into the road,necessitating an emergency stop. We monitor how the driver responds to the situation. The driver’s performance can be compared to their performance if they had been driving the whole time to determine whether a relatively sudden request to intervene results in poorer driving or an accident.
It is unlikely that OEMs are intending for emergency situations arising during L3 autonomous driving to trigger a handover to the driver. The case of a pedestrian stepping into the road would be handled by a pedestrian detection system as part of the L3 algorithm. With our emergency test scenarios, we are looking at a controlled handover of control, as the car leaves the operational domain of the L3 algorithm, immediately followed by an emergency that the driver must deal with. In effect, we are assessing whether the fact that they were being driven autonomously moments before the incident makes them a worse driver.
From the simulation we extract the acceleration and velocity of the car, as well as its proximity to pedestrians and other vehicles, to create performance metrics. We are also able to determine which object the driver was looking at any point in the drive through gaze tracking in the VR headset. An interesting early result suggests that patterns in gaze direction are as strong predictor of event-free handover.
Design opportunities with VR
VR also makes it possible to gather data early without the need for expensive physical prototyping. In this case it can be used to help develop a human-machine interface which can quickly and intuitively communicate with the driver.
An area that we have explored is how the vehicle can better communicate with the driver when they are asked to take over. Some styles of alert induce a panic response, which decreases human effectiveness. Other alerts are generally ignored. For driver assistance systems, what type of audio-visual alert is appropriate, and which information should the alert contain? The alert should minimise the cognitive load on the driver by being clear about what hazard the vehicle has detected, or what action it suggests the driver takes.
Additionally, a key part of the value proposition of purchasing many private vehicles is driving enjoyment – so it’s important that the technology adds to this rather than detracts from it. With rapid design-testing iterations this may be achieved.
We expect that VR will play an increasing role in human behaviour studies for automotive design. It is a vital tool for developing systems where human behaviour is important, and where physical prototyping is costly or real-life user trials would be dangerous.
At Cambridge Consultants we are developing tools and expertise for building virtual learning environments and simulating sensor inputs for use in both on and off-road (agriculture and mining) automation systems. These tools can be used for developing and validating artificial intelligence algorithms before collecting real-world data. Our work in VR complements this and enables us to put human behaviour in the development loop early in projects. We’d be delighted to discuss how our expertise can help you unlock the huge potential of autonomy in your business. Meanwhile, you can discover more about our work in mobility technologies here.
