Exploring Voice Controlled Autonomous Systems

  • Client: Dstl/SERAPIS
  • Project Lead: Paddy Little
  • "Voice Control is now being evaluated and even implemented by other nations, most notably Russian Ground Forces. However we see ample scope to exploit this work, and to build much faster and more integrated cycles properly linking virtual, AI enabled simulation and live experiments and trials."

The project offered an opportunity to demonstrate how effective it can be bringing Commercial off the Shelf (COTS) Artificial Intelligence (AI) from the gaming industry together with an existing Government Off The Shelf (GOTS) simulation environment, Bohemia Interactive’s Virtual Battlespace 3 (VBS3).

This type of approach highlights ways to generate a MUM(T) Testbed / Concept Capability Demonstrator (CCD) that can rapidly evolve doctrinal and requirements thinking in this area.

This project was proposed through and funded by the SERAPIS Simulation and Synthetic Environments (SSE) research framework, run on behalf of the Defence Science and Technology Laboratories (Dstl) by Newman and Spurr Consulting Ltd.

01. Problem

Experience in and observation of experiments on the integration of Robotic and Autonomous Systems (RAS) into Land Forces suggested that there are significant challenges with regulatory and safety constraints when operating with real systems in the field, particularly around the full exploitation of autonomy. This experience and observation suggest that a substantial blocker to realising the true potential of these systems was the degree of cognitive burden operating these systems places on operators and the relative lack of trust over more autonomous modes.

Meanwhile Voice Control has become a feature of day to day life with many systems now containing an element of voice interaction, for example Apple’s Siri assistant.   Cervus, with partners Wingman AI, developed a logical project goal to demonstrate the art of the possible with Voice control in a UK Defence owned synthetic environment, in the context of Manned-Unmanned Teaming (MUM(T)) and in ways that may be otherwise impractical or constrained by safety and regulations.

This project was proposed through and funded by the SERAPIS Simulation and Synthetic Environments (SSE) research framework, run on behalf of the Defence Science and Technology Laboratories (Dstl) by NSC. The project also offered an opportunity to demonstrate how effective it can be bringing Commercial off the Shelf (COTS) Artificial Intelligence (AI) from the gaming industry together with an existing Government Off The Shelf (GOTS) simulation environment, Bohemia Interactive’s Virtual Battlespace 3 (VBS3). This type of approach highlights ways to generate a MUM(T) Testbed / Concept Capability Demonstrator (CCD) that can rapidly evolve doctrinal and requirements thinking in this area.

02. Approach

A key part of the project was the Technical Interface Design which entailed the creation of a voice processing stack by Wingman, an engine to map the voice outputs from the voice stack into an action vector, and a means to map that intended action or tactic into action selection within VBS3, which moves the unmanned vehicle. This is illustrated in the diagram below:

The project also created a simple test plan to verify performance of the capabilities designed, and then conducted a series of missions in VBS3 comparing the assisted Voice AI Control mode against unassisted, conventional modes of Human Machine Interface (HMI). Simple measurements were made of performance in these comparisons, primarily for exploratory purposes, as the project scope did not warrant conducting this on sufficient scale to be statistically robust.

The project findings suggest that the premise that Voice Control could improve the efficiency and situational awareness, and reduce the cognitive burden of UGV operators, is worthy of deeper analysis and experimentation. The project successfully generated a new technical interface with VBS with the potential to allow for deeper and broader experimentation with AI in VBS. It was recommended that this technical interface is explored and developed more widely within this context. There are implications here for the relationship between simulation and real-world experimentation and development and these are recommended for further consideration by Defence stakeholders. Two broad exploitation avenues were identified involving further maturation of the concept and efforts to scale the concept.

 

03. Relevance

  • This rapidly developed project proved to be a showcase for the successful introduction/integration of non-traditional, COTS games orientated supplier into UK defence/SERAPIS environment.
  • It demonstrated the successful development of an AI Voice Control interface for VBS – which also lays foundations for use of VBS to explore new, adaptive behaviours and for extended use of AI agents. Voice Control layer for controlling UGVs in VBS was: straightforward to master, reasonably effective, and appeared to somewhat reduce level of cognitive burden of conventional (unassisted) HMI methods under some mission scenarios.
  • As a consequence, complex AI functionality within VBS3 platform can now be executed rapidly and with confidence, not only through extending Voice Control but also including a range of new object recognition applications built from the same platform.

04. Conclusion

This was a very successful, short, but high impact project. It demonstrated Cervus adaptability, working with innovative non-traditional suppliers into Defence, working at pace, as well as helping us maintain our interest in emerging operational concepts, and how they are most effectively measured and evaluated.

Voice Control is now being evaluated and even implemented by other nations, most notably Russian Ground Forces (Tass – “Russia begins Trials on Voice Controlled Robots”). But we see ample scope to exploit this work, and to build much faster and more integrated cycles properly linking virtual, AI enabled simulation and live experiments and trials.

 

Our Mission is to provide performance insight to secure Frontline advantage and save lives.

Our Vision is be recognised as unique providers of analysis, analytics and decision support using both Cervus’ core capability and an innovative network of partners and collaborators across industry, government and academia.

Linkedin Twitter Youtube Created by potrace 1.16, written by Peter Selinger 2001-2019

Quick Links

About

Newsletter Sign-up

By submitting this form, you agree that we may use the data you provide to contact you with information related to your request/submission. You can unsubscribe from Cervus marketing emails at any time by clicking the unsubscribe link in the email. To learn more, see Privacy Policy.