Robotics and Intelligence Systems >> Dynamic Autonomy >> Collaborative Cognitive Workspace

Dynamic Autonomy

Collaborative Cognitive Workspace

If we want humans, air vehicles and ground vehicles to work as a team, we need to develop an appropriate level of discourse, including a shared vocabulary and a shared cognitive work space collaboratively constructed and updated on the fly through interaction with the real world. This cognitive work space could consist of terrain overlaid with semantic abstractions generated through autonomous recognition of environmental features with point-and-click user validation and iconographic insertion of map entities. Real-time semantic maps constructed collaboratively by humans, ground robots and air vehicles could serve as the basis for a spectrum of mutual human-robot interactions including tasking, situation awareness, human-assisted perception and collaborative environmental “understanding.” This work will allow natural human-robot communication within the context of a mission based on shared semantic maps between the robotic system and the operator.


The INL is working with the Naval Research Laboratory to demonstrate a robotic system in which one operator supervises a team of four robots as they move through an unknown urban search-and-rescue (USAR) environment. As robots identify environmental features and victims, these will appear automatically as icons within a map that is being generated by the robot on the fly. The human will assist in identifying and classifying targets in the environment using the interactive visual display. Using a point-and-click interface, the user will be able to add, verify, remove or annotate abstractions within the map. Collaborative construction of the map enhances each individual team member’s understanding of the environment and provides a shared cognitive work space for the human-robot team to interact. Robots can use the map to communicate about the task and environment both graphically (e.g., “The highlighted area has been searched.”) and verbally (“Victim found near door number 3 in room 6.”) using the semantic names assigned within the shared cognitive work space. The human team members can use the map abstractions for tasking either graphically (e.g., “Search this region for human victims!”) or verbally (“Go to room 4.”).

Cognitive collaborative workspace with semantic annotations Cognitive collaborative workspace with semantic annotations.

« Prev Next »

Page Contact Information:

Department of energy

DOE Office of Nuclear Energy
DOE-Idaho Office