5G-VINNI: Remote Robotic Control with 360° VR-based Telepresence

Overview

In February 2020, the 5G-VINNI UK site demonstrated a trial of 5G connectivity being used to undertake remote robotic control.  The trial used of two simultaneous paths through the 5G network, one to provide a low-latency control path to the robot, and the other providing the robot’s controller with a 360° view from the robot’s cameras, viewed through a virtual reality headset.

The trial was conducted at the UK facility of 5G-VINNI, part of the Horizon2020 ICT-17-2018 programme. 5G-VINNI provides a set of connected end-to-end 5G testbeds across Europe with the UK facility located at BT’s Applied Research centre at Adastral Park, Suffolk, and built using Samsung Networks equipment. 

BT Applied Research and Interdigital Digital Labs collaborated on the trial using one of Interdigital’s turtlebot robots. The robot can move around on a flat surface as determined by the robot’s controller who wears a VR headset, and who controls the robot’s movement with a game controller. The robot is able to move forwards/backwards, left/right as well as being able to spin clockwise/anti-clockwise.

In February 2020, 5G-VINNI UK site demonstrated a trial of 5G connectivity being used to undertake remote robotic control.  The trial used of two simultaneous paths through the 5G network, one to provide a low-latency control path to the robot, and the other providing the robot’s controller with a 360° view from the robot’s cameras, viewed through a virtual reality headset.

The trial was conducted at the UK facility of 5G-VINNI, part of the Horizon2020 ICT-17-2018 programme. 5G-VINNI provides a set of connected end-to-end 5G testbeds across Europe with the UK facility located at BT’s Applied Research centre at Adastral Park, Suffolk, and built using Samsung Networks equipment. 

BT Applied Research and Interdigital Digital Labs collaborated on the trial using one of Interdigital’s turtlebot robots. The robot can move around on a flat surface as determined by the robot’s controller who wears a VR headset, and who controls the robot’s movement with a game controller. The robot is able to move forwards/backwards, left/right as well as being able to spin clockwise/anti-clockwise.

Architecture:

Connections between the robot and its camera, and between the robot controller’s headset and movement controller, were made over the 5G-VINNI UK facility at Adastral Park, Suffolk, UK. The experiment makes use of the 5G 28GHz mmWave fixed-wireless access radio system deployed at the facility by Samsung. This mmWave radio system provides up to 1Gbit/s (download) connectivity to a set of Samsung indoor CPEs which then present an Ethernet interface to a laptop PC (which connects to the game controller and VR headset) and to a WiFi access point (used to provide radio connectivity to the turtlebot). 

Deployments:

The trial demonstrates the use of two simultaneous paths through the 5G network:

• The first is a low-latency path, used to control the robot’s movement. The low latency provides closed loop robotic control and ensures that the movement of the robot takes place without any lag, that is, when the controller tells the robot to move in a certain way, the robot does so immediately. The user enacts the movement with the use of a game controller as shown below.
• The second is a high bandwidth path, used to stream the video content from the robot’s 360° camera to the VR headset worn by the robot’s controller. This gives the user an immersive video experience in which the high bandwidth ensures that the video is received by the headset immediately, enabling the controller to see the movement in real time. The images seen by the headset wearer can also be viewed on the laptop screen.

Results

Using this configuration, we were able to safely demonstrate full control of the robot’s movement, including successfully negotiating routes around obstacles placed in the robot’s path.  This is only possible through the low latency exhibited in both connection paths, providing immediate control of the robot, and immediate visual reaction through the video path, when the movements actioned by the controller of the robot take place. With any previous technology, the latency incurred in the network would make the control of the robot much less interactive and require persistent stop-start action.

5G Empowerment

The trial is the first of its kind at the UK facility and has demonstrated the support of simultaneous low latency and high bandwidth capabilities of 5G for a high performance use case of this type. Further trials are planned which will build upon this use case, using the robot’s camera to perform resource recognition and an artificial intelligence system to help determine the robot’s actions

@5gVinni | https://www.5g-vinni.eu/