CoaroboCoarobo GK

02Immersive XR Teleoperation

Coarobo Sightline

Walk around your robot, bring its digital twin on site, or step into its simulation, free of the limits of body and space.

Coarobo Sightline: Immersive XR Teleoperation · 1:00This is the English version, with subtitles in English, in Japanese, and in both. 日本語版
Contact UsWatch Video · 1:00

Let's discuss a pilot deployment on your robots.

Coarobo Sightline is an extended reality (XR) interface that moves people and robots between the real site and the cyberspace of the robot, across the whole span from reality to virtual reality, so operators observe and teleoperate any robot from inside its workspace or from afar.

  • Third-Person View

    Operators walk around the robot and guide it from any side, rather than through its camera as in the usual first-person teleoperation.

  • Hybrid Simulation

    The digital twin of the robot comes out of its simulation onto the real site, beside the real hardware of the line, for a dry run of each task before the real robot moves.

  • Three Control Schemes

    Operators guide the arm with two devices, the motion controllers, one device, a stylus, or zero devices, their bare hands, and pick the one that suits each task.

How It Works

  1. 01

    Two Devices

    Hold a controller in each hand, grab the gripper hologram with a trigger, and the arm follows it live or moves to it along a planned path.

  2. 02

    One Device

    Hold a stylus like a pen in one hand and touch the hologram to guide it with the precision of handwriting.

  3. 03

    Zero Devices

    Pinch the hologram with a bare hand and move it, with nothing to hold, and a second pinch opens or closes the gripper.

  4. 04

    Digital Twin

    Rehearse the next task on the digital twin beside the real robot, on site, then deploy the real robot to repeat it.

Display Modes

Move along the whole reality-virtuality continuum, on site next to the robot from reality to augmented virtuality, or from afar in virtual reality.

RealVirtual
RealityARAVVR

Reality

The real site only, next to the robot.

Passthrough
On
Overlays
Hidden

Augmented Reality

The robot and its sensor data over the real room, on site.

Passthrough
On
Overlays
Visible

Augmented Virtuality

On site, the data of the robot in an abstract virtual space that mimics the real room.

Passthrough
Off
Overlays
Visible

Virtual Reality

Fully virtual, to operate a real robot from afar or to enter the simulation of a robot.

Passthrough
Off
Overlays
Visible

Key Features

Use Cases

What You Need

Robot
A robot on ROS 1 or ROS 2 that publishes its joint states and transforms, with rosbridge running on its computer.
Arm
For the arm, an end effector velocity input such as MoveIt Servo, and optionally a MoveIt planner and a gripper action.
Sensors
Point clouds, compressed camera images, laser scans, maps, and markers for the overlays, all optional.
Profile
A profile of your robot, which Coarobo prepares from its robot description during the pilot.
Validated
UR5e, Fetch
Headset
A standalone OpenXR headset with passthrough, used with its controllers, hand tracking, or a stylus.
Network
The headset and the robot computer on the same network.
Interface
English and Japanese, with a dark and a light theme.
Delivery
A pilot deployment, in which Coarobo installs the application on your headset and sets it up for your robot.
Works With
ROS
ROS / ROS 2
OpenXR
Meta
Meta Quest
Logitech MX Ink
Android
Android
Topics
extended realityteleoperationspatial computinghand trackingthird-person viewsim-to-realimmersive interfaces

Get Started

Common Questions

What makes Sightline different?

Operators guide the robot in a third-person view, walking around it rather than looking through its camera. They move along the whole span from reality to virtual reality, on site or from afar, and they switch among three control schemes, two devices, one device, or zero devices.

What is hybrid simulation?

Hybrid simulation brings the digital twin of the robot out of its simulation onto the real site, beside the real hardware of the production line. Operators rehearse each task on the twin as a dry run, then deploy the real robot to repeat it.

Can people enter the simulation of the robot?

Yes, in virtual reality people step into the cyberspace of the robot, its simulation, to spot problems early, such as while fine-tuning a policy or training it with reinforcement learning. Newcomers also practice there before working with the real robot.

Can operators control a robot from afar?

Yes, virtual reality lets operators control a real robot from afar with its perception around them, while reality, augmented reality, and augmented virtuality serve on site, next to the robot.

Which robots does it work with?

Sightline works with a robot on ROS 1 or ROS 2 that runs rosbridge, once Coarobo prepares a profile of the robot, and it is validated on the UR5e and the Fetch.

Which headsets does it run on?

Sightline runs on a standalone OpenXR headset with passthrough, and the headset family validated today appears under Works With.

How does a pilot start?

Write to us about your robot and your goal, and Coarobo prepares the profile of your robot, installs Sightline on your headset, and sets it up for your robot.

How does Sightline differ from Viewport?

Viewport shows the data of a robot in a web browser, while Sightline lays the robot and its sensor data over the real room in a headset, so operators observe and teleoperate it from inside its workspace.

Related News

All Press Releases

Coarobo Suite

Coarobo Sightline is part of the Coarobo suite, one family of products made to work together around our open-source core.

Open Core

Platform

Applications

R&D