Resource Monitor by Freedom Robotics is now live! As part of the launch, we are making it so that anyone can test out our platform free for a whole year. Use code "MONITOR" (case sensitive) to access it today!

Sign Up for a 1 Year Free Trial X

Operator Teleop: Faster, more reliable, and with advanced gamepad control

Achille Verheye (Sr. Robotics Engineer)
Sep 10, 2020 |

Teleop

Most mobile, logistics, agricultural and other robots can now work autonomously between 90 and 98% of the time, however, they are not able to fully function without human support when they encounter a situation out of the norm.

In talking to over 800 roboticists and companies, 68% flagged stable teleoperation is mission critical at some level to remotely deploy fleets of robots. Being able to remotely control your robots stably matters. And, it is very hard to do.

Companies have requested operators being able to:

  • Override mission planning and send a robot to a location
  • Take over driving to get a robot out of a safety situation
  • Enable manipulation and control of lifts and arms
  • Manually execute the pick of an object if it can not autonomously complete it
  • Seamlessly switch between control modes of a robot (Driving vs controlling an arm or elevator)

With the goal of quickly enabling a robot to return to automatic operation quickly. Without this, we have had several companies report that their entire production line has gone down repeatedly for minutes at a time due to being starved. This is because a single robot got into a situation it could not autonomously navigate and it took a significant amount of time to safely reset it and deliver the missing parts.

MiR Robot

In complex environments, AMRs can easily get into situations where human level intelligence is needed.

 

We are excited to announce the release of our Operator Teleop, or colloquially known as Pilot 1.5, which builds on our June initial release of Freedom Pilot.

This major release enables Freedom Robotics’ users to intervene and tele-operate robots in poor connectivity environments. It takes into account significant real-world feedback and feature requests over the last few months from Freedom users who have deployed fleets which rely on Pilot. Additionally, it has been optimized with hundreds of hours of testing and tuning in poor cellular connected areas on a variety of robotic platforms.

The new release introduces a much faster, smoother, and more responsive teleoperation experience, several bug fixes and UI improvements and adds support for using physical game controllers for advanced mobile robot and robotic arm control.

 

A Game Changer in Remote Teleop

To be able to confidently control a robot from anywhere in the world, a fast, smooth, and stable connection is needed. In areas with poor connectivity and limited bandwidth, the connection algorithm has to properly scale down video properties that matter the least to ensure a connection that guarantees safety and a smooth driving experience. This has always been our focus and today we are announcing a major step forward in this direction, after very significant testing and tuning of the Freedom Agent and underlying transport infrastructure.

Freedom Robotics - WebRTC Latency Graph

In extremely poor connectivity environments, latency can spike through dead spots in wifi or cellular connectivity. Through multiple infrastructure iterations which increased the resilience of the connection and optimized bandwidth, areas of poor control have been almost eliminated.

 

Poor connectivity doesn’t just happen outdoors on cell networks. In many logistics facilities, where they can be up to a quarter mile long, there are major weak or dead spots in connectivity which robots have to navigate.

 

Huge increase in performance in poor environments

With the improvements in Pilot 1.5, in real-world, constrained, cellular connectivity, environments, you will see significant increases in stability during teleoperation:

  • 48% increase in frame rate with the same settings, resulting in enhanced control at all speeds
  • 35% lower control latency with a complete elimination of latency spikes due to sudden drops in bandwidth
  • 61% more stable bit-rate for improved bandwidth usage
  • 16x fewer disconnects in low-connectivity areas

image2

This is a short video showing the smoothness of webrtc control. Watch the correlation between the movement of the joystick and the viewport.

 

100s of hours of A/B testing in harsh environments

In addition to customer beta testing through this process, A/B testing was done on many platforms on wifi, cellular connectivity and with computer resources as small as low-power, ARM systems up to NVIDIA-enabled, 300 lbs mobile platforms.

image9

Platforms like the Rover Robotics and Fetch Robotics platforms need high-quality, low latency remote control.

 

A key to remote teleoperation is stable connectivity, either through wifi or, more regularly, cellular modems. Definitely check out our previous blogs on 8 Common Mistakes when Connecting your Robot with a Cellular Modem and 18 Ways to Stabilize Robotics Compute Resources.

Please see the Freedom Robotics documentation for how to set up and tune teleoperation to your use case.

 

Additional Pilot 1.5 Features

Fast and Powerful Gamepad control

We now support full gamepad or joystick buttons and axes through webRTC, enabling real-time control of robot arms, hybrid control of complex systems, and fast command issuing. The message sent through is formatted to work like ROS’ sensor_msgs/Joy message. We support most major gamepads, including the PS and Xbox controllers. For a full list, see here.

 

Here's a video of us controller a Fetch Robot through Freedom Robotics' teleop feature with a standard Xbox controller

For more information, please visit our documentation on how to set up gamepad control.

 

Improved UI and Map View

With this update, the settings page was overhauled to make it easier to find the right settings and be more clear. Additionally, maps are sent through in higher quality and now support visualization of known obstacle-free space, making it easier to use for navigation.

Freedom Robotics - Pilot Settings

It is extremely easy to set up game controllers by going to DEVICE -> SETTINGS -> PILOT and changing from a twist to a joy message.

 

We’re always looking to hear your feedback. Shoot us an email at hello@freedomrobotics.ai.

 

your time is valuable
Use it wisely.

Mission critical software infrastructure to enable the next generation of
robotics companies to build, operate, and scale robots and robotic fleets.