Dexterity leverages SignalFlag to accelerate development and bring visibility to robot performance

Dexterity understands testing

It is no easy feat to launch an enterprise-level, industrial superhumanoid robot that is warehouse ready, but Dexterity’s release of the Mech does just that. Dexterity understands that comprehensive testing is essential to the development and release journey of robots. When they were ready to automate their — already sophisticated — testing solution to accelerate their growth, SignalFlag’s evals platform gave them the tools they needed to do so.

Increased testing without increased cost

With SignalFlag, Dexterity was able to transition from small, locally-run regression tests, to automated and scale-ready testing in the cloud. SignalFlag’s cloud-based orchestration is optimized for cost and scale, giving Dexterity the confidence to expand their regression test sets without worrying about increased costs. Simultaneously, SignalFlag’s automated testing and metric generation allowed for more frequent regression tests without additional developer effort.

Testing as a force multiplier

Additional testing is most meaningful when test results and metrics are quick to create, easy to understand, and simple to share. SignalFlag’s evals platform enabled Dexterity to make strides on all three fronts.

“Our collaboration with SignalFlag has been great. Their expertise in building and scaling automated testing, and software quality monitoring infrastructure, gives us great confidence in expanding Mech testing across more applications, and against varied production scenarios.” — Shitij Kumar

Dexterity’s Mech, an industrial superhumanoid robot, at work in a warehouse. — Pictured is Dexterity’s Mech, an industrial superhumanoid robot designed with warehouses in mind. Image source: dexterity.ai

Automated metrics accelerate development

Automated test metrics on every batch allows Dexterity’s robotics engineers to quickly dive deep when problems arise, and casually explore curiosities that would have been time-prohibitive to look at previously. They leverage SignalFlag’s AB Comparison tool to take a closer look at interesting or bothersome test results, and SignalFlag’s Parameter Sweeps to test the behavior of a wide range of inputs in a single launch. The result is an impactful increase in the number of bugs caught before code is merged to main.

A sample Executive Overview showing key metrics and their performance over time. — A sample view of SignalFlag’s Executive Overview with generic data. These give an at-a-glance view of test performance, and key longitudinal metrics.

The right data at the right time

Data-driven decisions require the right data at the right time. With SignalFlag, Dexterity can take the day-to-day performance metrics used by robotics engineers, and make them available in real-time for team stakeholders. Through direct linking and Executive Dashboard views that highlight top-level metrics, these day-to-day metrics are easy to share, and easy to understand.

“SignalFlag’s dashboard tools make it easy to align the Dexterity team around the performance metrics that move the needle for our customers, and the ability to analyze metrics for both systems and subsystems accelerates our progress against critical targets.” says Kumar.