The game of 9s for robotics startups

Robotics startups are playing a very particular flavor of the startup game. In robotics, technical performance metrics are very tightly coupled to company growth. In particular, the engineered system’s overall reliability, sometimes referred to as its “9s”, becomes the fundamental metric which drives the company’s progression.

What’s a startup?

Venture capitalists will tell you that a startup is a business which is designed to grow very quickly. This may be true - but the reason they’re designed that way is because in the beginning, startups are expensive, and they lose a lot of money.

For startups, rapid growth is not an arbitrary or intrinsic characteristic, it is a financial necessity.

Typically, venture-backed startups require significant up-front investment to achieve the growth necessary for long-term financial sustainability. Venture capital is supposed to act as a catalyst, lowering the activation energy they need to grow up to support themselves. A startup’s goal is to make enough money that it no longer requires outside investment to survive. VCs call this becoming “default alive”.

This is the fundamental characteristic which separates startups from other businesses. From its beginning, every startup has an end date, and is racing against the clock. They may raise additional capital to postpone their inevitable demise, but unless they start making more money than they spend, all startups will go out of business eventually. This reality defines them.

What are 9s?

Engineers often describe a system's reliability in terms of percentages. If, on average, your software backend is up and running for the internet to access 99% of the time, it has “two 9s” - if it’s up 99.999% of the time, it has five. In engineering, “how many 9s you have” is a bit of a slang term for how reliable your system is.

As companies scale, engineering teams often focus heavily on this metric. Seemingly small downtime percentages for individual components can stack up to high risk percentages for a system. If your system has three critical components, each of which has 99% uptime, on average your system will be down for 43 minutes every day.

What is robotics?

Generally, robots are hardware devices controlled by software systems which work together to automate some physical task in the tangible world. We have been automating manual labor with robots for half a century - robotic arms have been welding automotive parts together since the 1980s. These robots are deterministically programmed - they move along predefined kinematic paths, executing their tasks with certainty.

For our purposes, though, “robotics” refers to probabilistic robotics - robot systems which rely on some degree of probabilism in order to accomplish those tasks. In the 21st century, most new applications for robotics rely on some degree of probabilistic execution. When a farming robot sees a dandelion with a video camera in a field of butterhead lettuce, its machine learning model can probably identify the weed correctly - but due to the innumerable variations possible in the camera image, it will never catch the weed 100% of the time.

Realistically, if someone is starting a robotics startup today, their robots accomplish their tasks probabilistically. Most of the physical tasks which are easy and financially worthwhile to automate have already been automated. Those that haven’t lie outside the realm of what industry experts would consider “modern” robotics. And regardless, programmable logic controllers (PLCs) tackle those types of manual tasks at a much lower price point and level of up front investment, making a robotics-based approach ill-advised from a business perspective.

Robots are hard

It’s difficult to start a startup in any domain, but it’s especially hard to succeed in robotics. Robotics startups face high capital costs and long hardware development times. They not only need to hire software engineers, but also mechanical and electrical engineers. On top of all that, robotics startups need domain expertise - an intimate knowledge of the physical tasks their robots are attempting to automate.

To make matters worse, each of these different disciplines builds their deliverables on top of each other. If the mechanics don’t work properly, the electronics can’t, either - and the software can’t function properly unless the entire electromechanical system beneath it operates correctly. The nature of robotics forces the startup’s “tech stack” to be much deeper than it is for a software startup.

Usually, due to specific details of the tasks that they’re automating, robotics startups have to develop each layer in this tech stack for themselves to some degree. Often, the hardware has to be somewhat customized to the task at hand. Robotics startups can not take hardware reliability for granted like they could if their entire product ran in an Amazon data center. They have to build those 9s themselves.

The fantasy Gantt chart

With this engineering reality in mind, a robotics entrepreneur might be tempted to develop each technical layer in sequence, ensuring the degrees of reliability necessary in each layer as they progress from one layer to the next. First they develop their hardware platform, then they write the software to control the platform to automate the physical task, and then they scale up their business.

Unfortunately, this is a fantasy which no startup can afford. This path might be available for research teams within large companies, but the expensive realities of robotics make linear technical development impossible for startups. With such an approach, the startup would run out of funding before it ever got to market.

The real Gantt chart

For robotics startups, simultaneous development at all layers of the stack is a business necessity. They have to build the hardware, software and business operations simultaneously in order to have any chance at reaching sustainability before they run out of funding.

From an engineering perspective, this entails rapid iteration on small-batch or one-off prototypes. The central goal here is to develop the robot’s hardware systems as quickly as possible. Engineers build just enough physical capability so that the device can be programmed to accomplish the task at hand, and then develop those pieces of software as quickly as possible.

For the rest of the business, simultaneous development means starting out by manually doing the task that your startup is hoping to eventually automate with its robot.

If your company is building autonomous tractors, you start by hiring some humans to remotely operate your tractors. If your robots are drones that capture video from the skies and identify ground features from the footage, you hire pilots and label the data manually. If you’re building a driverless car, you hire human drivers and drive around.

For many robotics startups, this is a technical necessity - autonomous vehicle companies need to drive cars around to collect data to train their machine learning models. But most robotics companies need to start with manual operations for business reasons, too. This approach gives them a head start on building the relationships necessary with suppliers and customers that will be essential when scaling the startup to sustainability.

The Game of 9s

From the moment simultaneous development begins, robotics startups are playing a game of 9s. Over time, their automation progresses from 50% reliable, to 75%, to 90%, and so on, increasing their business's efficiency, letting them scale operations sustainably.

Their central goal is to develop their robotics system so that it becomes more autonomous and reliable at the same rate that they grow their business. If a startup strikes the correct balance, its technical and operational development should progress evenly alongside one another. Over time, increased scale will bend the startup's cost curve, moving it closer and closer to “default alive”.

Robotics startups must maintain a keen and ongoing understanding of how many 9s they’ve truly achieved with their technology. If a robotics startup delays scaling up operations for too long, developing an excessively reliable system, they’ll run out of runway before they get big enough to achieve profitability (or at least justify another funding round to new investors). If they scale up too quickly, their lack of autonomous reliability will make larger-scale operations too expensive, squandering their remaining funding and bankrupting the business.

Most startups fail, and robotics startups are no exception. Robotics startups have to move quickly enough to eventually reach profitability without scaling themselves out of business in the pursuit. The only way to win the game of 9s is to carefully balance the startup’s technical progression with its operational ambition.

But there are lots of ways to lose.

Bad cost modeling

An easy way to lose the game of 9s is to incorrectly model the cost of the task the robot is automating. For example, if the robot is an autonomous tractor, the cost of the manual task is the equivalent cost of a tractor operator (either remote or on-site). If the robot sews T-shirts together, the manual task cost is how much it costs to pay someone to operate the sewing machine manually.

It’s important to keep in mind that these manual task costs can change as the startup scales. It might be harder and more expensive to find and hire 100 manual operators at a larger scale than to run the business with the one manual operator they already have at their current scale. Additional operators simply might not be available - perhaps the operators are too skilled or are in too high demand for the startup to afford.

It is just as dangerous to under-estimate the cost to manually complete the task the startup is trying to automate. This is because the robotics startup bears the cost of completing the task for the percentage of the time when its automated system fails. If the task costs twice as much as estimated, that is financially equivalent to thinking the system is 90% reliable when in fact it is only 80% reliable.

Fake 9s

It’s easy for startups to convince themselves that their technology is better than it actually is. This sometimes happens when engineering teams focus on proxy metrics for automated system performance. Sometimes these do not translate directly back to savings on operating costs. Is it sufficient to simply “minimize required manual interventions”? Are all manual interventions equally expensive?

Sometimes robotics startups can develop 9s for themselves that they measure accurately, but that don’t scale. Does the autonomous system really work in all environments, or just in the environment where they’re gathering the performance metrics? When the business is operating at 10x the scale, how will the environment change?

Will scaling necessitate environmental changes that upend any existing reliability assumptions? The ways that environments can change at 10x scale can easily invalidate any 9s a robotics startup has developed. At 10x the facility size, walls and other obstacles might be out of range of a robot’s lidar, forcing a complete redesign of the autonomous navigation system.

Conclusion

When you build a venture-backed robotics startup, you are signing up to play this game of 9s. It's a tightrope walk, and it makes starting a robotics company extremely challenging. Entrepreneurs in this space ignore the game's rules at their peril. In order to succeed, they need to remain very intimately aware of their technology’s reliability metrics to throttle business growth accordingly.

For engineers who are working in this space, communicating this aspect of your technical reality to your leadership should be of paramount importance. Without a keen understanding of how many 9s your automated system has actually achieved in the real world, your robotics startup will almost certainly fail. It is your job to make sure the metrics accurately represent reality.