What robots need that the internet never had
In the future we'll download skills into robots
In a Whatsapp group I’m in, a few months ago, someone asked about how to get in touch with AI companies that want to record workers in factories going about their jobs.
Shift X launched in New York last week with an offer that is appears too good to be true:
“Book a shift cleaning. A vetted shift operator comes to your home wearing one of our devices. They clean. They leave. You pay nothing. In exchange, we record the cleaning. Robotics is being built on data about how people do daily tasks, and the value of that recording is what funds the service.”
Shift X is a physical AI data company using home cleaning as the mechanism to get cameras inside homes. The cleaning is the access cost.
The data is the product, and the reason it is worth funding free cleaning is that nobody has yet solved the single biggest bottleneck in physical AI development: real-world training data from inside homes.
Pronto, a home services startup in India, has been running a version of this pilot for months, and this landed it in some controversy last week. Its investor Glade Brook Capital described it to other investors as “piloting real world training data with leading physical AI labs.” When that became public, the CEO of Urban Company, which provides home services as InstaHelp, said that his platform would never do this. I can’t find the link now, but I remember reading that an investor in the space said something on the lines of how Urban Co is giving up a huge monetization opportunity, and saying they won’t do this will be disastrous for Urban Co. An overview here.
A couple of things:
A few things to consider here:
1. The home is physical AI’s hardest and most valuable training environment: At SuperAI last year Nicolaus Radford, CEO of Persona AI, identified the home as “hyper unstructured and insanely complicated”, and this being the reason why his company chose industrial deployment first:
“one activity in just different lighting conditions in a slightly different context is breaking these things.”
Homes have infinite such variation: object placement, spatial layouts, clutter, surfaces. That requires more data to understand parse the innumerable variances that AI has to navigate physically and probabilistically.
Radford said: “the next 50 years are going to be the ubiquitous manipulation of the physical world,” which requires “appropriating data, training the models, getting them distilled and deployed to produce action.”
2. Home services is the only structural path to that data: Radford estimated a million humanoids deployed by 2030 and ten million by 2035, but it’s likely that most of these will be industrial. There are already deployments in Ship building in South Korea, and the pitch is simple: these robots work 24x7 and do not get tired, do not take leaves, do not fall sick, and potentially are better at repetitive tasks.
Interestingly, Radford identified software as bottleneck, not necessarily the hardware. Industrial robots got their training data through decades of factory floor deployment.
Apart from perhaps floor cleaning robots (like a Roomba), there is no equivalent exposure to a home environment, and even this is not very rich data. Robotics labs cannot collect this data independently at scale and synthetic simulation is insufficient.
So nothing captures task-oriented, first-person video that trains physical manipulation models, which home services platforms can get consent for. The race to collect and sell this data is already underway, and we’ll see more such companies.
3. A future platform consideration: Radford described humanoids as platforms, saying that they’re App Store ecosystems rather than single-purpose devices.
“One of the ways that this is going to speed up the adoption, is when you get third-party groups to start developing applications for the robots. I think the business model is also open on how we are selling the platform? Is it a service? We’re still working through these issues on how customers are going to take it. Once those become solidified, then you’ll have the whole environment generate the commercial elements around it. And I think one of those is probably going to be a humanoid app store.”
“We believe that it’s going to be disparate and segmented into different Personas, which is what we’re calling it. Our first Persona we’re putting out is Melvin,”
I’m not sure whether this will work, but you buy the robot (device), and download the persona (app). Cue a SOUL.md joke here.
Or even better: remember Neo downloading combat training skills in The Matrix?
Each persona comes with its own set of capabilities, and you might get subscription models, and open sourced models for your needs. This means that every service, and type of service, becomes a specialised skill set that can be rented/subscribed to, and hence a data opportunity for companies building humanoid apps.
In this context, every in-home service platform is a potential data collection vehicle. Shift X says:
Today, cleaning in New York. Soon, handymen, repairs, and errands across the globe.”
Cleaning was the starting point because it is high-frequency and covers the whole home. Every factory floor becomes a training data set.
What this tells us
First, you’re still the product: I’ve gone into consent and privacy considerations here. Aakriti did a deepdive at MediaNama. There’s a line there that I want to point out:
It’s said that on the internet if it’s free, you’re the product. In the AI age, even if you’re paying, you’re still the product.
We are all training data for AI.
Second, physical skills are also fungible: in my posts on AI agents I mentioned about how repeatable processes are being positioned as skills, and skills are being generated both for agents and by agents (based on tasks you perform for them). Skills, at least online, are fungible, and coders are beginning to bear the brunt of this. Now it appears that offline and physical skills are also fungible.
Third, the physical world complications strengthen my replacement thesis: Flying back from SuperAI last year, I wrote down a replacement thesis that went something like this:
First, digital creation will be replaced. Second, digital actions. Third, manufacturing and fourth, Physical actions.
The counter argument to this is that there are already robots in factories and packaging, and drones area already doing deliveries. That’s a fair pushback, but I guess more complex actions in more complex environments are hard to replace.
I expect there will be more conversations around this at SuperAI.
Robots in manufacturing have a long history. So if we are not looking only at humanoids, they possibly are the first.