The Robot Puppeteers Of Silicon Valley Teaching Humanoids How To Make Your Morning Coffee

Fernando Flores tin walk 8 hours a time pouring nan aforesaid cup of coffee.

He is not a barista. He’s a robot puppeteer, trying to train humanoids.

He manipulates mechanical arms remotely, utilizing manus and limb sensors to make them prime up a cookware of coffee, move it into a mug and put nan cookware backmost successful nan java maker. Flores checks for spills, past empties nan mug backmost into nan cookware by manus and does it again — hundreds of times.

“The repetitiveness, it tin origin immoderate discomfort,” said Flores, who has nan title of elder robotic aviator astatine San Francisco startup Encord. “It becomes 2nd quality aft a while.”

This Sisyphus of Silicon Valley is connected nan beforehand lines of a quickly expanding manufacture of robot trainers, preparing to thatch and run nan service of humanoid robots scheduled to march retired of adjacent factories successful nan coming year. Encord practices, records and sells information astir activity to nan companies racing to bring humanoids to homes, offices and factories.

If tech companies’ optimistic plans are to beryllium believed, a swarm of American-built robots is astir to deed nan market.

Tesla’s Fremont mill stopped car accumulation this twelvemonth to make measurement for accumulation lines for its Optimus robots, pinch unbelievable plans to ramp up capacity to 1 cardinal units a year. Palo Alto-based 1X Technologies is already manufacturing its 66-pound, 5-foot-6 humanoid named Neo astatine its mill successful Hayward. The institution received 10,000 preorders, and its first shipment is expected later this year. Figure AI’s humanoid factory successful San Jose has accrued its manufacturing capacity to nutrient 1 Figure 03 robot an hour, pinch nan extremity of producing 12,000 a year.

Fernando Flores demonstrates nan articulation of a robot performing a whisking mobility astatine Encord connected May 21.

(Paul Kuroda / For The Times)

Goldman Sachs projects nan world marketplace for humanoids could scope $38 billion by 2035.

The AI of these humanoid robots needs an immense magnitude of information connected quality movement. How humans write, speak, codification and constitute was easy scraped disconnected nan internet, but nan bots request much accusation to maestro really to stand, step, lift, squeeze, move and execute different beingness movements. That is wherever companies for illustration Encord travel in.

The $10 billion invested successful robotics successful 2026, according to CB Insights, has spawned an manufacture focused connected training robots. Initially, that meant humans strapping iPhones to their foreheads, signaling actions for illustration cooking, cleaning and performing family chores. That, however, doesn’t seizure nan nonstop torque, unit and grip required for a robot manus to activity flawlessly.

Now, humans are straight guiding robots done costly rigs that fto them power nan robots’ movements. Data collected utilizing robot arms connection richer insights into centrifugal skills and entity manipulation. Encord charges clients up to $1,000 per hr for training data.

The accusation gathered from trainers controlling robots is “super important to span nan adjacent level of learning,” wherever robots will study to correct mistakes and do nan chores connected their own, said Vineeth Velmurugan, caput of robotics learning astatine Encord.

The institution is already moving pinch immoderate of nan apical companies successful robotics, but said it couldn’t stock astir names. Among nan clients it could mention were Toyota Research Institute and Weave, which already has laundry-folding robots successful a fewer homes.

Brian Gonzalez pulls an ethernet cablegram utilizing a robotic limb astatine startup Encord connected May 20.

(Paul Kuroda / For The Times)

Many of nan caller robotic information companies are focusing connected business usage cases. Robots tin execute amended successful a structured, predictable environment, for illustration a mill aliases warehouse.

Home tasks are tougher, arsenic layouts and tasks are much varied and messy. While galore bots person mastered walking, they still struggle to unfastened doors, fridges and washing machines smoothly. They don’t cognize wherever aliases really to grasp a doorknob, grip aliases doorway separator aliases really overmuch pulling, pushing aliases twisting unit to apply.

Flores has mastered making nan robot arms move coffee, but he still often spills. When that happens, he deletes records of nan attempt.

“Typically, we don’t want immoderate mistakes,” he said. “If we person much than 3 consecutive mistakes wrong a 15‑second window, that’s not going to beryllium bully data.”

Inside Encord’s trial installation successful Hayward, it has replicated a modular American location pinch a afloat furnished surviving room, room and bathroom.

In nan surviving room, a aviator rearranges an untidy study desk. She first scatters AA-size batteries, pens and scissors connected nan table, and walks backmost to nan adjacent power rig to make nan robot arms spot each 1 wrong nan tray of a desk organizer.

Depending connected nan day’s training, nan pilots could beryllium opening and closing refrigerator doors, whisking liquids successful a bowl, sorting silverware aliases turning a h2o faucet connected and disconnected complete and complete until nan robot arms get it right.

Cortney Weintz, left, and Tony Schiller grounds information pinch cameras astatine Encord.

(Paul Kuroda / For The Times)

In different area of nan facility, group wearing smart glasses spot and prime up playing cards and benignant integrative plates by hand, collecting first-person videos.

One cardinal accomplishment for nan coming bot invasion: plugging successful cables.

Companies want robots that tin crawl into duct spaces, place ports and plug cables to thief build nan monolithic information centers needed for AI. Encord replicated a existent information halfway server rack, wherever an usability inserts bluish cables into penny-sized sockets each day.

Many companies person entered this business. Meta-backed Scale AI and Palo Alto-based Micro1 are awesome players successful nan space. China has much than 40 state-owned robot data-collection accommodation wherever hundreds of on-site humans mimic train bots really to move successful nan existent world.

In Watertown, Mass., Tutor Intelligence has group up a 100-robot installation dedicated to harvesting activity data. Its robot arms, which are being trained to do mill work, are controlled by a quality squad divided crossed Mexico, nan Philippines and Boston. This is successful portion to train its robot, Sonny, which will deed nan marketplace later this year.

Elaine Batchlor sorts screws and bolts pinch a robot successful a mockup astatine Encord.

(Paul Kuroda / For The Times)

“We built nan Data Factory to bootstrap nan first intelligence for nan Sonny robot, truthful that we tin statesman to deploy Sonny into nan field,” said Josh Gruenstein, co-founder of Tutor. Ten of its distant operators are based successful Boston, and nan remainder are international.

Remote cognition is emerging arsenic an integral part of nan humanoid robot business. Employing teleoperators successful countries wherever wages are overmuch little than successful nan U.S. could, successful theory, mean a robot controlled by a quality successful different state could do a task astatine a fraction of nan costs of having an American do it.

This month, a humanoid robot cleaning work successful San Francisco called Gatsby completed a robot cleaning of a U.S. location utilizing a teleoperator successful Mexico.

The exertion is still evolving, said Aron Frishberg, co-founder of Gatsby, but being a first mover intends Gatsby is getting much training.

“There’s evidently worldly that goes wrong,” he said. “It’s really difficult to get precise manus movements aliases limb movements and drawback something.”

Encord co-founder Ulrik Hansen said it will beryllium mounting up a teleoperations halfway successful its Hayward installation successful nan adjacent 3 months. Even arsenic much robots are deployed and maestro progressively blase tasks, they will still request humans to occasionally return power remotely.

“They will request immoderate objection handling erstwhile they get things wrong,” he said.

Hundreds of teleoperators will study wherever nan strategy succeeds, wherever it breaks and measurement successful erstwhile needed. Once those patterns emerge, Hansen said, they tin move teleoperations to cheaper locations overseas aliases to nan Midwest.

Back successful Hayward, Flores created caller coffee-pouring challenges for his robot arms. He changed what was connected nan antagonistic astir nan java shaper and moved nan mug to different spots. It takes a batch of know-how to puppet and train a robot, he said.

“A batch of group would (guess) this mightiness beryllium easy, this is dumb,” Flores said. “There really is thought here. There really is captious thinking.”