The Future Of Grocery Shopping Is In This Robot’s Hand
Congratulations is in order, because someone has just graduated from the robotics lab at online grocer Ocado.
They’ve spent the last year and a half practicing the same routine over and over, a relentless training regimen that’s about to get even tougher with a transfer to a bigger and more modern facility.
The student in question isn’t a person and, sadly, doesn’t have a name yet.
It’s a large robotic arm that can “see” and grab dozens of different products. Built from aluminum and featuring a simple suction nozzle as its “hand,” it has practiced picking up plastic pots of porridge, boxes of herbal tea and packets of popcorn from one plastic box, before placing them in plastic bags stretched open in a separate box. Once the bags are full, it stops. Each object takes about five seconds to move, making the arm over time about as fast as a human picker.
For the industrial-looking, orange prototype, two Microsoft Kinnect cameras are bolted to a frame above the arm and peer down into a box of food products. These help the arm see the objects as a mountainous terrain, on which it designates white dots to target with its nozzle.
After it picks up a pot of noodles it scans the product to identify it. You can watch the arm at work in the video below:
“The goal is to eventually have one robot that can grasp all 50,000 grocery products,” says Alex Harvey, who heads up a team of around 100 people working Ocado’s robotics and autonomous systems department. His company claims there’s nothing quite like this working in the world today.
For those outside the UK, Ocado is an e-commerce grocer comparable to Trader Joe’s in the United States in terms of quality – but based totally online. Calling itself the world’s largest dedicated online grocery retailer, it was founded 17 years ago and booked sales of $1.4 billion in 2016.
It’s something of an unsung hero for innovative grocery delivery. While Instacart in the U.S. hires humans to collect products from physical stores and other grocers build “dark stores” for their staff to collect goods, Ocado has highly-automated warehouses manned almost entirely by robots.
The bots “swarm” around a giant cube filled with grocery products, a kind of hive that’s as wide as a football field. The robots then deliver boxes filled with products for humans to sift through before those humans bag and load them onto trucks.
Would the robotic arm replace these people? Harvey pauses. “It would.” But he adds that staff whose roles get replaced will get retrained and moved to other parts of Ocado.
The grocer currently processes 160 products per hour, per person, across its four British warehouses, and it’s been improving that rate by around 3% each year. Robots won’t just speed up delivery and bring down prices, though – they’ll also improve on warehouse technology that Ocado is selling to other grocery firms.
Ocado is already building automated warehouse technology for fellow British grocer Morrisons, and this week the company announced its first international customer: France’s Casino.
These are crucial deals for Ocado, whose profits have been only gradually increasing for the last five years on the business of selling groceries alone. The company’s real opportunity is in licensing its futuristic hive technology to other grocers. News of the French deal sent Ocado’s shares up more than 20% in London on Tuesday.
No surprise Ocado’s robotics team is under pressure to put their innovations to work quickly. Instead of teaching the arm to recognize all the grocer’s 50,000 products, which would have taken too long, the company quickly developed a 3D vision system so it could just look for grasping points.
The robotics team are based inside Ocado’s first-ever fulfilment centre in Hatfield, North London. Their “lab” is a spartan corner in the vast warehouse, which is itself a lofty, almost awe-inspiring cavern filled with miles of steel beams and tracks on which colourful plastic boxes filled with groceries zoom towards the next leg of their journey — the customer’s front door.
A minute’s walk away from the engineers, people in bright green Ocado shirts stand at the end of conveyer belts, checking the screens in front of them and putting grocery items into plastic boxes. There are hundreds of them working in different parts of the warehouse. Soon they will become all too aware of their artificial replacements, because they’ll be asked to help develop them.
Harvey’s team plan to ask the picking staff to start wearing a glove with electronic skin next year. As they use their hands, the gloves will send data back to train Ocado’s algorithms about better grasping strategies and learn things like, “how much force damages a tomato,” Harvey says.
For now, Ocado’s robot is unique in being able to grasp lots of different products. Industrial robots are typically designed to pick up one thing well, like uniform bottles of water or tubes of toothpaste. But there’s been little research on grabbing objects randomly scattered inside a box.
Even Amazon, which took 55% of e-commerce sales on Black Friday and Cyber Monday and whose warehouses are teeming with wheeled Kiva robots and cranes, relies on human pickers for the intricate task of taking individual products out of one bucket and placing them in another.
Among the closest comparable machines are the Sawyer and Baxter robots from ReThink robotics, which are adept at performing specific tasks but can struggle to place objects down safely, according to an Ocado spokesman.
Harvey’s team have become obsessed with tracking how we use our hands to pick things up. They have for instance, watched videos of a human picker scooping several heads of lettuce out of a box, pressing the final head against the box’s side to lift it out.
“We are inspired by looking at how people pick,” says Harvey.
Once Ocado’s arm gets moved to its bigger warehouse in Andover, it’ll practice on a wider range of goods and support the human staff whose job it is to put them in groceries.
The arm is the third major robotics projects that Ocado’s engineers have been working on. There’s a human-shaped hand made out of translucent plastic tubes, controlled with compressed air.
There’s also a life-size humanoid robot that runs around on wheels, which I was forbidden to take pictures of during my visit on Monday. This robot has cameras in the place of eyes in its head and a telescopic torso that allows it to reach high places. “Like Inspector Gadget,” says Harvey.
In the middle of the robotics lab stands a wooden mock-up of the warehouse’s steel tracks. This is where the humanoid robot, known as SecondHands, can practice skills like holding a sheet of metal in place above its head — using its “go-go Gadget arms,” Harvey says — while a human maintenance worker screws one of the edges into place.
Once SecondHands proves itself in the warehouse, the plan is to deploy it with Ocado’s fleet of driverless delivery trucks, currently being tested in London, to replace the human drivers that bring groceries from a truck to people’s front doors, according to Harvey.
Ocado has looked outside its walls for help in developing its robots, using off-the shelf components to build the arm, building its software on the ROS (robot operating system) developed by Willow Garage, and partnering with academic researchers at the University of Pisa and the company Shadow Robotics in London.
Harvey’s engineers have also developed algorithms to control the arm, which will increasingly use machine learning to get better at recognising products and how best to pick them up as well as a human would. Harvey says Ocado has been engaged in “hiring ping pong” with Google’s Deep Mind to bring on board specialists in artificial intelligence.
Ocado was one of the first fully-online grocery delivery firms in the world, and while it now faces online competition from the likes of Tesco, Sainsbury’s and Amazon Fresh, Ocado is one of the reasons why Britain has the second-highest proportion of citizens ordering their groceries online, after South Korea.
The company delivers to around 250,000 households each week, and if the arm helps save on labour costs, speeds up packing and ultimately brings down the prices of its goods, that number could increase.
Tim Steiner, who co-founded Ocado 17 years ago, has been open about his ambition that “every human touch point is designed to one day be replaced by a robotic solution.”
His robotics team seem well on their way to making that happen.