Instacart recently introduced its approach to Physical AI for grocery.
When a retailer deploys a Caper Cart, it’s not just capturing data – it’s processing what’s happening in the aisle in real time, directly on the cart. That includes what’s in a customer’s basket, and precisely where they are in the store, said the company.
That’s what makes it “Physical AI” – the intelligence runs on the cart itself – so retailers can influence the shop while it’s happening, not after checkout. Mistimed recommendations don’t just fail – they train customers to ignore the next one, added Instacart.
“For example, prompts like, “Got everything you need?” are driving a nearly 1% lift in basket size on average, highlighting the opportunity to use real-time context and location data to drive incremental value for brands and retailers,” it said.
“That in-store activity is then connected to retailers’ e-commerce systems, creating a more complete view of how customers shop across the physical store and online.”
- The result:
- For retailers: larger baskets, retail media revenue, reduced shrink, and fewer out-of-stocks.
- For customers: relevant recommendations, timely savings, fewer missed essentials, and a more fun, intuitive shopping experience.
David McIntosh, Instacart’s Chief Connected Stores Officer, said Physical AI is intelligence that perceives and responds to the physical world in real time, directly at the point of interaction – in Instacart’s case, on the shopping cart itself.
“We’re deploying a continuous learning system – powered by real-world data, captured and processed on-device at the edge – that builds the best possible understanding of customers, the store, and the shelf as a shop is happening,” he said.
“Brick-and-mortar stores are highly complex environments – from inconsistent Wi-Fi to shelves changing in real-time to people shopping in groups. Serving customers well in this environment requires a fundamentally different approach – one that requires both real-time analysis and low latency.
“Caper Carts address this through sensor fusion – combining basket-facing cameras, a weights-and-measures certified scale, and location-tracking systems – all processed on-device with NVIDIA Jetson. Accurate basket understanding is only part of the picture. What makes our Physical AI system truly differentiated is the combination of signals we’re capturing simultaneously: cart location, what’s on the shelf in front of the customer, what a customer is actively scanning into their basket, and how they’re interacting with the Caper Cart screen.
“Getting all of them right, in real-time, is what enables relevant and timely influence. This is where the in-store experience diverges from traditional analytics: rather than analyzing a shop after the fact, we engage customers in the moment through on-screen recommendations and gamification, shaping decisions as they’re being made. This rich view and ability to influence customers is unparalleled.
“Underpinning all of this is the flywheel, built on our more than 1.6 billion lifetime online grocery orders across more than 100,000 store locations. Our recommendation systems were trained on that scale of online data, and we’re now combining those signals with the millions of real-world sensor inputs from Caper Carts we’re capturing daily to deliver a continuously improving system. In-store data makes online recommendations better, and in turn, online data makes in-store recommendations better.”
McIntosh said the results retailers are reporting are well beyond the one per cent figure that’s gotten attention: retailers are reporting double-digit percentage basket lift from Caper Carts.
“It’s worth clarifying what that one percent actually refers to – that’s the lift driven specifically by the “Got everything you need?” prompt, a single feature that uses real-time cart location and basket context to surface a timely reminder. On top of that, another recent ranking improvement leveraging our online data signals and systems drove more than 1% in basket lift. Caper Carts are driving meaningful impact for retailers,” he said.
“What makes that additionally compelling is the flywheel behind it. Everyday, Caper Carts are capturing millions of sensor inputs daily – what’s going in and out of the basket, how customers move through the store, how they interact with the screen – and that data is continuously improving our models. We’re unlocking the power of combining cart location, omnichannel purchase history, and a live digital screen inside the store. The system gets better everyday.
“We’re also seeing ads on Caper Carts perform at engagement rates similar to online. This is a meaningful signal for CPGs who have historically had limited ability to reach and influence customers at the shelf. Brands are excited because this is a brand new surface: a digital screen in the hands of a customer, in the moment they’re making purchase decisions, informed by what’s actually in their basket and where they are in the store. Ads on Caper Carts are already live at retailers like Wakefern Food Corp – which currently has the carts live across nearly 20% of their co-op member stores.
“On out-of-stocks: the side-facing cameras on Caper Carts give us a near real-time view of the shelf – what’s there, what’s not, all while not having to rely on outdated planograms. That alone is a step-change for retailers who’ve historically relied on manual audits or stale data. We recently introduced AI Solutions, where agents don’t just detect out-of-stocks, it acts on them. For example, we might identify that a brand of sourdough bread consistently sells out across six locations in a store. An AI agent could automatically surface that demand signal, negotiate with the vendor on the retailer’s behalf, and coordinate setting up a seventh placement to capture that unmet demand. Stores become not just observable, but optimizable.”
McIntosh said Caper Carts are designed to fit into a retailer’s existing ecosystem rather than require them to rebuild around us. That means native integration with a retailer’s POS system and loyalty programs, so customers can seamlessly access their deals, coupons, and personalized offers as they shop.
“Additionally, Caper connects seamlessly to retailers’ e-commerce sites via an integration with Instacart’s Storefront Pro, which powers e-commerce for more than [380+] retailers. Caper and Storefront Pro are designed to work seamlessly together as a unified platform that spans in-store and online, sharing signals in both directions.. For example, users can sync their shopping list they create online to Caper, and will be reminded if they forgot something. We recently announced that we’ll be introducing Cart Assistant, our omnichannel chatbot shopping companion. This is rolling out initially with Sprouts and Kroger, and will be available across Storefront Pro and Caper Carts to create a continuous, personalized experience no matter how a customer chooses to shop,” he explained.
“Caper Carts can also drive adoption of a retailer’s ecommerce platform from within the store. Customers can access items they’ve purchased in-store while shopping online and retailers can promote offers to sign-up for online grocery delivery directly on Caper’s digital screen – ultimately driving customer acquisition across both channels.
“On the operational side, we’ve been deliberate about designing Caper Carts to be easy for store teams, not just customers. Cart Manager gives associates a real-time view of basket activity to support audits and reviews without disrupting the customer experience. Beyond Cart Manager, we’ve designed Caper to be operationalized as easily as possible – for example, Caper supports stackable charging, enabling carts to be charged while nested together in a row, which eliminates the need for associates to individually charge dozens of carts every day. The goal is a system that modularly integrates into existing store operations and is genuinely easy to run.”
McIntosh said Caper Carts were built with privacy in mind. Caper Cart’s cameras are focused solely on recognizing products. Instacart also helps retailers use their data to improve the customer experience and make grocery shopping more personalized.
McIntosh said there are only a handful of moments to gain a customer’s attention and influence what goes into their basket. Timing is incredibly important – if recommendations are served too late or are irrelevant, customers will start to ignore the prompt.
“This is precisely why Physical AI is so important. Delivering a relevant recommendation requires knowing where the customer is in the store, what’s actually on the shelf in front of them, and what’s already in their basket. If any one of those signals is wrong or missing, the recommendation breaks down. A prompt for an item that’s out of stock, that the customer already has, or that’s in a different aisle entirely isn’t helpful – it’s noise,” he said.
“By leveraging real-time location, basket data, and our online grocery data, we’re able to deliver precise recommendations that feel useful to customers instead of acting as a distraction. For example, we’re seeing simple prompts make a difference – “Got everything you need?” right before checkout is driving nearly a one percentage point lift in basket size on average. Using real-time context can create a win/win/win for retailers, brands, and customers.”
More from Retail Insider:
