As cameras become an intrinsic part of our everyday lives, their use is becoming more commonplace, with AI trained to analyse moving images and entirely new products and opportunities emerging from them. For example, we can already use our phones to visually search for products in apps like IKEA Place simply by pointing our cameras at items we like and waiting for the app to find the most ‘matching’ products from the IKEA catalogue. In the near future, the visual intelligence software in our phones will be able to recognise items and put them in our virtual shopping bags (with the actual items being compiled for collection or delivery later).
AI in apps like IKEA Place enables you to visually search for items you like and wait for the app to find the most matching products.
This isn’t especially far-fetched. Consider the Amazon Go store, in Seattle, which allows shoppers to take what they want and leave without paying. A network of 100 cameras equipped with image-recognition software tracks each shopper, connecting them with their phone and Amazon account, and charging them for every item they put in their basket. There are three Amazon Go stores in Seattle and one in Chicago, and the tech giant plans to open 3,000 more by 2021. Similarly, the Chinese chain BingoBox has 400 ‘smart supermarkets’: unstaffed grocery stores that enable you to enter via QR code or WeChat account, scan items you want to buy and pay for your goods with AliPay or WeChat Pay.47
The technology underpinning both the Amazon Go and BingoBox stores is highly sophisticated, and—as with all the innovations outlined in this report—the team developing it isn’t working in a vacuum. Many major corporations and leading university computing labs are tackling the challenge of visual intelligence. For example, Microsoft Cognitive Services powers Picdescbot, a Twitter bot that describes pictures (with varying degrees of success). And Google Lens, which was launched in June of 2018, tries to classify any object it sees through the camera lens of a smartphone.
A speculative proposal for BingoBox storefronts, created by Zhou Pu Fang.
Having successfully classified the objects you are pointing the lens at, your phone’s assistive technology—or, in other words, software which makes your daily life easier—may one day be able to interpret and analyse those images and determine your individual taste, style or preferences. Indeed, visual intelligence is only half the story. When it comes to providing personal experience, assistive intelligence counts too.