What is visual search – and why should ecommerce businesses care?

Graphic of man standing in centre of human eye with visual search telescope

From influencer culture to shoppable ads to video content, the world of ecommerce is increasingly image-dominated – and search is no exception. Google, Pinterest and Bing have all now introduced visual search tools, and leading marketplaces including eBay and ASOS are following suit.

But what is visual search, and what can etailers do to prepare themselves for the snap-and-search revolution?

What is visual search?

First off, it’s important to note that visual search and image search aren’t the same thing. Image search is nothing new; Google Images currently accounts for 26% of all search engine queries. On the other hand, visual search remains a burgeoning technology.

The difference between the two is in the type of information input. While Google Images relies on textual metadata to understand image content, visual search allows a user to search for an image with another image. To summarise:

-> Image search Text goes in, relevant Image results come out.

-> Visual search Image goes in, relevant Image/Text results come out.

The accuracy of a visual search result depends on the engine’s ability to decipher the information in an inputted image pixel by pixel, then translate those pixels into data that can be processed by an image or product search tool.

This translation process requires huge amounts of data and advanced machine learning to work effectively. But why machine learning? Information processing that humans take for granted can prove immensely difficult to replicate digitally. For example, imagine that you are shown a picture of a chair. You recognise immediately what you’re looking at – but a machine doesn’t have any concept of what a chair is, let alone how to identify one from a collection of pixels.

To overcome this issue, visual search tools have to be ‘taught’ what various objects look like. The only way to do this currently is to feed thousands of related images into the engine, until – through machine learning and analysis of commonalities – the engine develops the ability to accurately recognise and identify objects.  

Why should etailers be particularly interested in visual search?

Visual content is obviously an essential prerequisite for selling online; in fact, research from eBay Labs shows that images increase buyer attention, trust and conversion rate. But visual search has the potential to offer an entirely new pathway to product discovery.

Let’s say that Customer X sees an Instagram fashion influencer sporting a desirable watch with a green strap. They want to find similar products online, but ‘watch with green strap’ isn’t necessarily going to give them the results they want in a standard text-based image search; it’s too vague. So they save the Instagram image and feed it into a visual search tool.

This tool, after processing the visual information in the image, correctly identifies its components (including the watch and its core features, such as colour and material). It presents the consumer with a range of relevant, similar products, increasing the likelihood of purchase. The brand’s organic site traffic, product discovery process and conversion all receive a boost, and the customer gets their nice green-strapped watch – a win-win all round.

In fact, this service may come at precisely the right time. Research from Slyce indicates that 74% of ecommerce customers believe text-only search is insufficient for finding the products they want. The same paper shows that visual search holds particular possibilities for apparel, homeware and electronics vendors; on Slyce’s visual search tool, the three segments account for 39%, 31% and 24% of searches respectively. 

Image-based search: a sign of the times?

In many ways, visual search is just another symptom of the increasingly visual world we live in. With social commerce and celebrity influencer culture becoming more and more of a factor in consumer behaviour – especially that of younger consumers – visual search offers a way for etailers to bridge the gap between aspirational lifestyle content and their own product offerings.

Research from Accenture indicates that 44% of Gen Z consumers cite social media as a popular source of buying inspiration, while 37% have increased their use of social media for purchase decision-making in the last year. The ability to feed aspirational social images into a visual search engine and locate relevant products at the click of a button is clearly important to this audience. As a result, visual search could represent a great way for ecommerce businesses to engage with a crucial demographic.

A valuable tool for improved UX?

Large (and seasonally rotating) product inventories are a fact of life for many online retailers. While this comes with obvious advantages – as demonstrated by Amazon, whose vast product range has made them a formidable shopping destination – it also poses issues.

Perhaps the most annoying of these is ‘the discovery problem’, which arises when a customer is faced with simply too much choice; overloaded with options, they make no choice at all. This analysis paralysis means that, the more options you provide, the lower your conversion rate actually falls. This is a particular problem with mobile shoppers on the move, who are more limited in terms of both browsing speed and attention span.

Undiscoverable products can also irritate ‘spearfishers’ – that is, customers who enter your site with a very specific idea of what they want. Forced to sift through hundreds of products in search of one specific item, these shoppers quickly become frustrated, prompting many to eventually leave your site.

Visual search offers an antidote to this problem. Time-poor customers can narrow their options down quickly and easily by searching with a specific image, rather than through more traditional, time-consuming methods of text search or navigation through category and sub-category pages. The result is increased conversion as well as a hugely improved user experience.

How can I make use of visual search?

Visual search is still a relatively new and incredibly data-intensive technology, and the existing tools rely on being fed extensive visual datasets to ‘learn’ what they’re looking for – with even a comparatively low resolution 640×480 picture containing more than a million data points to process. It’s for this reason that developing a proprietary visual search tool is not a viable option for many ecommerce businesses, simply due to the level of development expertise and technological manpower (as well as cost!) involved.  

However, off-the-shelf third party visual search solutions – for example, Clarifai – do exist and may be an integration option to explore for some online retailers. What’s more, the information required to ‘train’ an AI-powered visual search tool is becoming more and more available – for example, in the form of the Zalando Fashion-MNIST dataset.

So, while it’s likely to be some time before visual search is widely integrated across the majority of ecommerce sites, there are a number of steps online retailers can take to mitigate the UX issues addressed by visual search. These include:

– Intuitive site navigation, to facilitate the user journey.

– A strong on-site (text-based) search functionality, allowing customers to discover relevant products quickly and easily via a search bar.

– Introduction of targeted interactive ads – for example, shoppable posts on Instagram, leveraging the current ‘social commerce’ trend to sell direct through online social media outlets.

To find out how Quill can help you deliver better customer experiences through optimised multi-channel content, get in contact.

More posts from the blog


[Infographic] Ads vs. information in the age of consumer cynicism

Find out more

Webedia Group acquires Quill – the award-winning UK content production pioneer

Find out more

On the rocks: designing content for hard luxury ecommerce

Find out more

Get in touch with the team

Contact Back to top