Reprinted from our friends at Andertons Music Company, an EasyAsk customer in the U.K.
Originally posted on January 28th 2016 at 17:52 by Andrew Chart from Andertons
Part of our “Behind the Scenes” series
No no no no no! Search Laboratory: Avoiding having to tell our customers this!
Last month we wrote about making use of dedicated site search technology by EasyAsk to improve the experience for our online browsing community. We also manually optimise our site search to ensure it continually returns the best results for you.
Behemoths of search like Google & Bing are incredibly good at interpreting users’ questions. We figured that all we needed was billions of dollars and a crack team of – say – nine hundred people, and we too could build ourselves an infallible
search engine that could basically read peoples’ minds.
Unfortunately, the combined resources of the web team piggy bank and the sales office swear jar (someone has clearly been neglecting to pay their dues into that swear jar) only amounted to £4.60.
We needed a humanised process to continually keep our site search performing exceptionally. The Andertons Search Laboratory was born.
What is Search Lab?
How many times have you been on a website where you were left frustrated by the search experience? Maybe you searched for something and received no results, only to stumble upon the thing you were looking for later whilst trawling through the category
navigation?
It might feel something like this:
We devised Search Lab to avoid being ‘one of those’ sites.
Every Tuesday members of the Andertons e-commerce technical & content teams get together in what you might as well imagine as a long, white, strip-lit room with test tubes, gerbils and an arsenal of fire extinguishers in all colours of the rainbow.
Because, safety first.
O.K. well, it’s not entirely like that… For historical reasons that I can only speculate about, I wasn’t allowed to buy an office rodent, and we don’t conduct our sessions in lab coats (although I must admit I’m seriously considering it).
Search lab is however where we act as data scientists and observe lab rats *cough* I mean… real user activity to help us improve our search engine for the masses.
Searches with no results
The most obvious optimisation is to look at is the search phrases that return zero results, and see whether we can fix them!
Our search engine, EasyAsk, provides us with a suite of analytics that lets us know words which real people searched for that provided zero results. Either we don’t sell the item/s, or we’ve ballsed something up and users are searching for
something we do have, but not seeing it come up!
Two years ago, we were seeing two phrases being searched-for again and again. They were for two brands we did not sell at the time: “Schecter” & “Peavey”.
Armed with our search terms report, and knowing anecdotally that customers had often requested Schecter & Peavey in store and via email, we regularly fed back to management about the noise. In 2014 the powers that be struck up a deal to start
stocking some Schecter & Peavey gear in store and online.
Popular Searches
Sort of the opposite of no-results searches, there are search phrases that come up as sure as the sun. We like to make sure that the results for these searches are as useful and efficient as they can be for users.
Our most popular search terms are broad. Brand terms like ‘Fender’ or ‘Gibson’ and popular models like ‘Telecaster’ or ‘SG’ are always top of the reports. I think there are two issues worth solving here:
- We have a lot of products for these terms; how should we order the results? When all else is equal (a telecaster is a telecaster is a telecaster), we generally show you results that we think are the ‘best deals’ at the top. If we’ve
got a great price on something due to a bulk buy, or we’re offering 18 months 0% finance on an item — that tends to float to the top. - Lots of users are typing these terms so it may be helpful if we put some of them into the navigation menu. The only trouble is working out which phrases deserve a spot! Typically we’ll try a few, conduct tests, and look at other
data to prioritise.
Relaxing results
In the video above, the user changing his search from “semi-skimmed milk” to “milk” is an example of relaxing search terms in order to provide at least some results, if the initial search returns nothing.
As the video demonstrates this can be almost as frustrating an experience as providing no results at all. To help, we also investigate searches that only returned results after one or more terms were removed.
The added bonus for our lab technicians is that these are normally the hardest imperfections to fix!
We currently don’t list the Mapex Black Panther Nomad snare drum for sale. If a user asks for “Mapex Nomad” (a steady trickle of people do), a drum specialist would know that that person was trying to buy a snare drum. Computers
aren’t this clever.
By default, what the search engine would choose to show you is all the results for “Mapex” which ranges from hardware to full drum kits! To add insult to injury, the message we use to tell you that we didn’t understand you makes nearly-no
sense…
Please, take a seat whilst we find that for you…
We do sell other snare drums from the Black Panther series, as well as specialist drums by other manufacturers. If you’re lucky you might happen upon a great alternative later, but the journey would have royally sucked.
To make any attempt at solving this, we need to help EasyAsk understand the context surrounding search terms (e.g. that “Mapex Nomad” could fall back to “Snare Drums”). There’s a high likelihood that EasyAsk will have no notion of the context of our
industry-specific terms, so we’ll need to ‘teach’ EasyAsk about the term. This is slow work and a problem that I don’t think we’ve cracked yet… We’re working on it.
Scientists vs. Robots
Gahhh. It’s a dangerous web out there
The land of search is rife with evil robots (and not the Phil X amplifier kind). Our reports show a ton of queries that are being made by web crawler software. Well… either that or the person that searched for this is some kind of cyborg.
Version:1.0 StartHTML:0000000203 EndHTML:0000002215 StartFragment:0000001974 EndFragment:0000002155 SourceURL: ://wwwgooglecouk/search?clientsafarirlsenqboss dd-6ieutf-8oeutf-8gfe_rdcreioz9hvse6b47j8wfnnigabw
We have to sift through bad data to troubleshoot real user problems. The good news is that Search Lab is helping us get there. Simply by knocking heads in the lab, we came up with a great way of cross referencing multiple data points
to work out which queries are real human users and which are robots (or nefarious cyborgs).
I won’t share exactly how we cleanse the data because it’s a bit boring and — we thought — rather clever, so I’m loathe to share! But what it means is that we can clear the smoke and improve search for real users,
all the while heroically defeating robot master races with our science.
Experiments coming out of Search Lab
Finally: looking at search data sometimes turns up a behavioural aspect of users that we hadn’t thought about before. We devise experiments to test new features that complement the behaviour we observed in search lab.
We noticed that returning users that had visited Andertons.co.uk recently were more likely to use the search on their second (and subsequent) visits. A little more data-digging reveals this:
A lot of people use site search to quickly get back to a product they have viewed before.
Whilst not exactly a mind-blowing revelation, we had hard data to prove this behaviour; we also had an idea that could help these users.
Sticky Search Term
We wrote a little code that allowed the user’s browser to ‘remember’ the last phrase that the person searched for; when they come back to the site we automatically populate the search box with their last query.
Welcome back, fire extinguisher enthusiast.
We felt this made sense for Andertons. With a high-value purchase like an instrument, people take their time and continually view the same product a number of times before they decide whether or not to purchase.
Our change meant that the product was two clicks away, and users would not need to re-input the search term with their keyboard — particularly useful on mobile devices.
To test the efficacy of the sticky search term, we tracked the usage of the feature by recording whether or not the user modified the search term that we placed in the box before they hit ‘go’. We also made a note of the type of page a user was on
when they searched. Here are the results:
As you can see, the sticky search feature was used by close-to one third of the people searching from the homepage, but by a smaller proportion of people on other types of page. Again, this makes sense as you’re more likely to use the homepage as
a starting point when your intention is to return to a previously-viewed product.
We had helped 30% of people searching from our homepage get to where they wanted to go more quickly, but added inconvenience to the other two thirds (and 75% of searchers across the site as a whole) because to perform their searches, rather than just
tapping the search box and beginning to type they now had to erase the old (sticky) search term too.
But that’s cool. As a result of the experiment we intend to keep the feature but modify it as follows:
- Only recall the sticky search term on the site homepage. Even on the product page where users may use the feature to go “back” to the search results, we felt the usage was low and that the back button was intuitive enough.
- Highlight the whole search term as soon as the user taps the search box. This ensures that the 70% of users that do not want to re-use the search term on the homepage can simply start typing their query as they would have done if
the box was empty.
Honestly — we’re only making you our guinea pigs to make you happier in the end!
Search lab is still a relatively new concept for our team. We have always paid attention to what our users are searching for, but never have we been so empirical and iterative about improving our site search experience.
By way of regular reporting, analysis using EasyAsk’s tools, and action by the content & tech teams, we hope to make the Andertons site search stand out. It should be effortless and enjoyable to discover music gear at Andertons.
Are you having a frustrating experience with the Andertons search engine? Tell us how you think we could improve the search in the comments and we’ll discuss it in our next Search Lab!
Andertons Music Co. is a UK based, rock ’n’ roll music instrument retailer which has served the community for over 50 years. The brand is known by musicians worldwide, with videos by Chappers & The Captain reaching nearly half a million YouTube
subscribers.