I’ve just got back from 10 days holiday in a farm 5 hours drive from London, with plenty of hens, goats, horses, ducks, rabbits, cows and sheep (toddler paradise really – well my toddler thought so anyway) but no internet access. So please excuse the tardiness of, this, my third post!

For today’s post, I’ll focus on understanding better how internal search works and how important it is that it works effectively so that you don’t end up losing business and visitors and sending them on a galaxy quest. We don’t want our visitors to be wondering how to get to what they want or if they are there yet. The Eisenberg brothers tll us that 50% oif nternal searches result in a failed search which is a considerable number of dissatified visitors and customers to your site. So, this really is of crucial importance if you want to improve site experience and ultimately the effectiveness of your site and turning visitors into customers.

Internal search refers to the keywords that people use while exploring your site (not the keywords they use on the search engines such as Google).

For many websites, in particular holidays, recruitment, publishing and large retailers, internal search can be the most important and used feature on the website and can account for 50% of all pageviews on the site. Obviously, that isn’t the case for all sites as small sites don’t need or have internal search in most cases (so we’ll ignore these today).
In order to be able to analyse internal search keywords at all, you’ll need a paid-for analytics solutions – though not necessarily an expensive one, as both Clicktracks and Indextools can combine internal search parameters with visitor segments. Google Analytics does not currently provide any data at all on internal search keywords (don’t be confused with the “keywords” heading under traffic sources, these are the words visitors used on search engines to arrive at your site).

How do we assess how well (or not) internal search is working?
1. What do people search for and do any keywords stand out?
2. What searches results in failed searches and what proportion are failed searches?
3. Let’s do some segmenting

1. What do people search for?
Rather than getting too bogged down with the exact words and words and common misspellings that visitors use (although interesting and at times surprising), it is better to start with much broader strokes and then drill down and do segmenting later. By which I mean, how many visitors as a percentage of all visitors to your site use internal search and which are the most and least popular keywords over a representative time period, 2 months for example.

If for example, less than 1% of visitors use internal search and this is large recruitment site, then make sure that search button is placed in an intuitive and easy to find place for visitors. If the internal search area was moved, would it make a difference to the number of visitors who search? On the other side of the scale, if for example too many visitors (over 10% – it really does depend on the site) are searching on the site, then your site’s products and/or services are not easy enough to find on the site.

If the most popular keyword search is a noticable percentage of all searches, then this clearly signals that this should be clearly displayed on the site or on a site navigation bar. If a popular keyword is not even an item or service offered by your site, this is a clear signal that this should be something you either should be offering or linking up with someone who does (affiliate marketing perhaps?). Again, this seems simplistic but it is so easy not to notice how important popular keyword searches can be! When internal search is working effectively, we should not expect to see any one search keyword as a noticable proportion of the total – expect to see more of a long, long tail (lots and lots and lots of different slightly obscure keyword searches).

Here is how to get your long tail of internal search keywords:
First, create a report of all the internal search keywords and unique visits for a 2 month period and upload to excel. Grab all the keywords and visits into smoothed line chart with data points (so that you can easily see the keywords that stand out). See my chart below:
Then try making a list of the keywords that stand out. These are words that need to be looked at carefully as they will be benefit from being presented on the site in an easy to find way so that your visitors do not always need to search for them.


2. Which searches result in failed searches
A failed search is when a visitor doesn’t find what they are looking for. For example, keyword searches on products or services that you do not offer would be a failed search as would a time sensitive product or service that is not available within the results of the keyword search made. For example, visitors that click on the back button after making a search would be classified as failed/frustrated searchers. To reduce failed searches make sure the site reflects at the minimum easy to find information on the more significant failed search keywords (this is just a quick fix and not the solution if only information is presented but is a needed first step until a good solution is found).
Then look at the percentage of all visitors that have a failed search. In addition, you can create a visitor segment where the search results page is also the exit page and compare this against all failed searches to see how many “failed search” visitors, leave the site immediately.

3. Then we segment, to confirm our suspicions and insights
Assuming we are using an analytics solution where we can label visitors segments with specific keyword searches, we begin to drill-down further. If on a recruitment agency site, a noticable search is for “web analyst”, we can see which were the most popular pages they visited before searching. From this, we could learn that they visited the “marketing jobs” page and the “Web jobs” page and as a result of not being to find what they were looking for, searched for the term “web analyst” and subsequently left the site. Therefore, it would appear that both of these jobs page would benefit from having information about web analyst jobs on them (until the recruitment site started posting web analyst jobs that is).

We can segment against new versus returning visitors, time spent on site, by navigation path etc. For example, we can see how these visitors came to the site in the first place by looking at search engine keywords. If there is a noticable percentage of visitors who arrived at the site after having searched for “analyst job” or “web analyst job” on search engines, then it is clear that the hopes and desires of visitors coming in from the search engines is not being met by the site – as well as a PPC (pay per click) overspend on keywords that are resulting in a high number of failed searches and exits from the site.
Some final thoughts – this really is all about marketing, effective marketing (Kotler anyone?).

The key is to reflect on the site what visitors are looking for, in a holistic and thorough way. Not just add-on a quick note on one of the pages that this site does not currently offer web analyst jobs (although in the short-term, a quick fix is better than nothing), but have a think about the big picture, what the site is trying to achieve and use this information about what visitors want and aren’t getting to improve and add to the service offering – in this case, widen the remit of the recruitment agency itself and add web analyst jobs to the site.

I really do (I promise) welcome your agreement, disagreement and opinions, so please do share some of your thoughts by commenting on this post.