Logs - the golden resource

Written by Dave Collins, SharewarePromotions Ltd.
http://www.sharewarepromotions.com

A website may be many different things to many people, but to the company selling a product or service online, it is everything. The company's website is probably their store, brochure, means of contact and the beating heart at the centre of their sales campaign, all at the same time. Yet many make the mistake of setting up their website, modifying and updating it here or there, and then often simply leaving it alone.

If you happen to regularly visit a small, local village shop, where the nearest competitor is some 20-30 miles away, you'll probably notice that very little changes with time. But watch what happens if and when a competitor appears - you're almost certain to see a sudden flurry of activity, followed by an endless cycle of tweaking, modification, improvement and expansion. On the internet, your competitors are never further than a click away. So, what should you do about it?

Your web referral logs are one of the most useful sources of information that you have access to. Amazingly, many businesses still ignore them entirely, or spend only the smallest amount of time in giving them an occasional quick glance.

Most web hosts provide some form of online web statistics, such as Analog or Webtrends, but while these are certainly quite useful, they often only provide you with the chance of scratching away at the surface. Many web hosts are also able to give you access to some form of raw referral logs, that can then be downloaded and analysed locally.

How to analyse your data
While you can analyse the logfile data manually, it is incredibly slow and time consuming work, and you're well advised to use one of the many software packages that already exist for this purpose. While there are more than a few free or low cost options for doing so, I personally recommend using one of the more powerful options that exist. There are few sources of information likely to be of such benefit to your business.

My own favourite application is 123LogAnalyzer by ZY Computing, at www.123loganalyzer.com, as the software presents the information in the clearest way that I have ever come across. It also includes the ability to apply a series of filters, making it easier to locate and isolate the information that you may be looking for.

Whatever you choose to work with, be sure that it contains the tools that you need to really get to grips with this most important of data sources. It's probably also a good idea to make sure that it is compatible with your specific log file format, as certain applications may not work with certain file types. Also bear in mind that some may process the online files directly, some may work with locally stored files, and others may even download and unzip the data for you. Choose wisely.

Once you have the application installed on your machine, you're almost ready to use it. But before you do so, a small amount of planning is advisable. Start by making a list of the main pages in your website, and prioritise them in order of importance. We all have pages on our site that we hope the visitors will see and notice, so work out how you rate and rank their respective importance. Once you've done this, it's time to put your software to work.

Let's start with the basics. There should be some sort of summary in the report, so begin by having a look through this. Unless you're selling advertising on your website, the commonly requested facts such as the number of hits or page views aren't actually that important. Instead, let's look at the figures that really matter.

One example is the average number of page views per visitor. If your website contains thirty plus pages of publicly accessible information, yet the average visitor only looks at one or two pages, then you're probably doing something wrong. There's no hard and exact formula here, but you should be able to judge for yourself what feels right and what doesn't.

While you're looking through the summary, keep an eye out for other important information such as bandwidth. Many web hosts either cap or charge your bandwidth beyond a certain limit, and be honest… do you already know how much you're using? It's probably a good idea to find out before you receive an unexpected bill in the mail!

Get into the habit of really reading through what's in front of you. Don't skim through it, looking for the "relevant" bits, as none of it should be completely ignored. Most of the information is truly worth its weight in gold.

Have a look for some kind of activity breakdown, often analysed in terms of page views per day, by time, by hour, by day of the week and so on. While much of this information may not have such an impact on you, there should still be some useful data here. For example, you'll almost certainly notice some kind of relationship between the volume of traffic and the day of the week, which may be useful when choosing a new product/version launch, announcements or even site maintenance and scheduled downtime.

Do you know where your visitors are going?
The resources accessed section of the report will contain data on page views - of obvious importance and significance. The first thing you should do however is to compare your own list of "important" pages with the facts presented in the report. Are most of your visitors actually going to these pages at all? If not, why not?

Chances are that you can do something about this. After all, you control what's on your site, and how it is presented to your site visitors. Maybe they're not coming to these pages because of how the links are presented, or perhaps they're not even seeing or finding the links at all. Dig deep here, as the rewards are more than worthwhile.

123LogAnalyzer also lets you see the most popular browsing sequences, meaning that you get to see the most common paths taken through the pages on your website. This is staggeringly useful, as rather than having to guess what the visitor may do, you can actually see this mapped out in front of you.

If you use any form of page or URL tracking (for example pointing individual advertising campaigns to specific pages), then you may well find that different sources have different usage patterns.

The referral stats is another key part of the report that you want to pay a great deal of attention to, as it will allow you to see exactly where most of your traffic is coming from. Be prepared for some surprises however, as you'll invariably find that you have severely misjudged the amount of traffic that some sites send your way. If you're contemplating advertising, this information may give you some very useful pointers.

However, when it comes to the download and shareware sites, don't forget that some of them host the actual file themselves. Even though they may send very little traffic your way, they can still account for a significant number of downloads. These include TuCows, Winsite, Simtel and others.

Do you know where they're coming from?
The Search Engines section of the report will give you a good overview of how much traffic you're getting from this source, but be aware that many web analysis tools often mix-up and confuse individual referrers and search engines. For example, if you're listed in the top two or three positions on Overture, your listing should be featured on a variety of other search engines as well. Yet visitors who arrive at your site from these places may not be shown as coming from Overture.

Most log analysis software will also dissect the Browser and Operating systems of your visitors, although it's worth pointing out that some of this information may be slightly inaccurate, due to the way that some users choose to cover their tracks. But there still may well be a small diamond or two in there. If for example you were to discover that 10% of your visitors are using Windows ME, wouldn't now be a good time to make sure your software works under this operating system? And if your website makes use of technology that can only be viewed in certain browsers, how many of your visitors may be missing out?

Most of the log analysis tools also contain some sort of technical and error summary that may show you any missing files. You may well find that there are some sites out there linking to a page on your site that no longer exists. To get around this you could either copy an existing page to this address, or notify the site that the URL has changed. Either way, stop throwing visitors away!!

Two final hints. First of all, whether you're analysing the data locally or using an online solution, you should make sure that you keep this data safe and secure. You never want it to fall into the hands of anyone outside your company, let alone a competitor. Second, if you go to the trouble of downloading your logs, then never, ever delete them. Saving the reports is not enough, as you may at some point wish to view a given time period in one report, or to look at long term trends. Back them up somewhere safe, but don't throw them away.

The information contained in your web referral logs isn't just useful, it's invaluable. Without this data, you are blind and utterly clueless as to what works and what doesn't on your website, where your visitors come from, where your visitors are choosing to go, and what they may be missing. Your visitors are leaving a constant trail of powerful information, that under no circumstances should be ignored. Be seen, be sold.