Submitting to the search engines

Written by Dave Collins, SoftwarePromotions Ltd.

Please note that due to the content of this article, much of the information may now be out of date. For historical purposes, I’ll leave it here… but follow the advice at your own risk!

It already seems like a long time ago since I began this series of articles on the search engines, and by now we’ve covered a fair amount of ground. We began with the HEAD and META tags of your site pages, and moved onto some of the basic rules, tips and techniques. We then moved up a gear and looked at more advanced ground, dealing with hallways, doorway pages, the robots.txt file and frames.

This means that we’ve more or less covered the necessary preparation work, and are now ready to jump in at the deep end, and submit our site to the search engines. This article aims to cover all the remaining issues; what to submit, where to submit it, the issues of time, and some of the many solutions that are out there to make life easier.

The obvious starting point is where to submit. The most commonly voiced view is some variation of the “95% of my site traffic comes from ten (or so) search engines, so there’s little point in aiming at the rest”. Many authors feel the same way when it comes to submitting their software to the software sites too. A listing on CNET’s may generate many times the number of visitors than Quality-Shareware produces… as Google will doubtless generate in the region of 500+ times the amount of traffic than will.

However, traffic alone probably isn’t the name of the game here. You’re not an excited teenager watching the counter on their homepage hit double digits, and ultimately you’re after one main thing – sales. My own view is that while a significant amount of time should be spent working with the main sites and search engines, neglecting all the smaller sites is a wasted opportunity. How long does it take you to fill out a form at these sites? Twenty seconds? A minute? If one copy of your software is sold as a result of doing so, doesn’t that make it worthwhile?

If 95% of your site traffic comes from ten search engines, then it makes sense that you should probably spend around 95% of your “search engine” time working on these listings. But don’t forget the other 5% – they can and will produce visitors, and visitors will hopefully lead to sales.

The next obvious question is which engines are the main ones; while ten people will probably produce ten different lists, most would probably more or less agree with the following. Please note that this information may now be a little out of date: AltaVista, Yahoo, Excite, Google, HotBot, GO, Lycos, Northern Light and WebCrawler.

Oh, and in case you’re puzzled, some of the search engines do use the same databases; submit to one of them and you should appear in the other. Excite and Magellan are one combination, as are Inktomi, HotBot, AOL, Goto and MSN. InfoSeek and the Go Network are another.

Now that we know where to submit, the next question is what to submit. Some engines will “spider” your pages, following the links from page to page, while others will only look at certain pages. While there isn’t the space to go into all the sites here, the main ones are as follows.

Yahoo can potentially generate more traffic than most of the other search engines put together. When submitting to Yahoo, you only need to send your root page. Yahoo use live personnel only (yes, real people!) to decide whether to list your site or not, and the data you fill out on their forms is critical, second only in importance to your site itself. Bear in mind that sites are listed alphabetically; other than that one element of control, you’re entirely at their mercy! The ultimate search engine – where content, and only content, is king. Two important points for Yahoo. Firstly that NO software should ever be used for their submissions. None. Many claim to be capable of this task, but none will actually work; the only way to be sure that the submission is carried out correctly is to do it by hand. The other point is that this is the one place where you should never use any tricks or techniques to get listed. No multiple submissions, no doorways, and no hallways. They won’t work, and will only damage (if not destroy) your chances. Don’t say I didn’t warn you!

AltaVista, on the other hand, will allow you to submit multiple pages to their site, and also spider your pages for you, so there’s no need to submit every single page on your site. Their time to index and list your site varies, and over the past six months there have been massive delays in processing new requests. At the time of writing this article, most new listings appear after two to three days. META tags are important here, and bear in mind that keywords are also case sensitive. Page popularity (i.e. how many sites refer to your own) is another important factor, and doorway pages were born for this search engine!

Excite and Magellan will allow you to submit your root domain only, and usually display new additions within two weeks or so. META tags are important, as is page popularity, and the important thing to realise is that only the initial page will be spidered. InfoSeek and the Go Network however, work in a quite different way. Each page must be submitted individually, and there’s no spider to scuttle across your site from link to link. A handy tip for InfoSeek is that fresh content is popular – so try to update and resubmit your main pages every month or so. New listings can take anywhere from two days to three weeks to appear.

HotBot are the single most frustrating engine to keep on top of, as their relevancy criteria seem to change with amazing frequency! The good news is that you only need to submit your main URL, and the listings may appear in a couple of days to two weeks. Well worth the effort though – our sites usually owe around 20% of their search engine visitors to this engine.

So now we know where to submit, what to look out for, and how long we may have to wait. The other commonly-asked question is how often to resubmit your site. The nutshell answer is that this varies from engine to engine, but a good rule of thumb is to make it at most a monthly event – no more than that.

The final hurdle is how to submit. Manual submissions are certainly the most effective, but are time consuming, and should be logged if you’re going to develop any sort of routine. The good news is that there are a large number of applications and services out there that can do the work for you. The bad news is that many of them are garbage!

In terms of services, you don’t need to look too hard too find the en-masse submitters, many of which are free. I have never and will never use one of these services, as I consider almost all of them to be little more than SPAM bait, and a total waste of time and money. However, the software that’s available for this sort of thing comes in two main forms; the majority are useless, and a handful are actually useful.

I personally use AddWeb Pro by CyberSpace HQ for all my en-masse work; it’s very fast, is updated regularly, and it does have a notable effect on traffic to my site. But I use it only for the link-lists and FFAs, not for the main search engines. When it comes to the main engines, there’s only one piece of software that is worthwhile; WebPosition Gold.

This is one superb piece of software, and can take care of pretty much every single aspect of your search engine work. But note that it is as far from a “push-button-and-watch” solution as you can get. Using it is extremely time consuming, but the range of functions and sheer power of the software is highly impressive. You also need to subscribe to updates of the software’s database if you’re going to use it to it’s full capabilities; in this case, it really is money well spent. Worth its weight in gold one might say…

Another consideration that you should be aware of is that almost all the search engines, lists, links-lists and FFAs require an email address of some sort. I would strongly advise you setting up a temporary address with one of the many free services that are out there for this purpose. I have more than a few setup for this very task, and routinely clear out many hundreds of “garbage mails” each month, none of which I would want in my actual inbox. In terms of the main search engines however, I’ve never encountered any difficulties with giving them my real email address, and most if not all have very clear privacy policies.

If you’re still not sure you can handle this work on your own, there are also some highly reputable services out there that can do the work for you. Be careful though – the majority really seem to be little more than one person with their own copy of the same software, with little more experience than you yourself already have. Use common sense when choosing them – ask for referrals of past clients, follow-them up, and ask what guarantees they can offer. Throwing a few questions to test their knowledge is also a good idea – these articles should give you some good ammunition! As with anywhere else on the internet, there seem to be more sharks than water. Don’t get bitten.

It goes without saying that there is an obvious correlation between the number of visitors to your site, and the number of sales. Assuming that your product is good, and your methods of capturing the attention of your visitors are honourable, you can only gain by increasing the number of site visitors. Whether you’re in a market flooded with competition, or have your own specific niche, you still need to make sure that you’re seen. Even taking into account the work involved in search-engine activities, it is still one of the most time and cost-effective methods of publicity available. Seize the opportunity; be seen, be sold!

Written by Dave Collins, SoftwarePromotions Ltd.