Doorways, hallways, robots & resources

Written by Dave Collins, SoftwarePromotions Ltd.

In the previous article we looked at some of the more basic techniques that can be employed to raise your visibility in the search engines. We looked at the placement of your main keywords in links, the use of header tags, <ALT> tags, and multiple <TITLE> tags. The most basic and important rule is to keep it relevant. Don’t go trying to lure in traffic under false pretences; both the visitors and search engines themselves will take a very poor view of this sort of trick, and you’re not going to help yourself in the long run at all.

There are also some more advanced techniques that can be used, that require a considerable amount of time, and sometimes a little caution too. If used correctly, they can have a significant impact on the number of visitors to your website, and the fact that these visitors will be targeted can only improve your sales.

What Are Doorways and Hallways?
Doorway pages – by now you’ve probably heard the term, but may not understand what they are. A doorway is simply a web page tailor-made for a specific keyword on a specific search engine. As each engine has its own rules and regulations, each has to be setup differently. Example – file management software could have separate pages for file, viewer, graphics, backup, ZIP, GIF etc. Each of these could then exist as different versions for each search engine – a lot of work, but well worth it. Rather than having a general “one size fits all” page for all aspects of your software on all search engines, you can setup these individually tailored pages, and customise each of them accordingly.

Hallway pages – another well travelled term, often surrounded by a lot of nonsense. A hallway is simply a page whose main reason for existence is to link to other pages, in order to help the visiting spiders. This can be extremely useful if a search engine limits how many pages can be submitted at one time – time saving and efficient. Again, the aim of the game should be to make the page friendly to the visiting search engine spider and site visitor alike.

robots.txt – if you don’t have this file installed on your site, you may find many references to it in your logfiles and webstats. The mysterious robots.txt is a simple text file that tells the search engine’s spiders which pages and/or directories on your site to avoid. For instance you may not want your private pages, cgi-bin, stored data or “hidden” pages to be listed on the search engines. The format of this file is critical, and if you’re unfamiliar with this file, it’s best to tread very carefully. Bear in mind however that this file could also be used by someone to find the hidden pages and directories on your site; so don’t use it as a security tool.

Frames and what to do with them
There are two common misconceptions about frames and search engines. The first says that you can ignore them, that the search engines work with frames just fine. The second says that the search engines simply cannot list framed sites. Both of them are wrong. Most of the search engines do index framed sites, but unless you’re careful, they will be unable to go much further than your front page. There are two things that should be done with a framed site. First of all, submit the main pages individually; don’t assume that the search engine’s spiders will find you. The second thing to do is to ensure that your page is setup correctly; that when visitors arrive from the search engine at that page, the frames are correctly linked in.

Another good solution to the frames problem is the <noframes> tag, which will allow the search engines to spider the page. While most users will be unable to view the content of the noframes tag, the search engines certainly will – a great opportunity!

At this stage, you have a fair amount of information to get you started, and probably more than enough work to keep you busy, should you decide to make the search engines a priority. A few last points however. Firstly, while there is an almost endless source of information about the search engines and “how to” guides, be very careful. Many of them are out of date, misleading and badly written, and even more of them are simply incorrect. As with all information, question the source carefully.

Which isn’t to say that there aren’t some very useful guides out there; there most certainly are. Even more useful are the sheer number of resources that are freely available. Some of the sites, including AltaVista and Excite allow you to check how many sites actually refer to you; the link-factor is an extremely important one for some search engines.

Finally, one last gem. have a very nice tool, that allows you to enter any keyword, and see how many searches there were for it last month. This is real high-quality marketing information, and anyone can use it. You can find it at

Next month’s article will cover some of the more important rules of the main search engines, and start looking at where, how, and how often to submit your site. In the meantime, time to set the ball rolling. Be seen, be sold.

Written by Dave Collins, SoftwarePromotions Ltd.