Google SEO

It’s been a few years since my last article on getting your site noticed on Google, Tips on Improving Your Google Search Engine Ranking. The situation hasn’t changed much, and the article is still relevant today. However, over these years I have learned a couple more things about what to do and what not to do on your website with regards to your site’s placement on the search engine results.

Hidden Text
Everyone knows that search engines (not just Google, but probably every one of them) frown on hidden text. Some people use hidden text to stuff keywords on their pages so that when people type those words on the search engine, your site will show, even though your page does not have any visible sign of those words.

I have never believed in hiding my keywords in hidden text or any of those cloak and dagger stuff, and hence I don’t practise such things. Imagine my surprise, when one day, a few years back, I suddenly found my pages on thefreecountry.com receiving a Page Rank of 0 because of hidden text on the page. (See my other article on Google Page Rank if you don’t know what Page Rank is.)

No, I did not compromise on my principles or anything like that. I did not try any underhand tactics to stuff keywords, etc. The reason is more prosaic.

At that time, I had partially converted thefreecountry.com to use CSS for its site design instead of the traditional <font> and <table>tags. In those years, there were still a few people using the old Netscape 4 which did not support CSS correctly. To accomodate those people, I tried to make my site degrade gracefully when it detected an old browser being used. I also put a message that contained the following text:
This page uses Cascading Style Sheets (CSS) to control its appearance. Since CSS is poorly supported on old browsers like Netscape 4.X, you may notice numerous oddities in the appearance of the page. However, you should still be able to read the information and navigate using the links.

This text was enclosed in a box that was hidden from view if you were using a modern browser, but visible if you used Netscape 4. I used the usual methods of getting different browsers to load different stylesheets as mentioned in my article How to Use Different CSS Style Sheets For Different Browsers (and How to Hide CSS Code from Older Browsers).

Unfortunately, Google’s spider detected that some text was hidden, and not realizing that it was also visible in other cases, automatically assumed I was employing some sort of skulduggery to get those keywords into its index. It then penalized my site accordingly.

When I realized that my attempt to help my visitors was going to cost my site’s position in the search engine results, I immediately pulled the message, and its accompanying CSS, from the site. Visitors using old browsers would still be able to use the site, but they would not receive any friendly explanation.

Although the days of Netscape 4 are long gone, the lessons learnt from that incident are still useful today. Undoubtedly many of you will not even dream of using keyword stuffing in hidden text and the like. However, as I found out from the above incident, Google’s search engine robot is far from intelligent, and its hidden text detection algorithm can bite legitimate webmasters too. If it even senses hidden text, even for an innocuous purpose like mine, your site is history.

Some ways in which you might inadvertently use hidden text (possibly without knowing if you use third-party scripts) include the use of drop-down menus for site navigation and browser-specific tips for users (like in my case above). I’m fairly certain that the Google programmers keep refining their engine so that the more common use of CSS to hide text for harmless purposes like navigation menus (and so on) are recognized as benign, but if you are about to use hidden text for some fancy design gimmick on your page, you might want to test it out on some obscure page on your site to see if the Google robot chokes on it. After all, always remember your site’s life depends on a robot, not a human. It may be obvious to you and other humans that the code is harmless, designed for some acceptable purpose, but a computer program can only follow a set of preprogrammed rules.
Getting Your Link to Appear on the First Page of the Search Engine Results

I get many queries everyday from new webmasters reading thesitewizard.com, asking me how they can get the link to their site to be shown on the first page of the Google search engine results. In fact, for some of them, getting their site to appear anywhere in the first few pages of Google’s results would already be a victory.

If you are in a hurry to get your site noticed, and have a budget allocated for your site’s search engine promotion, one instant way to get to the first page of Google’s results page is to buy advertisements (called Google Adwords) on your keywords. Before you summarily dismiss this, read everything I have to say first.

Google’s Adwords actually operates according to your budget – that is, you can spend according to what you can afford. It also allows you to place the adverts on precisely the keywords you would have wanted for your site in the search engine results. Unlike the normal search engine indexing procedure however, here you control the exact keyword which will trigger your site’s advertisement and its prominence. Of course it comes with a price. But if you’re doing a business, this is often the surefire way to get people to see your site in relation to something that they are searching for.

Many new webmasters I deal with don’t consider this as a viable option, because it involves cash-outlay. I agree that if you are merely running a personal site or a hobby site that does not receive an income, advertising this way is probably out of the question. You should then work on applying the usual search engine promotion tips to your website and hope for the best in the long run. However, if your site is a business site that yields an income, you might want to calculate the costs and benefits to see if you might actually earn more this way. Treat your expenditure on the advertisements like all other business expenditure and calculate your return on investment (ROI). If you spend (say) 50 cents per click on your advertisement, and every 100 clicks gets you one person who spends $100 on your site, you would have made a profit of $50 every hundred clicks. This is income that you would not otherwise have made had you not advertised. Of course, as in all things, it’s possible to go overboard and spend more than you earn. You might want to consider setting aside an budget for a month or two as an experiment, and adjust as you go along to see if this approach increases your profits.
Conclusion

Google is now, arguably, the most important search engine around. Knowing what you should not do and still survive Google, as well as learning shortcuts to getting an instantaneous good placement on Google’s search engine results page, is now a very important part of website promotion.

Comments Off

Search Engine

How search engines work
======================
Search engines are based on computer programs that explore the Net in search of webpages. They do this by using so-called “spiders,” “crawlers,” or “robots”.

Like any human surfer, the robots are able to follow the links present on webpages. When finding a new webpage, they copy all or parts of the text present on the page into their search engine database.

Other spiders revisit these pages on a regular interval in order to register changes or dead links. The interval varies greatly from search engine to search engine and from site to site.

Ranking algorithm
================
The search engine will have billions of webpages in its database. In order to deliver relevant results, the search engine companies have developed special ranking algorithms that calculate the order of search query results.

By a search query we mean the keyword or keyword phrase the visitor enters into the search form of the search engine.

A special program analyzes the search query, and then searches the database for pages that contain this keyword or keyword phrase.

Next it analyzes every single one of the relevant pages in order to determine how important that phrase is on that page. The pages the search engine find most relevant will be listed first.

For instance, if you are searching for “Albert Einstein,” and the search engine finds a main headline on that page containing the phrase “Albert Einstein,” it will guess that this page is fairly relevant to your needs. If, on the other hand, it finds the phrase only once, buried in a long text, it will reckon it is less relevant.

Getting search engine points
========================
The search engine gives each and every page points based on how often and where it finds the keyword phrase (the keyword location). Some locations give more points than others, the exact factor varying from search engine to search engine.

Search engines also take other factors into consideration, including link popularity. We will come back to that.

One very important part of search engine marketing is to know where to put the relevant keyword phrases in your webpages.

Designing webpages for search engines
================================
The search engine does not “see” the webpage the way we see it; it reads only the text-based code behind it. This is will help you to have an elementary insight into HTML, the coding language used to produce webpages. If you do not know any HTML, you could for instance read through one of the beginner’s guides to HTML.

If you do not have the patience for this, by all means read on — we will help you understand. Keep in mind, though, that an HTML file is a plain text file that contains the text you can read on a web page, as well as information on how the web browser is to format it. We are going to look at the elements of this text file.

On search directories
==================
This guide is about search engines only. There are also so-called “search directories” produced by human beings that visit and consider websites for inclusion. The most important ones are the Yahoo! Directory and The Open Directory. In order to get listed in these, you have to use entirely different techniques.

See our article on “Preparing your site for the search directories”) for more information.

Select specific keyword phrases
==========================
The most important part of search engine optimization is selecting the right keyword phrases or “keyphrases”.

It is true that many searchers enter one-word queries into the search form. The chance of competing successfully for one-word search queries is fairly remote, however, unless you are trying to promote a very specific and rare scholarly term.

You will need a million-dollar budget to win the fight over general terms such as “car” or “holiday,” and what is the point, anyway?

Let us suppose you are promoting a hotel in St. Anton, Austria. What are the chances that someone searching for “holiday” is looking for a hotel room in St. Anton? They are more likely looking for general information on holidays and traveling.

If you search for the generic keyword “holiday” in Google you get millions of hits. A search for “alpine ski resorts” gives a few hundred listings. If you optimize your pages for “alpine ski resorts” the chances for getting a good listing increases dramatically. Moreover, you are more likely to get visitors that are genuinely interested in your hotel.

It is not the number of visitors (“the click-through rate”) that matter most, but the number of buying or returning visitors (“the conversion rate”).

Setting up a list of keyword phrases
============================
Set up a list of relevant keyword phrases. Ask friends and colleagues what query they would enter when looking for a service or product like yours. Do not trust your own instincts in this!

Then try to find relevant synonyms by using a thesaurus, and take variations of English into consideration (“baggage” or “luggage,” “optimization” or “optimisation”).

Finally, go to one of the online keyword services and see how popular the various search phrases are. The keyword services will also help you finding other relevant phrases.

We highly recommend the online Wordtracker service for this kind of work.

Why you should settle for only one keyword phrase? You should not! However, you must find one keyword phrase that is to be the crucial one. Optimize your home page for that keyword phrase, as many search engines give more weight to the home page in the top directory (e.g. yoursite.com/index.html) than other pages.

You may try to optimize the same page for more than one keyword phrase, but under no circumstance should you go for more than three. If you do so, you will dilute the effect of the individual keyword phrases.

Stop words
==========
Please note that some search engines ignore so-called “stop words,” which are words that are so common that they are left out of the search engine database.

Among these are a, an, and, for, is, on, or, the, and to, as well as commonly used words as “Web”, “html” and “title.” You can determine if a word is a stop word by searching for it. If you get “no results” it’s probably a stop word.

However, that does not necessarily mean that all search engines will fail finding a keyword phrase containing such words. Do some testing! If a certain stop word is absolutely essential, try placing the phrase within quotation marks: “keyword phrase”.

Metatags and other elements
========================
The <title> tag

The <title> tag is the text shown in the windows-bar of the browser window, is the most important element of the webpage. You must include your most important keyword phrase(s) in this spot.

However, make sure that you write a phrase that makes sense. Many search engines may use this line in their result listings, and you want people to click on that link.

Do not put the name of the site first, unless the name contains the essential keyword phrase.

Instead of -

<title>Alfred’s Reading Emporium</title>

go for -

<title>Used books, magazines, periodicals, posters and postcards from Alfred’s Reading Emporium in Boulder, Colorado</title>

Some SEO-experts would even argue that you should drop “Alfred” altogether. The only drawback with this is that the browsers use this text when generating bookmarks/favorites.

Please note the inclusion of the geographical location. If your business or activity is limited to one district, say so. Searchers are prone to include geographic names in their queries.

Some suggest that it helps to use multiple <title> tags. Maybe it does in some search engines. Still, it tastes of spam (cheating), and the risk of getting banned by the search engine does not make it worthwhile.

The <meta> tags
==============
Meta tags are text written into the HTML code that describes your webpage to the search engines, but which is not visible in a browser window. They are placed after the </title> tag, but before the <body> tag.

There are only three meta tags of importance for search engine optimization. All others can (and should) be left out. Here are examples of the three:

<META NAME=”KEYWORDS” CONTENT=”old and used books paperbacks periodicals magazines newspapers papers postcards post card poster old prints”>

<META NAME=”DESCRIPTION” CONTENT=” Alfred’s Reading Emporium in Boulder, Colorado offers you a large collection of used books, paperbacks, magazines, periodicals, newspapers, prints, posters and postcards.”>

<META NAME=”ROBOTS” CONTENT=”ALL”>

The NAME part of the tag, tells the search engine what kind of meta tag this is.

The keywords metatag
====================
In the KEYWORDS meta tag you may enter keywords and key phrases of relevance to the content of the page. Please note the double quotation marks. Many choose to separate the keyword phrases with commas.

We normally do not include commas, partly because they take up space, and partly because leaving them out makes it possible to include more combinations of keywords. “web search tutorial” will give you no less than three queries: “web search tutorial,” “web search,” and “search tutorial” in addition to the three individual words.

Do not repeat the same keyword more than two or three times. However, if you must repeat it, keep the occurrences apart.

Do not enter keyword phrases that are of no relevance to the content of the page.

Some SEO experts argue that you should not include any keyword phrases that are not present in the visible text on the webpage, as some search engines may interpret this as spam.

We have not found proof of this (unless you enter phrases of an “adult” nature to your page on home knitting).You may also enter misspellings, and variations of the same word (capitalized/non-capitalized, plural/singular). However, do not go overboard on this.

You should note, however, that there are very few search engines that take the keyword metatag into consideration these days. Yahoo! and Ask apparently do take it into considerations, which is why you probably should include one — just to be sure.

The description metatag
=====================
The DESCRIPTION meta tag is important as some search engines use this text for the description given in the search result listings.

This text should be a normal sentence that gives the searcher exact information on what you have to offer. Do include the main keyword phrase(s). Note that no search engine displays more than 250 characters (spaces, commas and periods included), and often less. This is why you should enter the most important part of your description first.

The robots tag
=============
The listed version of the ROBOTS- tag (ALL) tells the search engine to index the page. However, the CONTENT=”ALL” alternative is actually superfluous. Search engines will also index pages without this tag.

<META NAME=”ROBOTS” CONTENT=”NOINDEX, NOFOLLOW”>

indicates that a robot should neither index a document, nor analyze it for links.

<META NAME=”ROBOTS” CONTENT=”NOINDEX, FOLLOW”>

means that a robot should not index this document, but that it should index the pages it is pointing to.

Note though, that if you need to limit the search engines access to your site, it is better to put a robots.txt file in the root directory of your site (provided that you have your own domain). Read our article on the robots.txt file and search engine optimization.

<body> text
===========
We encourage you to include at least four or five paragraphs of informative text on every page (preferably more than 250 words, absolute minimum: 100). Include the selected keyword phrase several times throughout the text, as some search engines reward some repetition.

Again, do not go overboard on this. The language must seem natural and search engines will punish too many repetitions. We have also reason to believe that Google will become suspicious if you include only one form of the term, hence see to it that you include at least two variations of the keyword (e.g. “search engine” and “search engines”).

Also remember that people are going to read this. Good copy sells.

Make sure you include the keyword phrase in the first paragraph of text. By the first paragraph we mean a block of text delineated by a <p> and a</p> tag, consisting of at least two full sentences. Some search engine optimization experts even argue that it helps to start this very first sentence with the relevant keywords.

The search engines are looking for natural language, and give standard text a higher reward than, let’s say, a one-word menu choice.

Sometimes it helps to include the phrase in the last paragraph of a page as well.

Bold text and large sized fonts may help in some search engines. Again: Don’t overdo it.

Keyword spam
=============
Previously webmasters would include so-called “hidden keywords” in order to boost ranking. A popular alternative was to include phrases in comment links, like this: <!— search engine keywords –>. This no longer works.

Others would use a non-functioning form tag:

<INPUT type=”hidden” name=”butterfly” value=”mountain ringlet”>.

Don’t do it! The way the search engines feel about spam (“cheating”) right now, they may soon decide to ban sites for doing this.

Keyword density
===============
Search engine marketing experts often try to calculate the keyword density of each search engine and tailor pages for each and every one of them. The keyword density is the relationship between the total number of words on a page and how many times the keyword phrase is included.

To calculate the density, paste the viewable text of the webpage into Word or another word processor or editor and let the program count the total number of words. Then use the search (or find/replace) function to count the keyword phrase. Divide the occurrence of the keyword phrase with the total number of words to find the keyword density.

The optimal keyword density varies from search engine to search engine and over time. Normally it is somewhere between 2 and 5 percent.

However, do not get obsessed with this. Nowadays search engines consider so many factors that the density alone will neither win or lose the race for you.

The Pandia Search Engine Optimization Gateway includes a list of tools you may use when analyzing your use of keywords.

Headline Tags
============
Do include the keyword phrases in headlines. If possible, use the standard headline tags (<h1>, <h2> etc.)

We know that some Web designers avoid these in order to gain control over the visible presentation of the webpage.

However, by using cascading style sheets in combination with the headline tags, you will get even better control of the layout. Put your keyword phrase in the first top-level headline (whether this is <h1> or <h2>).

Other elements
=============
Many search engines will register the image ALT tags, which you can use to describe pictures to people who are using non-graphic browsers (such as Lynx or browsers for the visually impaired) or surfers that have images turned off in order to gain download speed.

Obviously these tags should include a description of the graphic element. Still, the fact that most Web designers include logos and image-based headlines gives you ample opportunity to include keyword phrases.

<A HREF=books/index.html”><IMG SRC=”graphics/books.jpg” WIDTH=”60″ HEIGHT=”72″ HSPACE=”4″ VSPACE=”4″ BORDER=”0″ ALT=”From Alfred’s collection of rare and used books, folios, paperbacks, hardbacks, manuscripts and publications. “></A>

You can also make use of the link TITLE-tag (not to be confused with the <title>-tag at the top of the HTML-file). This is an addition to the link-tag (or any tag) that that lets you describe the link more thoroughly. The text pops up when someone is moving his or her mouse pointer over the link:

<A HREF=”french-cars/renault.html” TITLE=”Renault car models of 2001″>more about Renault</A>.

Keywords in links and URLs
=======================
You should also include keyword phrases in links. Not only do these links improve the ranking of this page; it may also boost the relevancy of the page the link is pointing to. Hence <A HREF=”file.html”>Click here!</a> is a wasted opportunity.

By the way, you could also name files after the keyword phrase, as well as directories, as some engines take the URL into consideration when calculating the ranking. This is why so many optimization experts argue that you should get a domain name that includes your main keyword.

Given that all the good names are taken, this is easier said than done, but you can at least include the keywords in directory and file names, like this: <A HREF=”butterfly/african-butterfly.html”>on African butterflies</a>.

Search engines and Web design
==========================
Moving text to the top – From a search engine optimization perspective, the perfect webpage has no graphics, no javascript code, no tables; it uses no special plug-ins and contains a lot of normal, readable text.

From a designer’s point of view such pages are, however, painfully boring. We need that logo! Illustrations liven up the page and tables give you control over the layout. Don’t worry. There are ways around this problem.

Nevertheless, If it is in any way possible, include you keyword phrases in normal text at the very top of the page, using a relatively small — but clearly visible — font size if necessary. If your design does not allow that, do at least include the phrase in image ALT-tags. Reduce the number of images and elements before the main body text as much as possible.

The navigation problem
====================
Readers are used to having the navigation menu to the left of the page. However, if you put the menu in the left hand column of a table, all the text in that column will appear before the regular text in the HTML code.

There are several ways of solving this problem. You could, for instance, move the navigation list to the right).

The design of our Goalgetter Web Search Tutorial is another. If you look carefully at the code behind those pages, you’ll see that the text in the left hand column appears after the main center column text in the HTML code!

You may also use CSS and position webpage elements by exact positioning. However, it is harder to make such pages look good in all relevant browsers.

Or you may just let it rest. We tried to force our WordPress blog software to put the main content first, and found that life was too short!

Frames
=======
Sites using framesets (loading different pages into different sections of the browser window) have usually been in deep trouble, as search engines have listed the frameset page only. This means that none of the content pages are listed.

The trick has therefore been to include a keyword rich text in the <NOFRAMES> tag in the frameset file. The <NOFRAMES> tag lets you imbed a separate webpage to be read by surfers that are using browsers that are unable to interpret frames. Search engines that are not spidering the framed pages will normally read this text.

Fortunately more and more search engines are able to follow the links included in the frameset. As far as we can see Yahoo!, Google and MSN can now find framed content pages on a frame-based website.

On the other hand, this means that they will normally ignore the frameset page. Hence the <NOFRAMES>-portion of the frameset page will no longer give a good ranking in these search engines.

Instead you must optimize each and every one of the framed pages. This is actually a good thing, as you can use these pages as doorway pages focusing on one keyword phrase only.

There is one problem, though. The search engines will present a link to the framed pages, not to the frameset itself, which means that your visitors may find themselves stranded on an orphaned page, without navigation tools. You should therefore use a javascript that automatically loads the complete frameset.

The following works well in most browsers. The script is included in the tag of all framed content pages:

<BODY onLoad=”if (parent.frames.length==0) top.location=’http://www.yoursite.com/subdirectory/frameset.html’;”>

Change the address (frameset.html is the frameset file). Note that you may submit the framed content pages individually to search engines that do not resolve framesets.

Directory Structure
=================
Note that the level of the directory where the page is found is important. Higher is considered more important. If the page is buried too deep, the crawler may not go that far and will never find it.

This means that www.yoursite.com/index.html will have a better chance than www.yoursite.com/topic/subtopic/subsubtopic/index.html. Avoid more than two sub-directory levels.

Linkmaps
=========
It is quite possible to submit all your pages to the search engines. Some of them, however, prefer to find your pages themselves. Google will do so, provided it finds a place to start or you give it your home page URL.

The major search engines may give a higher boost to pages they find by themselves, at least on the short term. This is why you should include links to as many pages as possible, not only on your home page, but also on all pages. By doing so, you ensure that the search engine will find as much of your site as possible, wherever it enters the site.

There are limits to how many links you can include on a page, of course, especially if you have a large site. If this is the case, you should make a site map or “hallway page” presenting all sections of your site and including links to most of the pages. Make sure that you include descriptions of the various sections, as search engines like links embedded in normal text. Moreover, your visitors will thank you for it, as it makes it easier to navigate your site.

Flash and image maps
===================
The worst you can do, search engine wise, is to design a home page with no links at all.

Remember, most search engines do not understand Flash. Hence they cannot follow links embedded in your cool Flash intro.

A Flash-only page has no regular text, so there will be nothing for the search engine to base its ranking on. And even the search engines that do follow Flash links, find it hard to make sense out of what the pages are about. Meta tags are not enough.

Search engines also find it hard to follow links in image maps. If you do use Flash or image maps, make sure you include regular links as well, for instance near the bottom of the page. These will also be of help to visually impaired people and others who are unable to use these technologies.

Spam
=====
One way of including many links on a page, without making the page look crowded, is to hide them, for instance by including 1×1 pixel images that link to hidden pages. This is a popular technique among some search engine “experts” who want to include links to so-called doorway pages that are not be accessed by regular visitors.

Some also put in links with no link text, like this:

<A HREF=”hiddenpage.html”></A.>

Again, these are not techniques we would recommend. The way the search engines feel about spam these days, they are bound to do something about this as well. And remember, if one of your competitor reports your spamming to the search engines, it is very likely that the search engines will punish you!

The following techniques may also lead to your site being banned from many of the major search engines.

Redirect pages that use meta-tags or javascript to immediately load a new page. If you need to use redirects, make sure to include a NOINDEX metatag to tell the search engine not to index this page.

Large amount of keyword rich tiny text (FONT SIZE=”1” or relevant CSS tag) near the end of the page.

Keyword stuffing or spamdexing is the repetition of the keyword phrase over and over again. You must work phrases into regular sentences!

Invisible text is text rendered in the same color as the webpage background. White text <FONT COLOR=”#FFFFFF”> on a white (#FFFFFF) background is considered spam, as are small deviations from the same color, as in <FONT COLOR=”#EEEEEE”> or <FONT COLOR=”#EFEFEF”>. The font must be clearly visible for the average viewer.

Note that the search engine spiders do not see tables. Hence a white font in a table with a black background may be considered spam if the page background is white.

The Link Popularity Factors
=======================
Link Popularity – The endless arms race between search engines and search engine optimizers has inspired the search engines to find new ways of measuring relevance, preferably using factors that cannot be unduly influenced (read: spammed) by webmasters.

Google led the way by introducing link popularity as a major factor in its ranking algorithms. The idea is quite brilliant. Google reckons that a site with a lot of inbound links that other webmasters find useful must be good, so it gives it an extra boost in the rankings.

There are at least four types of links, giving gains of varying intensity:

Links from the major directories

In general links from major search directories, i.e. Yahoo!, and the Open Directory count a lot. Hence, you should do all you can to get listed in these directories.

LookSmart has become a pay per click directory, and will not help in this respect.

Commercial sites must normally pay a fee to get listed in the main Yahoo! directory (yahoo.com). The fee is pretty high (some US$ 300 a year) and at the moment Yahoo! directory listings do not get favorable treatment in Yahoo! results. Unless you have a large budget or a non-commercial site, we suggest you focus on the Open Directory.

Read our article on Preparing your site for the search directories to learn more about how to convince the editors that your site is worth listing.

Links from related sites
====================
Inbound links from sites related to your topic may also give a fair boost in engines such as Google and Yahoo!.

Google groups related pages in clusters devoted to a specific topic. At the center of these clusters you will find pages with a lot of inbound links from sites in the same cluster. If you can get a link from one of these major sites, you are in luck, as the search engine will find your site more relevant for queries related to this specific topic.

It helps a lot if the text of the inbound link contains the keyword phrase your are optimizing the page for.

Note that search engines such as Google try to find out what the main topic of your page is. One way of doing this is to analyze the main subject matter of sites linking to yours.

Teoma will also analyze the main keyword phrases of your entire site in order to pinpoint the main topic.

Links from unrelated sites

Links from unrelated sites may also count. However, do not expect links from your uncle’s sailing site to do much for your pet project on Tibetan philosophy.

Avoid link farms!
===============
Links from free-for-all link farms that are set up to increase the participants’ link popularity should be avoided at all costs.It didn’t take long before clever people out there found ways to spam the link popularity ranking, by establishing link farms where the members are guaranteed links from the other member sites in return for adding a similar link page to their own site.

Google has started punishing sites that have link-farm pages, and other search engines are expected to follow suit.

If you are a member of one of these schemes, delete the relevant link page and ask to be removed from the others.

There are also a lot of free-for-all link collections, “directories,” and “search engines” out there, that are set up to harvest the e-mail addresses of webmasters.

We have reason to believe that these also may lead to a loss of points in the near future. This is why you should not make use of automatic submission services that will submit your site to “thousands” of search engines, unless you can control which sites your site will be submitted to.

Find related sites
===============
Next you should try to get your site listed at the most important sites in your field of interest. In order to achieve this, you should do the following.

1. To find sites related to your own, search the relevant categories in Yahoo! and the Open Directory. Then study some of the most promising to find out which sites are the most popular in this field. You can also check out sites that have links to your major competitor (see below for link lookup).

2. Make a content-rich site of high quality that is bound to interest the editors of the sites you are approaching.

3. Make a “recommended links” section at your own site with sites that might interest your visitors. Yes, you may lose some visitors this way, but the chances are that they will come back to a site that offers a directory that will help them find what they are looking for. Each link should be accompanied by a description.

Actually, the most effective link pages are those that include short reviews of the relevant sites. Optimize these pages for selected keyword phrases, and voilà, you get some extra legitimate doorway pages for free!

Reciprocal linking

4. Write a personal mail to the editor or webmaster of the relevant site and tell her or him that you like the site so much that you have included a review in your resource section.

This is, of course, very flattering, and the person at the other end is bound to visit your site to read what you have written. Therefore, make sure to include the address of the review in the mail.

In your mail you may suggest (but not demand) a reciprocal link. Include a short presentation of your site (two or three paragraphs) and tell her or him how such a link will benefit their visitors. Suggest where on their site a link to your site will be appropriate.

By doing this, you show the recipient that this is more than a standard formula mail, and you are making it easy for them to include a link.

Make sure that the text of the proposed link includes the keyword phrase the page is optimized for. Hence, for the Pandia Search Engine Marketing 101 tutorial, a link named Search Engine Marketing 101 will be much more efficient than a link that simply said Pandia. After all, “search engine marketing” is what this site is about.

However, all your inbound links should not have the same link text. That will definitely make the search engines suspect foul play.

Google gives a lot of weight to the link text itself, and — to a lesser degree — also to the text surrounding the link. Hence having some links where the main keywords are in the surrounding text will be fine.

Do not forget to include your Web and email addresses in your letter. Keep the outgoing mail, so that you remember which sites you have approached.

To get more general inbound links, you may consider starting affiliate and award programs. You could also take part in relevant online discussion forums that allows you to include your URL in your signature or member profile.

You will soon find that some of your inbound links do not show up in a few of the search engines. It is tempting to submit these pages to these search engines.

However, we believe you should not do this, unless you have asked the webmaster of the site for permission. Search engines may punish sites that submit too many pages in a given period of time. The site you are trying to help may be approaching that limit.

Some search engines let you do a special search to find sites linking to a special site. We have gathered the relevant search syntax at our submit site page.

Submitting Sites
==============
Here is the good news: Sites that are already represented i a search engine index need not submit their pages on a regular basis. As long as there is a link between a page already in the search engine index and the new page, the search engine will find it.

(That does not guarantee its inclusion, however, but that is another matter.)

If you have a new site it may make sense to submit its home page to the search engines — while at the the same time getting a few inbound links from sites that are already included.

There are many software programs and online tools for submitting your webpages automatically to search engines. You should use these with the care, as more and more search engines punish excessive submissions.

AltaVista used to claim that as much as 98 percent of submissions entered at its Add URL page are spam, including links to irregular doorway pages or from webmasters adding an extreme amount of webpages daily.

Submission tools
===============
You should normally submit pages to the major search engines by hand. If you manage a large number of websites you may, however, find it useful to use software that submits Web pages for you (like Web Position Gold).

You may use Pandia’s search engine submission resource page when submitting manually. It contains links to relevant search engine submission forms, help files, pay per click and paid inclusion services and more.

The most important search engines are Google, MSN and Yahoo!. Also submit your site to Ask and Wisenut, and any major Non-American search sites that are relevant to your audience.

If you have a large site, and want to make sure that Google identifies all your pages, you may use Google Sitemaps, a system whereby you submit a text file containing links and descriptions of all your pages. This requires some more work, though!

Search engines vs. search sites and portals
==================================
Please note the difference between a search engine and a search site. A search engine is the database that delivers search results, a search site is the website that lets you search this database. Some search engines are used by many search sites.

There are now only four major search engines left: Google, the Yahoo! Search Engine, MSN and their little brother Ask.

To make things confusing, most search sites also present results from search directories (i.e. catalogs of sites hand picked by human editors, cf. the Yahoo! Directory or the Open Directory). This is why some search engines will ask you to submit your site to the directory as well as to the search engine.

Most search sites and metasearch sites will add links from fee-based pay-per-click search engines.

Do not resubmit listed pages!
========================
Some experts recommend that you resubmit you pages on a monthly (or even weekly) basis. Don’t! The search engines may consider this spam and give your pages a lower ranking.

If the page is listed, leave it alone. Only resubmit if you have made substantial changes to the page. Even then it can often be better to wait and let the search engine find out by itself.

Script generated pages
===================
Please note that search engines are having difficulty indexing dynamic database or script generated pages with URLs including signs such as “&” and “?” (generated by Perl CGI scripts).

The reason for this is that they are afraid the spider will get lost in an endless loop generating thousands of pages.

Fortunately, the search engines are getting better at this now, and the only type of URLs they really hate, are those that include a session ID.

Search engines will index active server pages (.asp), server side includes (.shtml), and Cold Fusion pages (.cfm) without difficulty.

Checking your ranking
===================
The time it takes for search engines to revisit your site and refresh their database will vary. Normally it takes from two to six weeks.

There are two main exceptions to this general rule:

If you update your site frequently, a search engine like Google may consider it newsworthy and start crawling it more often, normally every 48 hours. This may even get you a small boost in the rankings.

If you pay to get included through one of the various paid inclusion programs, you will also be revisited once every one to two days. Paid inclusion will, however, not get you a boost in rankings.

The good thing about being revisited frequently is that you may evaluate the effect of your search engine optimization efforts much sooner, which gives you more room for experimentation. If your new coding does not work out, you may return to your old version more quickly, without loosing significant amounts of traffic.

If you have a site with a large number of pages, or many sites, checking your ranking for all relevant keyword phrases, may be very time consuming.

Search engine submission and ranking software
=======================================
This is where the search engine submission and ranking software packages actually becomes useful. They will search the various search engines for you, and produce easy to read reports on search engine positions and trends.

Be careful, though, and do not overdo it. Search engines like Google do not like that webmasters make use of costly bandwidth in this way. Hence spread your number of request over a reasonable period of time (some search engine submission packages will do this for you), and reduce the number of keyword phrases to be checked.

Among the most well known search engine ranking software solutions are Web Position Gold, and AgentWebRanking.

Comments Off

Google SEO

It’s been a few years since my last article on getting your site noticed on Google, Tips on Improving Your Google Search Engine Ranking. The situation hasn’t changed much, and the article is still relevant today. However, over these years I have learned a couple more things about what to do and what not to do on your website with regards to your site’s placement on the search engine results.

Hidden Text

Everyone knows that search engines (not just Google, but probably every one of them) frown on hidden text. Some people use hidden text to stuff keywords on their pages so that when people type those words on the search engine, your site will show, even though your page does not have any visible sign of those words.

I have never believed in hiding my keywords in hidden text or any of those cloak and dagger stuff, and hence I don’t practise such things. Imagine my surprise, when one day, a few years back, I suddenly found my pages on thefreecountry.com receiving a Page Rank of 0 because of hidden text on the page. (See my other article on Google Page Rank if you don’t know what Page Rank is.)

No, I did not compromise on my principles or anything like that. I did not try any underhand tactics to stuff keywords, etc. The reason is more prosaic.

At that time, I had partially converted thefreecountry.com to use CSS for its site design instead of the traditional <font> and <table>tags. In those years, there were still a few people using the old Netscape 4 which did not support CSS correctly. To accomodate those people, I tried to make my site degrade gracefully when it detected an old browser being used. I also put a message that contained the following text:

This text was enclosed in a box that was hidden from view if you were using a modern browser, but visible if you used Netscape 4. I used the usual methods of getting different browsers to load different stylesheets as mentioned in my article How to Use Different CSS Style Sheets For Different Browsers (and How to Hide CSS Code from Older Browsers).

Unfortunately, Google’s spider detected that some text was hidden, and not realizing that it was also visible in other cases, automatically assumed I was employing some sort of skulduggery to get those keywords into its index. It then penalized my site accordingly.

When I realized that my attempt to help my visitors was going to cost my site’s position in the search engine results, I immediately pulled the message, and its accompanying CSS, from the site. Visitors using old browsers would still be able to use the site, but they would not receive any friendly explanation.

Although the days of Netscape 4 are long gone, the lessons learnt from that incident are still useful today. Undoubtedly many of you will not even dream of using keyword stuffing in hidden text and the like. However, as I found out from the above incident, Google’s search engine robot is far from intelligent, and its hidden text detection algorithm can bite legitimate webmasters too. If it even senses hidden text, even for an innocuous purpose like mine, your site is history.

Some ways in which you might inadvertently use hidden text (possibly without knowing if you use third-party scripts) include the use of drop-down menus for site navigation and browser-specific tips for users (like in my case above). I’m fairly certain that the Google programmers keep refining their engine so that the more common use of CSS to hide text for harmless purposes like navigation menus (and so on) are recognized as benign, but if you are about to use hidden text for some fancy design gimmick on your page, you might want to test it out on some obscure page on your site to see if the Google robot chokes on it. After all, always remember your site’s life depends on a robot, not a human. It may be obvious to you and other humans that the code is harmless, designed for some acceptable purpose, but a computer program can only follow a set of preprogrammed rules.

Getting Your Link to Appear on the First Page of the Search Engine Results

I get many queries everyday from new webmasters reading thesitewizard.com, asking me how they can get the link to their site to be shown on the first page of the Google search engine results. In fact, for some of them, getting their site to appear anywhere in the first few pages of Google’s results would already be a victory.

If you are in a hurry to get your site noticed, and have a budget allocated for your site’s search engine promotion, one instant way to get to the first page of Google’s results page is to buy advertisements (called Google Adwords) on your keywords. Before you summarily dismiss this, read everything I have to say first.

Google’s Adwords actually operates according to your budget – that is, you can spend according to what you can afford. It also allows you to place the adverts on precisely the keywords you would have wanted for your site in the search engine results. Unlike the normal search engine indexing procedure however, here you control the exact keyword which will trigger your site’s advertisement and its prominence. Of course it comes with a price. But if you’re doing a business, this is often the surefire way to get people to see your site in relation to something that they are searching for.

Many new webmasters I deal with don’t consider this as a viable option, because it involves cash-outlay. I agree that if you are merely running a personal site or a hobby site that does not receive an income, advertising this way is probably out of the question. You should then work on applying the usual search engine promotion tips to your website and hope for the best in the long run. However, if your site is a business site that yields an income, you might want to calculate the costs and benefits to see if you might actually earn more this way. Treat your expenditure on the advertisements like all other business expenditure and calculate your return on investment (ROI). If you spend (say) 50 cents per click on your advertisement, and every 100 clicks gets you one person who spends $100 on your site, you would have made a profit of $50 every hundred clicks. This is income that you would not otherwise have made had you not advertised. Of course, as in all things, it’s possible to go overboard and spend more than you earn. You might want to consider setting aside an budget for a month or two as an experiment, and adjust as you go along to see if this approach increases your profits.

Conclusion

Google is now, arguably, the most important search engine around. Knowing what you should not do and still survive Google, as well as learning shortcuts to getting an instantaneous good placement on Google’s search engine results page, is now a very important part of website promotion.

Comments Off

Search Engines Ranking

Common Reasons Why Your Site Is Not Showing Up in the Search Results

1. Flash Splash Screens for a Home Page: Words Speak Louder Than (Animated) Action

   Your home page is an extremely important page on your site. It should tell people what your site is all about. Very often, when I’m asked to review a site, I find a site that has the following characteristics:

   • The home page has nothing but a Flash splash screen with some fancy animation.
   • If there are any words on the page at all that are not embedded in the Flash animation, it’s “Enter Site”.
   • The HTML title tag of the page says “Flash Intro Page”.

   Ask yourself what the search engines will index in such a case. Chances are that your home page will probably only rank for “enter site” and “flash intro page” since those are the only words on the page many search engines can pick up. In fact, since there are so many sites with those words, you may not even rank for them (not that you’ll want to).

   Text content that can be indexed is also vital. The search engine cannot know intuitively what your website is about if it doesn’t have words to index. Embedding text inside Flash content and expecting search engines to be able to recognize them is a risky business – Google may be able to read some types of text, or it may not. The other search engines probably won’t be able to even extract anything.

   Get rid of the Flash splash screen for your home page if you have one, and replace it with a normal web page. Not only will your site rank better, it will also look more professional. Restrict your use of Flash to things that really need Flash, like interactive online games and videos, and not pages that you want indexed.

2. More Pictures Than Words: Even A Few Words is Worth More Than a Picture

   I’m sure you’ve heard the saying, “a picture is worth a thousand words”. Maybe that’s true to a human being. But to a search engine, using current technology, a picture is useless for determining what the site is about. The engine cannot see a picture and tell that it’s a picture of Britney Spears or your favourite cat. If you want the engine to know that you have a picture of some pop star, write words to that effect.

3. A Web 2.0 Website Without Traditional Web Content and Navigation: New is Not Necessarily Better

   Related to the above points is a website that is heavily dependent on Web 2.0 technology like AJAX or even just simple JavaScript. If the content or the links on your site are generated using JavaScript, you may encounter the same problem another of thesitewizard.com’s visitors experienced – the search engines’ inability to find other pages on the site.

   The solution to this is to always have plain, traditional web content and navigation links somewhere on your page. You do not need to drop web 2.0 stuff to do this. Just think of some way to put normal text and normal links onto your page, in a way that does not need a JavaScript interpreter to decode them. While you can also put up a traditional site map and create a file using the sitemap protocol to help the search engines, nothing beats normal text links on a page that works for both search engines and human visitors.

4. Nobody is Linking to You

   As mentioned in my article on How to Create Your Own Website, if your site is not linked to by any other site, some search engines, notably Google, may not even bother to visit your website to index it. Read that article for pointers to how to get started on getting some links to your site.

How to Check Your Website for Search Engine Friendliness

One way to check your website to see if it’s search engine friendly is to configure your browser to not display any graphics, not to use plugins and not to run JavaScripts. If your site only shows a blank screen in such a case, that’s what the search engine will see as well. If you only see a few meaningless words on the screen, then that’s what the search engine will think your site is about as well. If you find you cannot visit any other pages on your website from the main page, the search engines will have the same problem as well.

The easiest browser to use for such checks is the free Opera web browser. The latter makes it very easy to disable and re-enable everything, either on a site-by-site basis, or on a global basis. To globally disable the things I mentioned, hit the F12 key (F12 for Windows and Linux users; Alt+F12 if you use a Mac), uncheck “Enable Java”, “Enable plug-ins” and “Enable JavaScript”. You can enable them again easily by hitting the same F12 (or Alt+F12 for Mac) key and checking those items. To disable images, hit Shift+I (all operating systems). (This is a toggle between “No images”, “Cached Images”, and “Show Images”, so hitting Shift+I again will switch between these three modes.)

This is not a foolproof solution for checking your website though, since Opera is a modern frames-capable browser, so sites using something called “frames” will still show up correctly in Opera, even though they won’t necessarily be indexed properly by a real search engine. But for quick checks, this is by far the most user-friendly way to do that works on all the major desktop operating systems. Note that I’m not forcing you to switch to Opera or anything like that – use whatever browser you like as your normal browser. But as a web developer, since you will probably have all the major browsers around to test your site anyway, you should use Opera’s additional convenience features to give your web pages a quick going-over.

(Additional tip: while you do this, you can also check how your site will appear in small-screen devices like mobile phones by hitting Shift+F11 on Windows/Linux or Shift+Alt+F11 on the Mac. Hit the same key again to return to normal mode.)

What Do You Want to Rank For?

A very important thing to bear in mind when designing your website is to ask yourself what your site is all about. What do you want to rank for? If you want to rank for “Widget X”, the words “Widget X” must appear on your website in ordinary, visible text. This may seem obvious when I say it, but judging from what I see when I review some websites, it may not necessarily be apparent to some people. (Don’t take this as an accusation or anything like that, it wasn’t obvious to me either, when I first started.)

Comments Off

How to Use Meta Tags

If you view the HTML sources for the web pages on the 4ubd.com, you would have noticed that they all contain META tags. META tags influence the way some search engines list and index web pages.

There are at least three META tags that you will want to use to make sure your page is search engine ready.

What are Meta tags good for?

For search engines that support them, META tags are used in at least three ways:

1. They are used to describe your site in the search results. For example, if you do not have a META Description tag for your page, some search engines will simply grab the first few lines of your page to serve as the description of your site. This description is more important than you may think. If you were searching for “nosuchwordasthis”, which of the following two items listed would you be more likely to click?
   • Nosuchwordasthis.com: Ridiculous and Impossible Words
     Home About Example.com Contact Us Links Download FAQ
   • Nosuchwordasthis.com: Ridiculous and Impossible Words
     Some of the incredible words that netizens have come up with this year, along with their supposed meanings.

   The first item above had a snippet from what appears to be its top menu in the lines below the website title. The second item has a description which tells the person searching exactly what the site is all about.

   If you do not put a description tag, what appears in the listing would be likely to be whatever first appears on your page, usually the alt text of some graphic or banner or perhaps your top menu.

2. There are search engines which also look for a META Robots tag to determine how they index your site. For example, if Google does not encounter such a tag, it will assume that it can index and follow the links on the page. If you want some other behaviour from Google, you will have to use a META tag to specify it.
3. In the old days, they were used to determine which keywords your page will be listed under. So if you want your page to be listed in a search engine when the user searches for the term “nosuchwordasthis”, the latter should be present in your META Keywords tag as well as in your document. I’m not sure however if any search engine still uses this tag today.

There are other META tags, which individual search engines recognize. However, the tags listed here are the main tags relevant to most search engines.

How to Use the META Description Tag

A META Description tag, for those of you who write your web pages using raw HTML code, looks like the following:

Give a brief description of your page in this tag. Bear in mind that this description (or part of it) will be displayed in the search engine results so try to make sure you phrase it in such a way that the person searching can tell at a glance that he’s found the correct page for his search. Minimize irrelevancies and put the essentials near the beginning of the description so that if the search engine only accepts the initial (say) 150 characters, the essential parts of your description will still be displayed.

Not all search engines use this tag. For example, Google normally only displays the text surrounding the matching keyword that the searcher used.

How to Use the META Robots Tag

Although this Meta tag is seldom mentioned in search engine promotion guides, it is important if you want certain search engines to crawl through your website using the links on the page you submit. Since adding it involves little additional effort, you might as well do it while you add the other tags.

In general, you will add the following line to your web page if you want the search engine to index your page and follow the links on it.

In cases where you do not want it to index your page simply use “noindex” instead of “index”. Likewise if you don’t want the engine to crawl through the links on your page, use “nofollow” instead of “follow”.

Most, if not all, search engines today recognize this tag.

Furthermore, if you do not want the search engine to keep a cache of your page on their servers, you can affix a “noarchive” value to the “content” attribute, such as in the example below:

Google, for example, will see the “noarchive” value and not keep a copy of your page in its cache.

How to Use the META Keywords Tag

The Meta Keywords tag used to be an important way to get your page listed under specific keywords under the search engines. Nowadays, however, major search engines like Google and Alta Vista ignore this tag, and the keywords tag no longer has the significance it used to have in the early days of the web.

The META Keywords tag should be placed in the <head> portion of your web page. If you are using a WYSIWYG editor, there is usually some way for you to enter the keywords on the page using a separate dialog box. If you are typing it yourself into your web page, the tag looks like the following:

In place of the phrase “put your keywords here”, you should of course give a list of your keywords. The keywords should be comma-delimited. That is, if you want to your page to be listed when the user searches for “medical help” and “free diagnosis”, you should use the following tag:

Each search engine has its own limit to the number of characters in your keyword list. The general advice given is not to make your list longer than you need it to be. Some search engine experts recommend that you do not exceed 1000 characters in your list. In the days when it still recognized the keyword tag, Alta Vista did not use anything beyond 500 characters.

A word of caution. Although you might think that this is a good place to put in common misspellings of keywords that you spelt correctly on your page, you should probably note that in the days when search engines indexed the keyword list, some of them penalized web pages that contain words that cannot be found on the page itself. They were probably trying to guard against abuse by people trying to “spam” the index. The latter have been known to dump every conceivable keyword they can think of even if it is unrelated to their web page. Unfortunately, this policy also penalizes legitimate site owners who really want to catch those common misspellings of their page content.

You should also not repeat a particular keyword too many times in your tag. Again, some engines used to make a note of such things and penalize your site. Note that it is possible to accidentally repeat a keyword multiple times when you use a word in different phrases: eg, a tag that has the following keyword list, “search engine promotion, search engine ranking, search engine placement”, would have repeated the words “search engine” three times even though they occur in different phrases. Some repetition is apparently acceptable (possibly even up to three times), although excessive repetition is regarded as “spam”.

So how important is the Meta tag?

Although the benefits conferred by using the various meta tags on your page are not substantial, they are nonetheless useful in controlling how the search engines index and list your page. Since tags like the description and robots tag require very little effort to add to your page, it is probably a good idea to add at least these tags to your pages.

Comments Off

How to Make Search Engine Friendly WordPress Blog

Improve Your Blog’s Search Engine Readiness

After I wrote the article How to Create a Search Engine Friendly Website, a few visitors to thesitewizard.com, obviously blog owners, have asked me how they can make their WordPress blog more search engine friendly.

The good news is that the default installation of WordPress is already generally search engine friendly. If you wish, there are a few additional steps that you can take to improve that search engine readiness, while at the same time protecting your site’s search engine visibility for the future.

Keywords and Post Titles in the Permalink or Link Structure

The default installation of WordPress uses URLs like www.example.com/?p=1234 to point to your individual blog articles. While this works fine for modern search engines, changing the default so that your links look like www.example.com/category/title-of-post has a few additional benefits.

1. When your post title and category names are represented in your URLs or permalinks, the words that you use for your categories and titles automatically become keywords that the search engines can index every time someone links to you.Before you dismiss this as being of minimal benefit, look at it this from both the blog owner’s perspective as well as the average visitor’s perspective. As the blog or website owner, you will undoubtedly prefer that your visitors link to you using your post title as the anchor text – for example, you may want people to link to you using HTML code like <a href=”http://www.thesitewizard.com/gettingstarted/search-engine-friendly-wordpress-blog.shtml”>How to Make Your WordPress Blog Search-Engine-Friendly</a> since such code gives your full title and yet links properly to your article. However, the reality is that when people talk about your article in other blogs, many of them simply link to it using the URL – for example, they may refer to this article simply as http://www.thesitewizard.com/gettingstarted/search-engine-friendly-wordpress-blog.shtml

   When you put configure WordPress to place your category name and post title as part of your URL, even such links contain your title and keywords embedded in the URLs. As mentioned in one of my Google articles, Google takes into account the words given in the anchor text to determine if your page is relevant for any given search engine query. Admittedly, the help given in the anchor text is probably not enormous, but having it is better than the cryptic www.example.com/?p=1234 URL which neither gives keyword help to search engines nor describe your article adequately for human beings.

2. The lengthy text-based names for individual blog pages also help in making your blog future-proof. At this time, since you’re reading this article, you are probably using WordPress as your blog software. But who can say what will happen in the future? You may find another blogging tool that you prefer. If you leave your URLs in the form of www.example.com/?p=1234, how will you maintain such a script-specific URL? The new software may assign another number to your blog entries when you import your existing blog database. This results in the URLs of your pages changing.Changing URLs has enormous search engine ramifications. All the links from external sites that currently point to those pages will be instantly broken, and along with it, the “reputation” of that page in the eyes of the search engine. Sure, you can work around it with some mod_rewrite voodoo, but I personally prefer to pre-empt the problem by configuring WordPress to use page titles as the permalinks.

   When you make your article page URLs appear as straightforward category-and-page-title URLs, if you ever change blog software, and find that you cannot configure the new software to create identical URLs, you can simply save all your old posts as ordinary static web pages using those exact filenames. The links to your old pages will then work without modification.

How to Put Your Post Titles into Your URLs in WordPress

WordPress makes it easy for you to change your URLs, a.k.a. permalinks.

1. Log into your WordPress blog.
2. Click “Options”. It’s near the top of the screen.
3. Click “Permalinks”. It’s just below the line that says “Dashboard Write Manage…” (etc).
4. Before you change anything, make sure that you don’t have a category that conflicts with one of WordPress’ scripts or directory names. For example, check to see that you don’t have a category by the name of wp-admin. If you do, you should change your category names first before proceeding.Scroll down to the “Common options” section and select “Custom, specify below”. In the box below, type “/%category%/%postname%/” without the opening and closing inverted commas (quotes). Click “Update Permalink Structure”.
5. You should see a message that says, “Permalink structure updated”. If you get a message that says something like “If this file were writable you could edit it” together with a box showing you the contents of a .htaccess file, it means that your web host configured PHP so that it could not write to your web directories.One way to solve this is to copy the suggested .htaccess contents into an ASCII text editor like Notepad (on Windows), and save it as “.htaccess” (including the quotes if you are using Notepad on Windows). Then upload the file into your blog directory.
6. You can now view your blog and check the individual post pages to see your new site structure.

Post Titles in the TITLE Tag

One of the recommendations in the article on How to Create a Search Engine Friendly Website is to make sure that your HTML title tags contain meaningful and relevant text. The HTML title tags in WordPress is created automatically from your post titles. In the default template, the blogging software appends your post title to the name of your blog and the words “Blog Archive”, so that if your blog is called “XYZ Blog” and your post is called “How to Improve WordPress Search Engine Readiness”, you get a title tag with the words “XYZ Blog » Blog Archive » How to Improve WordPress Search Engine Readiness”.

While this is better than not having your post titles in the TITLE tag, the ideal is to have a title like “How to Improve WordPress Search Engine Readiness”. That is, having your post title at the start of your TITLE tag is preferable, since this is the description that will be displayed in search engine results listing your site. You want your post title to be first in the description in this case because you want the user to immediately realize the relevance of your post to their query. Having the name of your blog as well as the word “Blog Archive” first in the post title is an unnecessary distraction.

How to Modify the Default Template to Place Post Titles First

1. Log into your WordPress account and click “Presentation”.
2. Click the “Theme Editor” link.
3. Look at the right hand side of your browser window to locate the “Header” link, and click on it.
4. The browser should now display “Editing header.php”. Somewhere in the box below this, locate the line that says
   <title><?php bloginfo(‘name’); ?> <?php if ( is_single() ) { ?> &raquo; Blog Archive <?php } ?> <?php wp_title(); ?></title>

   Change it to the following:

   <title><?php wp_title(); ?></title>

   If you want to have the name of your blog at the end in the title, change it to the following instead:

   <title><?php wp_title(); ?> – <?php bloginfo(‘name’); ?></title>

   Note: if you have trouble cutting and pasting the code above, it probably means you are using Internet Explorer 6. Try using a different browser for this purpose, like Firefox or Opera.

5. Click the “Update File” button when you are through. You should receive the message “File edited successfully”.If you receive a message “If this file were writable you could edit it”, it means that your web host has configured PHP so that it cannot write to your web directory. Copy and paste the contents of the file to an ASCII text editor as before, save it as “header.php” (including the quotes if you are using Notepad on Windows) and upload it to your themes directory. If you are using the default theme, this should be in the “wp-content/themes/default” folder. Overwrite the existing header.php file.

Conclusion

With these two trivial steps, you have drastically improved the search engine friendliness of your blog. Be sure to read the rest of the How to Create a Search Engine Friendly Website article to get more general tips for your blog.

Comments Off

How to Set Up a robots.txt to Control Search Engine Spiders?

When I first started writing my first website, I did not really think that I would ever have any reason why I would want to create a robots.txt file. After all, did I not want search engine robots to spider and thus index every document in my site? Yet today, all my sites, including thesitewizard.com, have a robots.txt file in their root directory. This article explains why you might also want to include a robots.txt file on your sites, how you can do so, and notes some common mistakes made by new webmasters with regards the robots.txt file.

For those new to the robots.txt file, it is merely a text file implementing what is known as the Standard for Robot Exclusion. The file is placed in the main directory of a website that advises spiders and other robots which directories or files they should not access. The file is purely advisory – not all spiders bother to read it let alone heed it. However, most, if not all, the spiders sent by the major search engines to index your site will read it and take cognizance of the rules contained within the file.

What is the purpose of a robots.txt file?

1. It Can Avoid Wastage of Server Resources

   Many, if not most websites, have some sort of scripts (computer programs) that run on their website. For example, many websites have some sort of contact form, such as that created using the Free Feedback Form Script Wizard. Some also have a search engine on their site, such as that which you see in the left column of every page on thesitewizard.com.

   When search engine robots or spiders index your site, they actually call your scripts just as a browser would. If your site is like mine, where the scripts are solely meant for the use of humans and serve no practical use for a search engine (why should a search engine need to invoke my feedback form or use my site search engine?) you may want to block spiders from the directories that contain your scripts. For example, I block spiders from my feedback form, search engine and CGI-BIN directory. Hopefully, this will reduce the load on the web server that occurs when scripts are executed by removing unnecessary executions.

   Of course there are the occasional ill-behaved robots that hit your server at high speed. Such spiders can actually bring down your server or at the very least slow it down for the real users who are trying to access it. If you know of any such spiders, you might want to exclude them too. You can do this with a robots.txt file. Unfortunately though, ill-behaved spiders often ignore robots.txt files as well.

2. It Can Save Your Bandwidth

   If you look at your website’s web statistics, you will undoubtedly find many requests for the robots.txt file by various search engine spiders. The search engines try to retrieve the robots.txt file before indexing your website, to see if you have any special instructions for them.

   If you don’t have a robots.txt file, your web server will return a 404 error page to the engine instead. For those who have customized their 404 error document, that customised 404 page will end up being sent to the spider repeatedly throughout the day. Now, if you have customized your 404 page, chances are that it’s bigger than the standard server error message “404 File Not Found” (since you will want your error page to say more than the default error message). In other words, failing to create a robots.txt will cause the search engine spider to use up more of your bandwidth as a result of its repeated retrieval of your large 404 error file. (How much more depends, of course, on the size of your 404 error page.)

   Some spiders may also request for files which you feel they should not. For example, some search engines also index graphic files (like “.gif”, “.jpg” and “.png” files”). If you don’t want them to do so, you can ban it from your graphic files directory using your robots.txt file.

3. It Removes Clutter from your Web Statistics

   I don’t know about you, but one of the things I check from my web statistics is the list of URLs that visitors tried to access, but met with a 404 File Not Found Error. Often this tells me if I made a spelling error in one of the internal links on one of my sites (yes, I know — I should have checked all links in the first place, but mistakes do happen).

   If you don’t have a robots.txt file, you can be sure that /robots.txt is going to feature in your web statistics 404 report, adding clutter and perhaps unnecessarily distracting your attention from the real bad URLs that need your attention.

4. Refusing a Robot

   Sometimes you don’t want a particular spider to index your site for some reason or other. Perhaps the robot is ill-behaved and spiders your site at such a high speed that it takes down your entire server. Or perhaps you prefer that you don’t want the images on your site indexed in an image search engine. With a robots.txt file, you can exclude certain spiders from indexing your site with a robots.txt directive, provided the spider obeys the rules in that file.

    

    

How to Set Up a Robots.txt File

Writing a robots.txt file is extremely easy. It’s just an ASCII text file that you place at the root of your domain. For example, if your domain is www.example.com, place the file at www.example.com/robots.txt. For those who don’t know what an ASCII text file is, it’s just a plain text file that you create with a type of program called an ASCII text editor. If you use Windows, you already have an ASCII text editor on your system, called Notepad. (Note: only Notepad on the default Windows system is an ASCII text editor; do not use WordPad, Write, or Word.)

The file basically lists the names of spiders on one line, followed by the list of directories or files it is not allowed to access on subsequent lines, with each directory or file on a separate line. It is possible to use the wildcard character “*” instead of naming specific spiders. When you do so, all spiders are assumed to be named. Note that the robots.txt file is a robots exclusion file (with emphasis on the “exclusion”) – there is no way to tell spiders to include any file or directory.

Take the following robots.txt file for example:

The above two lines, when inserted into a robots.txt file, inform all robots (since the wildcard asterisk “*” character was used) that they are not allowed to access anything in the cgi-bin directory and its descendents. That is, they are not allowed to access cgi-bin/whatever.cgi or even a file or script in a subdirectory of cgi-bin, such as /cgi-bin/anything/whichever.cgi.

If you have a particular robot in mind, such as the Google image search robot, which collects images on your site for the Google Image search engine, you may include lines like the following:

This means that the Google image search robot, “Googlebot-Image”, should not try to access any file in the root directory “/” and all its subdirectories. This effectively means that it is banned from getting any file from your entire website.

You can have multiple Disallow lines for each user agent (ie, for each spider). Here is an example of a longer robots.txt file:

The first block of text disallows all spiders from the images directory and the cgi-bin directory. The second block of code disallows the Googlebot-Image spider from every directory.

It is possible to exclude a spider from indexing a particular file. For example, if you don’t want Google’s image search robot to index a particular picture, say, mymugshot.jpg, you can add the following:

Remember to add the trailing slash (“/”) if you are indicating a directory. If you simply add

the robots will be disallowed from accessing privatedata.html as well as privatedataandstuff.html as well as the directory tree beginning from /privatedata/ (and so on). In other words, there is an implied wildcard character following whatever you list in the Disallow line.

Where Do You Get the Name of the Robots?

If you have a particular spider in mind which you want to block, you have to find out its name. To do this, the best way is to check out the website of the search engine. Respectable engines will usually have a page somewhere that gives you details on how you can prevent their spiders from accessing certain files or directories.

Common Mistakes in Robots.txt

Here are some mistakes commonly made by those new to writing robots.txt rules.

1. It’s Not Guaranteed to Work

   As mentioned earlier, although the robots.txt format is listed in a document called “A Standard for Robots Exclusion”, not all spiders and robots actually bother to heed it. Listing something in your robots.txt is no guarantee that it will be excluded. If you really need to protect something, you should use a .htaccess file to password-protect the directory (if you are running your site on an Apache server).

2. Don’t List Your Secret Directories

   Anyone can access your robots file, not just robots. For example, typing http://www.google.com/robots.txt will get you Google’s own robots.txt file. I notice that some new webmasters seem to think that they can list their secret directories in their robots.txt file to prevent that directory from being accessed. Far from it. Listing a directory in a robots.txt file often attracts attention to the directory. In fact, some spiders (like certain spammers’ email harvesting robots) make it a point to check the robots.txt for excluded directories to spider.

3. Only One Directory/File per Disallow line

   Don’t try to be smart and put multiple directories on your Disallow line. This will probably not work the way you think, since the Robots Exclusion Standard only provides for one directory per Disallow statement.

It’s Worth It

Even if you want all your directories to be accessed by spiders, a simple robots file with the following may be useful:

With no file or directory listed in the Disallow line, you’re implying that every directory on your site may be accessed. At the very least, this file will save you a few bytes of bandwidth each time a spider visits your site (or more if your 404 file is large); and it will also remove Robots.txt from your web statistics bad referral links report.

Comments Off

Complete SEO summary.

The <title> tag

The <title> tag is the text shown in the windows-bar of the browser window, is the most important element of the webpage. You must include your most important keyword phrase(s) in this spot.

However, make sure that you write a phrase that makes sense. Many search engines may use this line in their result listings, and you want people to click on that link.

Do not put the name of the site first, unless the name contains the essential keyword phrase.

Instead of

<title>Alfred’s Reading Emporium</title>

go for

<title>Used books, magazines, periodicals, posters and postcards from Alfred’s Reading Emporium in Boulder, Colorado</title>

Some SEO-experts would even argue that you should drop “Alfred” altogether. The only drawback with this is that the browsers use this text when generating bookmarks/favorites.

Please note the inclusion of the geographical location. If your business or activity is limited to one district, say so. Searchers are prone to include geographic names in their queries.

Some suggest that it helps to use multiple <title> tags. Maybe it does in some search engines. Still, it tastes of spam (cheating), and the risk of getting banned by the search engine does not make it worthwhile.

The <meta> tags

Meta tags are text written into the HTML code that describes your webpage to the search engines, but which is not visible in a browser window. They are placed after the </title> tag, but before the <body> tag.

There are only three meta tags of importance for search engine optimization. All others can (and should) be left out. Here are examples of the three:

<META NAME=”KEYWORDS” CONTENT=”old and used books paperbacks periodicals magazines newspapers papers postcards post card poster old prints”>

<META NAME=”DESCRIPTION” CONTENT=” Alfred’s Reading Emporium in Boulder, Colorado offers you a large collection of used books, paperbacks, magazines, periodicals, newspapers, prints, posters and postcards.”>

<META NAME=”ROBOTS” CONTENT=”ALL”>

The NAME part of the tag, tells the search engine what kind of meta tag this is.

The keywords metatag

In the KEYWORDS meta tag you may enter keywords and key phrases of relevance to the content of the page. Please note the double quotation marks. Many choose to separate the keyword phrases with commas.

We normally do not include commas, partly because they take up space, and partly because leaving them out makes it possible to include more combinations of keywords. “web search tutorial” will give you no less than three queries: “web search tutorial,” “web search,” and “search tutorial” in addition to the three individual words.

Do not repeat the same keyword more than two or three times. However, if you must repeat it, keep the occurrences apart.

Do not enter keyword phrases that are of no relevance to the content of the page.

Some SEO experts argue that you should not include any keyword phrases that are not present in the visible text on the webpage, as some search engines may interpret this as spam.

We have not found proof of this (unless you enter phrases of an “adult” nature to your page on home knitting).You may also enter misspellings, and variations of the same word (capitalized/non-capitalized, plural/singular). However, do not go overboard on this.

You should note, however, that there are very few search engines that take the keyword metatag into consideration these days. Yahoo! and Ask apparently do take it into considerations, which is why you probably should include one — just to be sure.

The description metatag

The DESCRIPTION meta tag is important as some search engines use this text for the description given in the search result listings.

This text should be a normal sentence that gives the searcher exact information on what you have to offer. Do include the main keyword phrase(s). Note that no search engine displays more than 250 characters (spaces, commas and periods included), and often less. This is why you should enter the most important part of your description first.

The robots tag

The listed version of the ROBOTS- tag (ALL) tells the search engine to index the page. However, the CONTENT=”ALL” alternative is actually superfluous. Search engines will also index pages without this tag.

<META NAME=”ROBOTS” CONTENT=”NOINDEX, NOFOLLOW”>

indicates that a robot should neither index a document, nor analyze it for links.

<META NAME=”ROBOTS” CONTENT=”NOINDEX, FOLLOW”>

means that a robot should not index this document, but that it should index the pages it is pointing to.

<body> text

We encourage you to include at least four or five paragraphs of informative text on every page (preferably more than 250 words, absolute minimum: 100). Include the selected keyword phrase several times throughout the text, as some search engines reward some repetition.

Again, do not go overboard on this. The language must seem natural and search engines will punish too many repetitions. We have also reason to believe that Google will become suspicious if you include only one form of the term, hence see to it that you include at least two variations of the keyword (e.g. “search engine” and “search engines”).

Also remember that people are going to read this. Good copy sells.

Make sure you include the keyword phrase in the first paragraph of text. By the first paragraph we mean a block of text delineated by a <p> and a</p> tag, consisting of at least two full sentences. Some search engine optimization experts even argue that it helps to start this very first sentence with the relevant keywords.

The search engines are looking for natural language, and give standard text a higher reward than, let’s say, a one-word menu choice.

Sometimes it helps to include the phrase in the last paragraph of a page as well.

Bold text and large sized fonts may help in some search engines. Again: Don’t overdo it.

Keyword spam

Previously webmasters would include so-called “hidden keywords” in order to boost ranking. A popular alternative was to include phrases in comment links, like this: <!— search engine keywords –>. This no longer works.

Others would use a non-functioning form tag:

<INPUT type=”hidden”
name=”butterfly”
value=”mountain ringlet”>.

Don’t do it! The way the search engines feel about spam (“cheating”) right now, they may soon decide to ban sites for doing this.

Keyword density

Search engine marketing experts often try to calculate the keyword density of each search engine and tailor pages for each and every one of them. The keyword density is the relationship between the total number of words on a page and how many times the keyword phrase is included.

To calculate the density, paste the viewable text of the webpage into Word or another word processor or editor and let the program count the total number of words. Then use the search (or find/replace) function to count the keyword phrase. Divide the occurrence of the keyword phrase with the total number of words to find the keyword density.

The optimal keyword density varies from search engine to search engine and over time. Normally it is somewhere between 2 and 5 percent.

However, do not get obsessed with this. Nowadays search engines consider so many factors that the density alone will neither win or lose the race for you.

The Pandia Search Engine Optimization Gateway includes a list of tools you may use when analyzing your use of keywords.

Headline Tags

Do include the keyword phrases in headlines. If possible, use the standard headline tags (<h1>, <h2> etc.)

We know that some Web designers avoid these in order to gain control over the visible presentation of the webpage.

However, by using cascading style sheets in combination with the headline tags, you will get even better control of the layout. Put your keyword phrase in the first top-level headline (whether this is <h1> or <h2>).

Other elements

Many search engines will register the image ALT tags, which you can use to describe pictures to people who are using non-graphic browsers (such as Lynx or browsers for the visually impaired) or surfers that have images turned off in order to gain download speed.

Obviously these tags should include a description of the graphic element. Still, the fact that most Web designers include logos and image-based headlines gives you ample opportunity to include keyword phrases.

<A HREF=books/index.html”><IMG
SRC=”graphics/books.jpg” WIDTH=”60″
HEIGHT=”72″ HSPACE=”4″ VSPACE=”4″
BORDER=”0″ ALT=”From Alfred’s
collection of rare and used books, folios,
paperbacks, hardbacks, manuscripts and
publications. “></A>

You can also make use of the link TITLE-tag (not to be confused with the <title>-tag at the top of the HTML-file). This is an addition to the link-tag (or any tag) that that lets you describe the link more thoroughly. The text pops up when someone is moving his or her mouse pointer over the link:

<A HREF=”french-cars/renault.html” TITLE=”Renault car models of 2001″>more about Renault</A>.

Keywords in links and URLs

You should also include keyword phrases in links. Not only do these links improve the ranking of this page; it may also boost the relevancy of the page the link is pointing to. Hence <A HREF=”file.html”>Click here!</a> is a wasted opportunity.

By the way, you could also name files after the keyword phrase, as well as directories, as some engines take the URL into consideration when calculating the ranking. This is why so many optimization experts argue that you should get a domain name that includes your main keyword.

Given that all the good names are taken, this is easier said than done, but you can at least include the keywords in directory and file names, like this: <A HREF=”butterfly/african-butterfly.html”>on African butterflies</a>.

Comments Off