"Find something you love, and go for it."
DIY Dollars header image 2
snorg t shirts

Don’t Waste Backlinks: Tell Googlebot About Unindexed Referrers with a 3rd-Party Sitemap

June 2nd, 2007 · 14 Comments


Many of you readers are also bloggers - I know this because I sometimes get comments to posts like “Great post!” and there’s a link in there. (That’s fine. I’m glad you stopped by. But say something useful or ask a question if you could - we don’t take kindly to spam ’round these parts.)

As a fellow blogger, I want to let you in on something you may not have thought of before - a way that you can increase the value of your site by getting more backlinks that the search engines don’t know you have. There are backlinks just sitting there, links on other people’s sites that point to your site but don’t “count” in Google or other search engines because they simply aren’t indexed.

So let me break it down. You might know the following, but let’s review some basics of links and ranking:

  • The ranking of your site is muy importante in the crazy world of websites
  • The overall most important factor in being ranked high by search engines is the number of backlinks you have
  • A backlink is a link on another site to your site
  • The more backlinks you have, the better your ranking, and the more people find you in search, the more traffic you receive, and the more valuable your site is
  • You can optimize your site like crazy (it’s a very important factor), but at the end of the day I’d say it comes down to the number of backlinks you have

So, the task at hand is to get backlinks. Right? There are a few ideas about that. These are things you’ve seen everywhere else, too…

  1. Design a blog/CMS theme. Put a link to your site in the footer. Give your awesome theme away, with the stipulation that no one messes with the footer. Every time someone uses your theme, you got a nice link to your site on every page of their site. Too hard for most of us.
  2. Sign up for every link-exchange program you can and trade links. A good thing to do, but only accomplishes so much.
  3. Buy links. Go to eBay and you can find people who will put your link on 100’s of sites. Can be expensive and you don’t always get what you pay for. (Yes people get away with selling PR4 links that aren’t really PR4 links.)
  4. Comment on blogs, one at a time, with things like “Great post!” and put your link in there. Okay method, except it’s very time-consuming and if you make throwaway comments just for the sake of getting a link there’s a good chance your comment will just get deleted.

Not to say there’s no value in these methods and other methods. Use ‘em all. We need lots of backlinks so that when Google decides to do a PR update your blog can actually get a decent rank. So where do we get these backlinks??

There are a few additional methods I know of, some of them sneaky and some of them common sense. Today, I’ll tell you a common sense one. The sneaky methods will have to wait but I will (for now) just mention that social networking sites are a great way to get massive backlinks if you know a few ways to use their systems.

Also I have to clarify a couple things: This method will not get you new backlinks but it will ensure that no existing backlinks are being wasted. Also, don’t expect massive amounts of backlinks (I’m talking 100’s or more at a time).
Another thing I have to say is “backlinks != traffic”. You might not notice a traffic increase - or if you do, it will be a quick jump and then it will level off. Remember what backlinks really help with is Google’s view of your site’s popularity which increases your PR.

Anyway, here’s what to do

First, you will need to extract a list of all your referrers from your site logs. This can be done from Urchin (export to a CSV file), but you can get this from the raw server logs as well.

  • If you have cPanel hosting, go to Raw Log Manager and make sure “Archive Logs in Your Home Directory” is checked. You can access them here every day.
  • If you have Urchin, you can use Urchin to export a file.

Whichever method you use, you want to end up with a text file full of your referrers. Now, we need to make all of these plain text URLs look pretty so Googlebot will make sweet love to your site, get this list of URLs, and index them. (There are a few ways to get this part to happen.)

So now you have a messy file that needs to be cleaned up. You want anything with your URL, Google, Yahoo, etc. to be removed and keep only unique URLs from other sources. You also want the file to end up with one URL per line, with no extra characters. (Ideally you would remove any already-indexed referrers from this list as well. If you know a scripting language like PHP, this can be done. Anyone care to help?)

If you’re a regular ‘ol lunkhead like me and need a tool to help you do this stuff, download Crimson Editor and open up the textfile of referrers in Crimson Editor (CE).

  1. You first will need to take out all the junk using the Find & Replace function (CTRL-R). You should have one URL per line in your file, but each line will look like ” * www.someurl.com/index.html”. So you will do a CTRL-R and put ” * ” in the Find field, and put nothing in the Replace field - hit backspace. Crimson Editor will then take out the extra spaces and the * symbol.
  2. Next, we need each of these plain URLs to be actual HTML-formatted links. If you have a lot of time on your hands (or if you only have a handful of URLs in the list), you can manually type ‘<a href=”http://www.someurl.com”>www.someurl.com/index.html</a>’ for each one. But we’re all smart & lazy so you want to use Find and Replace in Crimson Editor to do this work for you. Enter these in the Find & Replace and check “Use Regex”.

Find What: (http[-_a-zA-Z0-9\:\.\/\.?]+)

 

Replace with: <a href=”\1″>\1</a>

You’ll have to wait about two entire seconds, and all the URLs should be properly formatted. Just like magic.

You now have a list of links to all URLs that link to you. Basically, you have the makings of a third-party sitemap.

Save it as an HTML file, upload the file to your site, and tell Google about it (via your preferred method). Or, break up the list into manageable chunks and post it to one of your blogs (you do keep a few free blog accounts on the side, don’t you?) in multiple posts and ping Pingoat. Or…well, you can get creative with it and figure it out.

So to recap: we all have referrers that link to us that aren’t indexed. These backlinks have the potential to count for something if they get indexed by the search engines. By putting all these backlinks in a third-party sitemap page and getting the third-party sitemap indexed, you’ve given value to your referrers which adds some additional value to your site.

And everyone lives happily ever after.

Share and Enjoy:
  • del.icio.us
  • Digg
  • Technorati
  • Facebook
  • StumbleUpon
Stumble it!

Related Posts: