Internet marketing resources, ecommerce web site design tutorials and  just for fun - free cell phone ringtones!
  Taming the Beast - quality web marketing and ecommerce development services

The value of server logs

Posted by Michael Bloch in web development (Monday June 5, 2006 )

The server logs for your web site can reveal an incredible amount of information about who, or what, is visiting and what they are doing.

Server log analysis is more labor intensive than just viewing visual traffic stats reports, but analysis should be carried out from time to time to get a clearer picture of exactly what’s happening on your site.

For instance, some stats applications will tell you that x number of 404 errors have been generated for a specific period, but which requests are creating this error and where are these requests coming from? A 404 error means “file not found”, so it’s important that you track these down wherever possible and rectify them.

Another example is where your stats app reports x thousand visits from Googlebot, but which pages is Googlebot actually retrieving? Googlebot is Google’s “robot”. A search engine robot (aka spider) is simply a software program that scours the web for content by following links. If a search engine spider is hitting just a couple of pages continuously, but nothing else, this can indicate problems on your site – and that’s something that may not be apparent from your visual stats application.

If there’s been a breach of security on your site, your server logs may also assist you in tracking down the culprit.

Server logs are basically just large text files that record all activity on your site. If you’re not sure how to access yours, contact your web host.

You might find my article on server log analysis of value in determining what each component in each line in your log file means. It also covers some of the terminology used in statistics applications.

If you’re interested in learning more about search engine robots, try my search engine spider identification tutorial, which also contains information on how to block search engine spiders from areas of your site that you don’t want listed.


Need visual web site traffic statistics? Try a free service.

HTTP error/server response codes reference


Comments for The value of server logs

No comments yet.

Sorry, the comment form is closed at this time.