application icon

Scrutiny 9 Generating an XML Sitemap

Scanning your site with Scrutiny and generating an xml sitemap


Scanning the site and viewing the data

Scrutiny can generate an xml sitemap after scanning your site, and even ftp the sitemap to your server. This operation can be a scheduled action.

First you need to create a configuration for scan your site if one doesn't exist already and scan the site. Follow this tutorial if you haven't done that before.

When the scan has finished, choose Sitemap from the list of tasks. Check that this list contains your pages as you expect. See the manual for more information about how to exclude pages. If pages have been excluded, you should see a message and be able to use the adjacent button to see the list of pages and reasons for exclusion.

Why excluded from sitemap

'Change frequency' indicates to Google how often you feel that you update your pages. This doesn't oblige them to crawl your site any more or less regularly. If you generally update your pages more regularly or less regularly than weekly, you might like to change the default setting and press 'Update all'.

The 'Priority' you give your pages is a measure of how important you think each of your pages. Priority must be a value beetween 0.0 and 1.0. I suggest that you mark your home page as 1.0, one important page as 0.9, slightly less important pages as 0.5 and then decreasing from there. If you choose the 'Automatic' setting, Scrutiny will mark your starting url as 1.0 and then calculate the others based on the number of clicks from the home page, and use a logarithmic scale. ie one click from first page = 0.5, two = 0.3, three = 0.2 with all other pages = 0.2

For more specific control, you can set up some 'rules' to specify priority and update frequency for certain pages or sections of your site. You don't need to enter a full url - like the black and white lists on the settings screen, you only need to enter a partial url. This way it is possible to specify a particular url or a section of the site, eg "/engineering/". If you wish to enter an entire url, check 'match'. With match unchecked, a partial match will do.

Exporting the sitemap and transferring to your server

When you're ready, go to Export & XML and choose where you save the sitemap file.

export xml

If you'd like to ftp the sitemap to a server when you export the file, visit Preferences > Sitemap. When the file is ready to ftp, you'll see a dialog for entering the server and authentication details.

options after generating the sitemap

To automate this a little more, you can set the ftp operation as a finish action. Finish actions are carried out at the end of a scheduled scan, or if you start the scan with 'Scan with actions'. If you have a bunch of actions selected and you already have data, you can perform those actions from the Tasks screen using 'Perform actions'


Visualisation

Visualisation images are fun and can also be useful for spotting pages which are 'out on a limb' or not very well connected.

sitemap visualisation

Scrutiny can export your sitemap as a .dot file which can be opened by graphing applications such as Omnigraffle.

Since version 6, Scrutiny can display the visualisation itself, using a choice of themes. Simply switch to the 'Visualisation' tab. You can still export and display the .dot file in any external app, including my own SiteViz. SiteViz may have themes not implemented in Scrutiny.