<head> web conference: October 24-26, 2008

vspider: Just get it working!

Ok. This quick and nasty solution assumes that you can in fact get vspider to work and generate a collection. Details on how to get vspider to generate a collection are available in the ColdFusion documentation. The example used throughout this tech note is a collection based on the http://localhost/CFDOCS web pages you get whenever you select documentation in a standard CF installation.

The collection was generated from the command line using:
C:\CFusion\Verity\_nti40\bin\vspider -common c:\cfusion\verity\common -collection c:\spider\cfdocs -start http://localhost/cfdocs -indinclude *

The following steps are all you need to get the collection to return the appropriate fields in the CFSEARCH record set:

  1. Locate the collection directory branch on the server hard disk. For example, in our example this would be at c:\spider\cfdocs\. 
  2. Delete all the directories in the collection *except* for the ..\style sub-directory and it's contents. (If you know what you are doing you could just purge the collection instead). 
  3. Replace the style.sfl configuration file with our modified style.sfl - just download the file and copy it into the directory. 
  4. Rebuild the collection. In our example we'd just re-run the command line above. The point is to re-populate the collection using the modified configuration instructions in the newly replaced style.sfl file. 
  5. Run a CFSEARCH on the collection. If all is well you should see the URL of the spidered web pages appearing in your resultset.
Good luck!