# robots.txt file for mysite.com, updated 12-0702004 # Pound sign denotes comment tag, not seen by robots # The * means all robots are to follow the same instructions User-agent: * # The following individual pages are to be excluded: #Disallow: /thispage.htm # The following entire directories, and their subdirectories are to be excluded: Disallow: /database Disallow: /images Disallow: /inc Disallow: /log Disallow: /scripts Disallow: /design Disallow: /oldsite Disallow: /Pictures # If we wanted to disclude one file from a lower level directory, it would look like this: # Disallow: /directoryname/filename.htm