#robots text file is used to control the access to web #site files and directories which should not be indexed # User-agent: * # #Block spidering of specific top level directories # #Updated 3/15/07 # # Disallow: /Admin/ Disallow: /scripts/ Disallow: /images/ Disallow: /backup-20060518/ Disallow: /backup-20060612/ Disallow: /bin/ Disallow: /bkup_08112004/ Disallow: /cgi-bin/ Disallow: /css/ Disallow: /Documents/ Disallow: /Drivers/ Disallow: /FARES/ Disallow: /includes/ Disallow: /import/ Disallow: /SubscriptionAgreements/ Disallow: /upload/ Disallow: /video/