############################### # # User-agent: archive.org_bot Disallow: User-agent: * Disallow: /cgi-bin/ Disallow: /_mm/ Disallow: /_notes/ Disallow: /_baks/ Disallow: /MMWIP/ Disallow: /search/ Disallow: /administration/ Disallow: /stayinformed/ Disallow: /resources/ Disallow: /current_agendas/ Disallow: /NewFiles/ Disallow: /getinvolved/ Disallow: /about/ Disallow: /images/ Disallow: /spotlights/ Disallow: /tourism/ Disallow: /citizens/ Disallow: /news/imagespdfsetc/ Disallow: /pdf/gisdocuments/ Disallow: /blogs/ Disallow: /blog/ Disallow: /public/ Disallow: /errors/ Disallow: /config/ Disallow: /1-unused/ Disallow: /staff-updates/ Disallow: /epayment/ User-agent: googlebot Disallow: /*.csi$ Disallow: /pdf/gisdocuments/ Disallow: /pdf/jobdescpdfs/ User-agent: MSNBot Disallow: /*.csi # # list specific files robots are not allowed to index # # # ###############################