# /robots.txt # # # User-agent: webcrawler # Disallow: # # User-agent: lycra # Disallow: / # # Wildcards are _not_ supported: instead of 'Disallow: /tmp/*' just say 'Disallow: /tmp'. # You shouldn't put more than one path on a Disallow line (this may change in a future version of the spec) User-agent: * Disallow: /tools Disallow: /work