User-agent: * Allow: /index.asp Disallow: /admin/ Allow: /pages/ Allow: / Allow: /public/