# robots.txt for https://www.niddk.nih.gov/ User-agent: * Crawl-delay: 10 Disallow: /system Disallow: /_internal Sitemap: https://www.niddk.nih.gov/sitemap.xml Sitemap: https://www.niddk.nih.gov/sitemap-sc.xml Sitemap: https://www.niddk.nih.gov/sitemap-news.xml