Bootstrap/robots.txt

mirror of https://github.com/twbs/bootstrap.git synced 2025-03-13 13:29:25 +01:00
# www.robotstxt.org
# Allow crawling of all content
User-agent: *
Disallow: 
Sitemap: https://getbootstrap.com/sitemap-index.xml
Add v5.0.0-beta1 docs. (#32256) 2020-12-07 18:28:07 +02:00			`# www.robotstxt.org`
merge in v3.2.0 changes; nuke a dupe _gh_pages dir 2014-06-26 09:46:59 -07:00			`# Allow crawling of all content`
			`User-agent: *`
Try out Astro docs deployment 2025-03-08 09:31:15 +01:00			`Disallow:`
			`Sitemap: https://getbootstrap.com/sitemap-index.xml`