0
0
mirror of https://github.com/twbs/bootstrap.git synced 2024-12-14 02:24:00 +01:00
Bootstrap/site/layouts/robots.txt
XhmikosR b46f05a948 Switch to Hugo
This commit includes all the needed workarounds and most changes from the main branch for everything to work, like:

* removing empty lines in raw HTML that break output
* read .browserslistrc, CSS variables from disk instead of duplicating it
* using Hugo mounts
* using Hugo for the docs CSS/JS
* move ToC Sass code to a separate file while adapting it for Hugo

Thus, this patch makes our npm scripts faster since lint runs on one step and there's no separate docs assets processing.
2020-11-19 11:22:22 +02:00

13 lines
360 B
Plaintext

# www.robotstxt.org
{{- $isProduction := eq hugo.Environment "production" -}}
{{- $isNetlify := eq (getenv "NETLIFY") "true" -}}
{{- $allowCrawling := and (not $isNetlify) $isProduction -}}
{{ if $allowCrawling }}
# Allow crawling of all content
{{- end }}
User-agent: *
Disallow:{{ if not $allowCrawling }} /{{ end }}
Sitemap: {{ "/sitemap.xml" | absURL }}