Jekyll Robots page to exclude from robots pages that are excluded from the sitemap

robots.txt

---
layout: null
permalink: robots.txt
---
# filter pages and documents for the noindex key
{% assign noindexPages = site.pages | where: 'sitemap', false %}
{% assign noindexDocuments = site.documents | where: 'sitemap', false %}
User-agent: *
# robotstxt.org - if _config production variable is false robots will be disallowed.
{% if site.production != true %}
  Disallow: /
{% else %}
  {% if noindexDocuments or noindexPages %}
    {% for node in noindexPages %}
      Disallow: {{ node.url }}
    {% endfor %}
    {% for node in noindexDocuments %}
      Disallow: {{ node.url }}
    {% endfor %}
  {% else %}
    Disallow:
  {% endif %}
{% endif %}

In config:

production: [true/false]

Also, in your document head:

{% if site.production == true and page.sitemap != false  %}
    <META NAME="ROBOTS" CONTENT="INDEX, FOLLOW">
  {% else %}
    <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
{% endif %}

Would it be a good idea to add this line at the top of robots.txt to indicate the location of the sitemap?

Sitemap: {{ site.url }}/sitemap.xml