robotstxt

Here are 4 public repositories matching this topic...

Makes it easy to add robots.txt, sitemap and web app manifest during build to your Astro app.

sitemap seo astro robots-txt sitemap-xml robotstxt webmanifest

The right robots.txt file for your project

open-source seo robots-txt robotstxt

Robots.txt parser / generator

parser generator robots-txt robots-parser robotstxt

An action to create a robots.txt file from different sources

actions robots-txt action robotstxt github-actions github-action gh-actions gh-action

Add a description, image, and links to the robotstxt topic page so that developers can more easily learn about it.

To associate your repository with the robotstxt topic, visit your repo's landing page and select "manage topics."