fix: disallow crawling from /_astro/ (#399)
Some checks failed
Code quality / quality (push) Failing after 4s
Build and Check / Astro Check for Node.js 22 (push) Failing after 5s
Build and Check / Astro Check for Node.js 23 (push) Failing after 3s
Build and Check / Astro Build for Node.js 22 (push) Failing after 3s
Build and Check / Astro Build for Node.js 23 (push) Failing after 3s

* fix: disallow crawling from `/_astro/`

* fix: add missing newline at end of robots.txt.ts file

---------

Co-authored-by: Joey Chen <JoeyC-Dev@users.noreply.github.com>
Co-authored-by: L4Ph <4ranci0ne@gmail.com>
This commit is contained in:
Joey Chen
2025-04-15 17:28:24 +08:00
committed by GitHub
parent 8cceb83e81
commit 174c5d8efc

View File

@@ -2,7 +2,7 @@ import type { APIRoute } from "astro";
const robotsTxt = `
User-agent: *
Allow: /
Disallow: /_astro/
Sitemap: ${new URL("sitemap-index.xml", import.meta.env.SITE).href}
`.trim();