mstar-webserver/robots.txt

User-agent: CCBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: GPTBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: Omgilibot
Disallow: /

User-Agent: FacebookBot
Disallow: /

User-Agent: Googlebot
Disallow: /cat/awawawa
Add some metadata (robots.txt, humans.txt, .well-known) as well as /static 2023-09-30 16:42:33 +00:00			`User-agent: CCBot`
			`Disallow: /`

			`User-agent: ChatGPT-User`
			`Disallow: /`

			`User-agent: GPTBot`
			`Disallow: /`

			`User-agent: Google-Extended`
			`Disallow: /`

			`User-agent: Omgilibot`
			`Disallow: /`

			`User-Agent: FacebookBot`
Update webpage 2024-12-30 14:20:25 +00:00			`Disallow: /`

			`User-Agent: Googlebot`
			`Disallow: /cat/awawawa`