From 4bc82a19a5f5fca47d19b0d076be2371110c4131 Mon Sep 17 00:00:00 2001
From: lizrea <51385152+lizrea@users.noreply.github.com>
Date: Wed, 24 Apr 2024 09:34:21 -0500
Subject: [PATCH] Add GPTBot to robots.txt

gptbot is very aggressive, let's disallow them along with the other very aggressive bots.
---
 code/web/robots.txt | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/code/web/robots.txt b/code/web/robots.txt
index 8818754eef..74573fcba5 100644
--- a/code/web/robots.txt
+++ b/code/web/robots.txt
@@ -80,6 +80,9 @@ Disallow: /
 User-Agent: MJ12bot
 Disallow: /
 
+User-Agent: GPTBot
+Disallow: /
+
 User-agent: 360Spider
 Disallow: /