diff options
| author | anoduck <9925396+anoduck@users.noreply.github.com> | 2025-04-05 04:25:02 -0400 |
|---|---|---|
| committer | anoduck <9925396+anoduck@users.noreply.github.com> | 2025-04-05 04:25:02 -0400 |
| commit | b76076e2e305d41adfd9e584a11636c58a32c99a (patch) | |
| tree | f0c6df12790eab554362f7c020653de8d1fe6433 /static/robots.txt | |
| parent | 640338b79641fa3a1957a38f9e449b05444f579c (diff) | |
fix(Search-indexing): fixed search indexing and working on fixing missing leaflet elements
Diffstat (limited to 'static/robots.txt')
| -rw-r--r-- | static/robots.txt | 25 |
1 files changed, 20 insertions, 5 deletions
diff --git a/static/robots.txt b/static/robots.txt index dce7fd6..8bc5c8a 100644 --- a/static/robots.txt +++ b/static/robots.txt @@ -1,5 +1,20 @@ -User-agent: googlebot -Disallow: https://corruptlamar.com/Documents/otter-logs/2020-05-02_jHVsfSzwuhH7hJ6aodzYp-DyBms.pdf -Disallow: https://corruptlamar.com/emails/CAGiqvQn6rVgM9.html -Disallow: https://corruptlamar.com/OtterLogs/otter-logs/2022-10-10_STW10TheGLRbnMFfLEUs_4o5RAI.pdf -Disallow: https://corruptlamar.com/otter-logs/2022-10-10_STW10TheGLRbnMFfLEUs_4o5RAI.pdf
\ No newline at end of file +User-agent: * +Disallow: +Crawl-delay: 20 +Disallow: /Documents/ +Disallow: /emails/ +Disallow: /OtterLogs/ +Disallow: /otter-logs/ +Disallow: /reasons/ +Disallow: /.well-known/ +Allow: /evidence/ +Allow: /persons/ +Allow: /docs/ +Allow: /news/ +Allow: /legal/ +Allow: /tips/ +Allow: /communication/ +Allow: /tags/ +Allow: /about/ +Allow: /video/ +Sitemap: https://corruptlamar.com/sitemap.xml |