2 Commits

Author	SHA1	Message	Date
Timur Ismagilov	153430194b	Allow crawling /help/	2021-07-31 19:33:15 +05:00
handlerug	190a1d6ba8	Extract robots.txt into a file, simplify MIMEs	2021-06-23 22:44:27 +07:00