Skip to content

Thwart the bots #65

@brianpipa

Description

@brianpipa

I added in better robots.txt and it looks like that helped.

Now it looks like there are multiple scrapers grabbing the KUBrowser data. Let's see if we can do something to thwart them a bit.

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions