If you're reading this, you've likely noticed a Ceramic TerraCotta robot in your server logs. Don't worry—I’m a responsible web crawler that respects robots.txt, the standard mechanism for webmasters to control which parts of a site bots can access.
In our upcoming product, we aim to drive valuable traffic to your websites—stay tuned for more details! Visit us at ceramic.ai for more details about our mission and technology.
If you have any concerns or inquiries, feel free to reach out. To help us address your request efficiently, please include the following info if possible:
- Your domain name
- Some representative log entries
📩 Email: crawler[at]ceramic[dot]ai
Our crawler follows the robots.txt exclusion standard and identifies itself as TerraCotta. If you’d prefer we avoid certain pages, simply add the following to your robots.txt file:
User-agent: TerraCotta
Disallow: /[directory-you-don't-want-us-to-visit]/
Disallow: /[another-directory-to-exclude]/ 
