The inparticular crawler
inparticularbot is the reader behind inparticular. If you found this page from a line in your server log, that line was us.
What it asks your store for
It requests the same public addresses a shopper's browser requests, and nothing else.
- your robots.txt, before anything else
- your public product listing at
/products.json - a single product at
/products/{handle}.js, for the fields that live only there - your sitemap, so the catalogue total can be confirmed by a second route
It never signs in, never submits a form, and never asks for anything behind a login. Carts, checkouts, accounts and orders are untouched. Everything it sees is what a shopper or a shopping assistant already sees when they open your store.
Why it comes at all
Shopping is moving from searching to asking, and an AI can only choose from the products it can actually read. Most catalogues cannot be read, because the facts a shopper needs sit buried in prose or were never written down. We read your catalogue and complete those facts from your own copy and named authorities, citing every value and inventing none.
The pace it keeps
One request a second to any one store, slower whenever your robots.txt asks for slower. It sends If-None-Match and If-Modified-Since on everything it has seen before, so a catalogue that has not changed costs you a 304 instead of a page. If your server answers with a rate limit, it slows down for the rest of the run rather than trying again at the same speed.
Your robots.txt decides, not us
Shopify serves a robots.txt for every store and you can edit yours from your theme. inparticularbot reads it before it requests anything and obeys what it finds, including the rules Shopify puts there by default. Nothing overrides it.
To keep it out of your store entirely, add these two lines.
User-agent: inparticularbot
Disallow: /To allow the product endpoints and refuse the rest.
User-agent: inparticularbot
Allow: /products
Allow: /sitemap.xml
Disallow: /To let it in at a slower pace, name the gap you want in seconds.
User-agent: inparticularbot
Crawl-delay: 10A change takes effect the next time your store is read, because the file is fetched fresh at the start of every run.
Anything else
If something here does not match what you are seeing, or you want your store left alone without editing a file, write and say so. A person answers.
Say hello at hello@inparticular.ai. Back to inparticular