QuarryBot fetches publicly available web pages, starting with official election and government sites, to build the search index behind OppIntell’s public-records research. If you found it in your server logs, this page explains what it is, how it behaves, and how to slow it down or keep it out.
Obeys robots.txt1 request per second per site, at mostOne published addressNo CAPTCHA or bot-protection evasion
How to recognize it
Every QuarryBot request carries exactly this User-Agent header:
QuarryBot/1.0 (+https://oppintell.com/quarrybot)
Its robots.txt token is QuarryBot. QuarryBot does not change or disguise its user agent, rotate addresses, or route requests through proxies.
The address it crawls from
QuarryBot sends every request from this one IPv4 address:
100.60.46.48
A request that claims to be QuarryBot but comes from any other address is not ours. You can allow or block this address at your firewall.
How it behaves
It reads robots.txt before crawling a site and checks it again for every redirect. If robots.txt cannot be fetched (a server error, a timeout, or a 401, 403 or 429 response), QuarryBot treats the whole site as off limits.
It waits at least 1 second between requests to the same host, and longer when your robots.txt sets a Crawl-delay (honored up to 30 seconds).
It keeps a daily request limit for every site it visits.
When it revisits a page it sends If-None-Match and If-Modified-Since (when it has them), so an unchanged page costs you a 304.
It does not log in, submit forms, keep cookies or run JavaScript, and it skips very large files.
When a site says no
QuarryBot never tries to get past bot protection. It does not solve CAPTCHAs, replay clearance cookies, imitate a browser or switch to another address or client. A challenge page or a 401, 403 or 429 response is treated as a refusal: QuarryBot stops, does not retry that request, and backs off the whole site for an increasing period.
Pages marked noindex are not added to the index as text. Pages marked noai or carrying a text-and-data-mining reservation can still appear as search results, but their content is kept out of AI-generated answers.
Blocking or slowing QuarryBot with robots.txt
Add a group for QuarryBot to the robots.txt at the root of your site. To keep it out entirely:
User-agent: QuarryBot
Disallow: /
To keep it out of part of your site:
User-agent: QuarryBot
Disallow: /private/
To slow it down (here, to one request every 10 seconds):
User-agent: QuarryBot
Crawl-delay: 10
QuarryBot reuses a robots.txt it has already fetched for up to 24 hours, so a change takes effect within a day. Rules written for all crawlers (User-agent: *) apply to QuarryBot too when your file has no QuarryBot group.
Opting out or reporting a problem
To have your site removed from QuarryBot’s crawl list, or to report a request that looks wrong, email info@oppintell.com or use the contact form. Include your site’s domain and, if you can, the time and path of the requests in question. A site we remove stays excluded even if its robots.txt changes later.
How to confirm a log line
A request is QuarryBot only when both identifiers match. The User-Agent header must be exactly the string above, and the connection must come from the one published IPv4 address. QuarryBot does not change that header, rotate addresses, or send the request through a proxy. The crawler refuses to send requests until its runtime is configured with the same address that this page publishes, so the address is public before the first request rather than after operators have already seen unknown traffic.
A header match from any other address is not this crawler. Traffic from the published address that uses a different user agent is not identified as QuarryBot either. There is no second client and no browser-impersonation mode. You can allow or block the published address at your firewall. Blocking a wider network range is not required to block this crawler, and it would also block addresses that are not QuarryBot.
What a fetch is
QuarryBot sends an HTTP GET for a publicly available page. The crawl starts with official election and government sites and builds the search index behind OppIntell’s public-records research. It reads robots.txt first. On every redirect it checks robots.txt again. If robots.txt cannot be fetched, because of a server error, a timeout, or a 401, 403, or 429, the whole site is off limits.
It does not log in, submit a form, keep cookies, or run JavaScript, so a page that exists only after a script runs is not collected that way. It skips very large files. It keeps a daily request limit for each site it visits. The spacing between two requests to the same host is at least 1 second. A Crawl-delay in your QuarryBot group, or in the User-agent: * group when you have no QuarryBot group, is honored up to 30 seconds. A longer delay is not waited out past that cap.
When QuarryBot already has a validator for a URL, the next request sends If-None-Match and If-Modified-Since if it has them. An unchanged page can answer 304, and that response does not require sending the body again. A fetched robots.txt file is reused for up to 24 hours, so a disallow or a crawl delay takes effect within a day.
A page marked noindex is not added to the index as text. A page marked noai, or a page that carries a text-and-data-mining reservation, can still appear as a search result, and its content is kept out of AI-generated answers. A challenge page or a 401, 403, or 429 is a refusal. QuarryBot does not solve a CAPTCHA, replay a clearance cookie, imitate a browser, switch addresses, or retry that request. It backs off the whole site for an increasing period.
What the index is for
The search index is one input to public-records research. Candidate profiles are assembled from public records under the rules on the methodology page, and the sourced file is described on opposition research. The live field is browsable on the race hubs and the state hubs. Names copied from official 2026 general-election lists are in the certified nominee file. An agent reads the current public serving views through the read-only research MCP server.
QuarryBot is the crawler identified on this page. It is not an account on your site, and it is not a browser session. Recognizing it, slowing it, or removing your domain from the crawl list does not change how a reader opens a candidate profile. To opt out, email info@oppintell.com or use the contact form.
Questions
How do I tell a QuarryBot request from something else?
Match both values. The User-Agent header is exactly QuarryBot/1.0 (+https://oppintell.com/quarrybot). Every request is sent from 100.60.46.48. A header match from any other address is not this crawler. QuarryBot does not change its user agent, rotate addresses, or route requests through a proxy.
What does QuarryBot fetch?
It sends HTTP GET requests for publicly available web pages, starting with official election and government sites, to build the search index behind OppIntell's public-records research. It does not log in, submit forms, keep cookies, or run JavaScript, and it skips very large files. When it already has a validator for a page, a later request sends If-None-Match and If-Modified-Since so an unchanged page can answer 304.
What happens when robots.txt cannot be read?
QuarryBot reads robots.txt before it crawls a site and checks it again on every redirect. If that fetch fails because of a server error, a timeout, or a 401, 403, or 429, the whole site is treated as off limits. A fetched file is reused for up to 24 hours. A group for QuarryBot is the specific rule. When the file has no QuarryBot group, the rules for User-agent: * apply. The wait between requests to one host is at least 1 second, and a Crawl-delay is honored up to 30 seconds.
How do I block it, slow it, or opt out?
A robots.txt group of User-agent: QuarryBot and Disallow: / keeps it off the site. Disallow: /private/ keeps it out of that path. Crawl-delay: 10 slows it to one request every 10 seconds. To be removed from the crawl list even if robots.txt later allows access, email info@oppintell.com or use the contact form. Include the domain and, when you have them, the time and path of the requests. A site removed that way stays excluded.
Does a noindex or noai page still get fetched?
A page marked noindex is not added to the index as text. A page marked noai, or a page that carries a text-and-data-mining reservation, can still appear as a search result, and its content is kept out of AI-generated answers. A challenge page or a 401, 403, or 429 is a refusal: QuarryBot does not solve a CAPTCHA, replay a clearance cookie, imitate a browser, or retry that request, and it backs off the whole site.