October 5, 2026 · Theo Popov

A Non-Search Google Crawler Read Almost the Whole Registry in a Day

TL;DR: The machine plane cooled off the week after its record. AI answer-engine traffic peaked at 227 requests and 18% share on Sep 28, well under last week's 454 and 29%, and the Sep 25 record stood alone, which is exactly the single-spike outcome I said last week I would watch for. The one real surprise is a crawler-identity story: on Oct 1 a non-search Google crawler, GoogleOther, read 482 registry pages in a single day, 511 requests and every one a clean 200, against a baseline of under 20 a day, and the search Googlebot did not follow. Search Console says the gain stuck: average position held in single digits for eight straight days, 5.1 to 7.0 from Sep 26 through Oct 3, before one Oct 4 reading I distrust, though clicks are still a trickle at 2 for the week. Bing is unchanged on the only question that matters there, and no new backlinks landed again. The registry stands at 454 entries, 420 open-source.

This site is two surfaces. One is a marketing site for humans. The other is a machine-first registry of open-source GTM tools, agent-operated, human-gated, updated daily, published with structured data and plain-text twins so AI engines can read it as easily as people can. It went public last month with essentially zero search presence, so this is a from-zero visibility program reported in the open. This is the fourth review.

The record did not repeat, and that is the honest answer

Last review closed on an explicit open question: whether the Sep 25 record, 454 AI-engine requests and 29% AI share, was the start of a slope or a single OpenAI-driven spike sitting next to a quiet day. A week of data answers it. It was a spike.

AI answer-engine requests (OpenAI plus Anthropic plus Perplexity), per UTC day, with human requests, total kept requests, and AI share:

  • Sep 27: 189 AI (847 human), 1,172 total, 16% AI.
  • Sep 28: 227 AI (940 human), 1,250 total, 18% AI.
  • Sep 29: 127 AI (3,114 human), 3,302 total, 4% AI.
  • Sep 30: 101 AI (830 human), 1,026 total, 10% AI.
  • Oct 1: 127 AI (1,209 human), 1,969 total, 6% AI.
  • Oct 2: 83 AI (1,791 human), 2,019 total, 4% AI.
  • Oct 3: 149 AI (1,769 human), 2,005 total, 7% AI.
  • Oct 4: 110 AI (2,280 human), 2,452 total, 4% AI.

The high day was Sep 28 at 227 and 18%, less than half the prior week's peak. From there the plane settled into an 83-to-149 band for the rest of the week. No new record, no slope, just a return to the ordinary weekly range after one loud day. I would rather report that plainly than dress a quiet week as momentum.

One detail in the high day is worth pulling out, because it cuts against the easy story. The Sep 25 record was OpenAI-driven, 356 requests in a day. The Sep 28 high this week was not OpenAI at all; it was Perplexity, 93 requests, its biggest day in the series, while OpenAI sat at 61. Across the whole week OpenAI never once broke 68. So the two best AI days I have logged were led by two different engines. No single engine owns this plane, which is the honest shape of a site that a dozen crawlers read for a dozen different reasons.

A non-search Google crawler read the registry in a day

Here is the week's genuine surprise, and it is a crawler-identity story, which is the kind this series exists to tell accurately.

On Oct 1 the Google category jumped to 598 requests, five to ten times every other day in the window. The tempting headline is "Google started crawling." That headline is wrong, and the logs say why. Of those 598, the search crawler Googlebot was 86, and only 4 of its requests touched a registry page; the rest was robots.txt. The spike was almost entirely GoogleOther: 511 requests in one day, 482 of them to registry pages, every single one a 200, against a GoogleOther baseline that ran 6 to 19 a day for the rest of the week.

GoogleOther is not Googlebot and not Google-Extended. It is the crawler Google uses for product and research fetching, the non-search side of the house, not the crawler that builds the search index and not the training-opt-in token. So the accurate sentence is narrow: a non-search Google crawler read almost the entire registry in a single day and then went back to background levels, and the search crawler did not follow it in. The user agent even told on itself, a mobile Nexus-5X GoogleOther string, so this was a mobile-first sweep.

What this is: the registry got read, in bulk, by a part of Google, which is the whole point of publishing machine-first. What this is not: a search-ranking signal, because the search crawler did not change its behaviour at all. Single-crawler, single-day events get noted, not extrapolated into a trend, the same rule I applied to the OpenAI spike last week. I am flagging it because it is the most interesting thing the logs did all week, and withholding the verdict because one day of one crawler does not earn one.

Search Console: the single digits held

Last review's headline was average search position breaking into single digits for the first time, 5.5 on Sep 25, and the open question underneath it was whether that would hold or snap back the way a young domain's numbers usually do. It held. The daily average ran 5.1 on Sep 26, the best single day in the whole series, then 5.9, 6.6, 7.0, 6.9, 6.5, 6.1 and 6.1 through Oct 3. Eight consecutive days in the five-to-seven band. I distrust exactly one number in that run, the Oct 4 reading of 15.1, because it sits on 27 impressions and Oct 4 is the last day of an export that lags two to three days, so it is the one still settling. Next edition confirms it or corrects it.

One line to keep the story honest: this has nothing to do with Oct 1's GoogleOther sweep. That was a non-search crawler, and search position is a different system entirely; reading a cause between the two would be exactly the kind of false line this series exists not to draw.

The clicks are the part I will not inflate. Two clicks across the whole window, and across the full query list the only searches that earn a click are navigational tool-name ones, taskt, mnemiq, openexecutive, people who already know the tool and find its registry page. Category queries bring impressions and no clicks. The clearest case is the one I flagged last week: the free-tools page still ranks about position 2 for "rank tracker software", now with a month of impressions behind it and still zero clicks. Ranked second for a month and not clicked once says the problem has moved off the ranking and onto the snippet, which is the next thing to fix, not the thing to celebrate.

Generative-AI-feature impressions keep inching, 43 now across about 26 pages, up from 39 across 23 at last review. One tool page, openexecutive, is the single biggest generative magnet at 9. Tiny, still directional, still pointed the right way.

The human plane had a loud day too

Sep 29 was the biggest human day in the series, 3,114 strict-human requests, against a week that otherwise ran 830 to 2,280. I do not have a referral source pinned to it, so I am flagging it, not explaining it. Naming a number you cannot yet account for is better than quietly folding it into a trend line and hoping it holds.

Bing: unchanged, and the lever that did not move

The Bing picture is where it was. At the edge, bingbot keeps pulling about 30 registry pages a day, as it has since late September; that was never the question. The question is index coverage inside Bing Webmaster Tools, and edge logs cannot answer it, so I am not going to pretend they do. What I can report is the input to that question, and the input did not change: no new backlinks landed this week. The three-backlink authority diagnosis from a month ago still stands unrefuted, and until the link graph moves there is no reason to expect the index to. Reporting that the lever did not move is the point of doing this in the open.

What shipped

The registry kept its daily cadence and now stands at 454 entries, 420 of them open-source, up from 424 and 389 at last review. That is the compounding the whole program is built on: a little more surface every day for the crawlers, human and machine, to find.

What next

Three things to watch, plus one to fix. Whether GoogleOther's sweep was a one-off or the first of a pattern, which only more days can say. Whether the search Googlebot ever follows where the non-search crawler already went, which is the signal that would actually matter for rankings. The backlinks, still the lever, still unmoved, still the honest through-line from the first review to this one. And the one to fix: the pages rank and do not get clicked, so the next bit of work is the snippet, not the position.