SERP API measurement series from October 31: almost no Google goto links left at three of five providers

Since the baseline measurement of Oct 31, I have kept measuring. At each measurement point, I send the same queries to five SERP APIs and count how many organic results come back without a real destination URL. This page covers Oct 31 through the evening measurement on Nov 3 from 20:34 UTC, the latest complete measurement point for Germany and the US.

Result: unresolved links by provider and measurement point

I count an organic result as unresolved if its destination link is a Google goto redirect or missing. Relative goto paths and Google country domains are included. Each provider and measurement point in the DE/US desktop comparison is based on 397 to 477 organic results, and all denominators are in the raw data. I repeated Bright Data for the Nov 3 measurement from 12:24 UTC in a catch-up from 16:09 UTC.

• Oct 31 – Baseline measurement, 5 providers, 8 countries
• Nov 1 – Serper and Oxylabs with no unresolved results in Germany and the US
• Nov 2 – SerpApi reports “Resolved”, DataForSEO describes the goto limit
• Nov 3, 12:24 UTC – Measurement point with all providers, Bright Data repeated after server errors from 16:09 UTC
• Nov 3, 20:34 UTC – Evening measurement point
• Nov 7 – next measurement point (planned)

Findings

Three providers with 0.0% to 6.5% unresolved results

At the evening measurement point on Nov 3, I find unresolved shares of 0.0% to 6.5% for Serper, SerpApi and Oxylabs in the DE/US desktop comparison. The table also shows Europe and mobile, with counts and denominators for each sample.

Desktop: Nov 3 from 20:34 UTC; usable responses in Europe 120/120 per provider, Bright Data 116/120. Mobile: Nov 3 from 20:44 UTC, 10 queries each for DE and US, 20/20 usable responses per measured provider.

Oxylabs: the rest sits on a few pages

At the evening measurement point on Nov 3, the unresolved results at Oxylabs are concentrated on a few results pages with at most one direct link. That is 3/50 pages in Germany and the US and 5/120 in Europe. All other pages in these two samples contain no unresolved results.

DataForSEO: depth 20 ranges from 0.0% to 99.9%

I also changed the requested depth in the standard queue. At depth 20, the share of unresolved results varies from run to run. Depth 20 also returns results beyond the first results page, so I show page 1 separately.

At depth 10, all results and page 1 are identical. The value from Nov 2 at 21:21 UTC comes from my daily automated measurement. I used its counts but did not recount them down to the individual raw response. Its sample differs from the other runs.

DataForSEO stated on Nov 2 that the standard queue resolves all links but takes longer. In my measurement of the standard queue at depth 10, the first results page still contains unresolved results.

Statement from DataForSEO, Nov 3: the team says it is working on fully eliminating the issue in the standard queue, and that most affected users have already received a dedicated solution that resolves the URLs. I do not have that solution yet. I will test it separately and report it as a separate result. My values on this page apply to the standard queue.

These values do not show that a greater depth reliably works around goto links. I do not turn them into a usage recommendation.

In my daily automated measurement, the standard queue had previously returned 331/543 (61.0%) unresolved on Oct 31 at 07:07 UTC, 329/544 (60.5%) on Nov 1 at 18:18 UTC and 337/539 (62.5%) on Nov 2 at 06:00 UTC. At 21:21 UTC it was 544/545 (99.8%). This separate series also shows why I cannot infer a lasting property from individual points in time.

SerpApi and Serper: a new parameter value on every request

When the same direct URL with the srsltid parameter shows up in several requests, the parameter has a different value each time. That holds for 20/20 such URLs at SerpApi and 16/16 at Serper. This argues against these values being reused. It does not show which route the providers take to resolve the links. Serper describes its own route as “likely temporary”.

My assessment

I suspect that SerpApi and Serper resolve the links freshly and in parallel on every request. This fits the changing parameter values and the short median request times, which do not rise consistently with the number of links. I cannot determine the exact route from the outside. For Oxylabs, signs of caching limit what I can say.

I also suspect that the tighter control of access to search data is connected to AI search. I cannot establish Google’s motive or the cause of the measured changes. Each measurement point remains a sample taken at a particular time.

For Serper and Oxylabs, the share in Germany and the US had already dropped to zero by the next data point. A provider’s state can therefore change from one data point to the next. So my own analyses keep reading the domain from the displayed link whenever the URL is a goto URL. I expect Google to keep making access harder without shutting it off completely. This data cannot prove that.

For the background, I rely on documented sources on Google’s JavaScript check, num=100, Google’s lawsuit, goto, grounding and the declined partnership in the US judgment, the Gemini API terms and Microsoft’s API retirement. This measurement series does not recheck that history.

Further analyses

Europe: repeat test and countries

Eight countries on Nov 2 (including DE/US)

Oxylabs still has some unresolved links outside Germany and the US. For Bright Data, three of the 170 scheduled responses are missing on Nov 2. The tables compare what each provider returned, not identical individual Google results.

For DataForSEO in these six European countries, I use the separate control run from 21:06 UTC. I do not use the run from 20:34 UTC, because 300 of my own test tasks were running at the same time. That may have distorted the results through load, but I cannot establish how this works internally. The later Poland catch-up from that run is excluded from the rates as well. The control run is a later measurement, not a simultaneous replacement in the provider comparison.

The control run covers 20 results pages per country.

Desktop and mobile

For a paired test on Nov 2 from 22:34 UTC, I queried 20 terms per provider on desktop and mobile. Bright Data has 18/20 usable mobile responses.

Link traces

The link traces below refer to Nov 2.

On Nov 2, all 5,161 direct links with a checkable displayed domain have a host that matches that domain. For DataForSEO, this combines DE/US from 20:34 UTC with the European control run from 21:06 UTC. The check covers the domain, not whether the specific destination page is the right one.

The srsltid parameter cannot be reconstructed from the displayed domain name alone. Its presence fits destination URLs that originate from Google, but on its own it does not prove how the URL is currently retrieved.

Resolution test

I cannot see how the providers work internally, so I limited the test to what their responses let me measure.

Google redirects remain visible in the responses. Even when SerpApi returns the direct destination URL, an additional redirect field still contains the Google redirect with a token. That field is not the destination link I count as unresolved. The raw responses alone cannot tell whether the token is encrypted or whether providers can read it without another request.

SerpApi and Serper return fresh parameter values on repeated requests. I queried 6 search terms three times in a row, for 18 requests per provider. At SerpApi, 20 URLs with srsltid appeared in at least two requests, and all of them had a different value on every request. At Serper, the same holds for all 16 such URLs. Unchanged values: 0/20 and 0/16. This argues against the values simply coming from a cache. It does not prove that no part of the response is cached.

For SerpApi and Serper, the median request time does not rise consistently with the number of links. The duration analysis only includes desktop responses in which all links are direct. Values are median request times in seconds, with the number of responses in parentheses. For SerpApi, I also show the processing time the provider reports itself.

For Oxylabs, the median rises with the number of links. At the same time, 5 of the 12 repeated-request comparisons have fully identical link lists, and the parameter value stays the same for 10 of 17 repeatedly observed srsltid URLs. That is consistent with caching. Since I did not set an option to disable caching for Oxylabs, I cannot separate its effect.

Serper itself describes a different route. On its status page (incident “/goto links”, resolved on Oct 31 at 16:50 UTC, https://serper.betteruptime.com/), Serper writes: “We've found another way to get the real URLs, and since Oct 31 ~16:30 UTC all result links have real URLs again. This is likely temporary: if Google limits it too, the top 3 stay resolved automatically and /resolve gets the rest after about 30 seconds.” This matches my values for Germany and the US. On Oct 31 from 12:24 UTC, 202/474 Serper results were unresolved (42.6%), on Nov 1 it was 0/470 (0.0%).

Position profile November 2

The position profile only describes the returned links, not how the providers work internally.

Raw data: Download CSV. For each measurement point, group, provider and country, the file contains the scheduled queries, usable responses, results, unresolved links and catch-up times. Missing catch-ups are marked.