A trap first: page vs origin field data

When Google has no real-user (CrUX) data for a specific page, PageSpeed Insights quietly shows the origin's data instead. So a slow, low-traffic page can look "fast" because the homepage is fast. Keep the two apart:

• fieldDataPage: real users of this URL (often null on small pages)
• fieldDataOrigin: real users across the whole site

The code

I use a hosted Actor I maintain on Apify (Bulk PageSpeed & Core Web Vitals Audit). With the Python client (pip install apify-client pandas):

from apify_client import ApifyClient
import pandas as pd

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("fjordlabs/pagespeed-audit").call(run_input={
    "sitemapUrls": ["https://www.example.com/sitemap.xml"],
    "maxUrls": 200,
    "strategies": ["mobile", "desktop"],
    "alertBelowPerformanceScore": 50,
})

rows = list(client.dataset(run["defaultDatasetId"]).iterate_items())
df = pd.json_normalize(rows)
cols = ["url", "strategy", "scores.performance", "lab.lcpMs", "lab.cls",
        "fieldDataPage.overall", "fieldDataOrigin.overall", "coreWebVitalsPassed"]
print(df[cols].sort_values("scores.performance").head(15))

Sitemap indexes are followed and URLs de-duplicated, so you can point it at the root sitemap.

Find the fixes that matter most

Each row has opportunities with estimated savings. Summing them across the site shows which single fix helps the most pages:

opps = pd.json_normalize(rows, "opportunities", ["url"])
print(opps.groupby("title")["savingsMs"].agg(["count", "sum"]).sort_values("sum", ascending=False).head())

Usually it's "Reduce unused JavaScript" or "Eliminate render-blocking resources" on a shared template, so one fix lifts dozens of pages.

Weekly monitoring

Schedule it weekly. Pages under your threshold come back with alert: true, which you can route to Slack or email with Apify's integrations.

Cost: $0.004 per page audit, and failed audits are free. A 200-page site on mobile + desktop is $1.60.

Disclosure: I built and maintain this Actor. Happy to answer questions in the comments. Links to Apify include my referral code.

Originally published on DEV.

Here's how I pull it for many countries at once and get clean JSON back.

The data

For each trending search you get:

• rank and search term
• approximate searches ("10K+") plus a number you can sort on (10000)
• when the trend started (UTC)
• related news: headline, source, link
• a ready-made Google Trends explore link

The code

I use a small hosted scraper I maintain on Apify (Google Trends Trending Now). With the official Python client (pip install apify-client):

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("fjordlabs/google-trending-now").call(run_input={
    "geos": ["US", "GB", "DE", "FR", "IN", "BR", "JP"],
    "includeNews": True,
    "minTraffic": 1000,        # skip the small ones
})

for t in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f'{t["geo"]} #{t["rank"]:>2} {t["title"]} ({t["approxTraffic"]})')

Output looks like:

US # 1 tesla stock (10K+)
GB # 1 strictly come dancing (20K+)
...

Cross-country trends in pandas

The fun part is seeing what trends in several countries on the same day:

import pandas as pd

df = pd.DataFrame(client.dataset(run["defaultDatasetId"]).iterate_items())
spread = (df.groupby(df["title"].str.lower())["geo"]
            .agg(["nunique", lambda g: ", ".join(sorted(g))])
            .sort_values("nunique", ascending=False))
print(spread.head(10))

Make it a monitor

Schedule the run hourly in Apify and connect the Slack or webhook integration, and you have a "what's spiking right now" alert for every market you care about. Each trending search costs $0.001, so 10 countries every hour is about $0.10 an hour at most, and failed rows are never charged.

Free history

I also keep a daily snapshot for 20 countries and publish the trend facts as a free Kaggle dataset: Daily Google Trending Searches, 20 Countries (updated weekly). It's handy for seasonality or "how do trends spread between countries" questions.

Disclosure: I built and maintain the scraper. Questions and feature requests welcome in the comments.

Originally published on DEV.