Learn more

We care about your data, and we'd use cookies to improve your experience. By using this website, you accept our cookie policy. Privacy Policy

WhatsApp us

Scan the QR Code to chat with Divyansh via your smartphone.

or chat via desktop
All Blogs
Is Astro Good for SEO and AI Search? We Tested 207 Live Sites

Is Astro Good for SEO and AI Search? We Tested 207 Live Sites

Divyansh Agarwal - Founder Webyansh
November 11, 2026
•
Updated
November 11, 2026
•
23
min read
•
Questions
Is Astro Good for SEO and AI Search? We Tested 207 Live Sites

Summarise this article using AI

ChatGPTGrokClaudePerplexityGoogle AI

Yes. Astro is good for SEO and for AI search because it sends every page as finished HTML with no JavaScript by default, so Google, Bing and AI crawlers that never run scripts all read the same content. What Astro doesn't do is the SEO work itself. Canonicals, schema, sitemaps and crawler rules are still yours to add.

We build Astro sites for B2B SaaS teams, so on 4 November 2026 we tested whether Astro is SEO friendly on live sites instead of repeating the usual claims. We fetched 207 Astro sites the way GPTBot fetches a page, with Next.js and Webflow sites as controls, and pulled September 2026 Core Web Vitals data for six frameworks from HTTP Archive. This is what Astro SEO looks like in practice, and the defaults that quietly cost teams visibility.

Key takeaways

  • Astro's HTML is crawler-ready. On the median Astro content page we tested, 99.6% of the text a browser shows was already in the raw HTML that non-rendering AI crawlers receive. Only 4 of 201 pages fell below 80%.
  • Astro sites pass Core Web Vitals more often than most stacks: 71.1% of Astro origins on mobile in September 2026, against 48.3% for WordPress and 35.7% for Next.js, per HTTP Archive. Webflow sits level at 71.3%.
  • The weak spot is the sitemap. 116 of the 194 Astro sitemaps we found (59.8%) carry no `lastmod` date at all, against 12.9% on Next.js and 18.1% on Webflow, because the official integration doesn't add per-page dates unless you tell it to.
  • Cloudflare, which bought Astro in January 2026 and sits in front of half the Astro sites we scanned, lets site owners refuse AI training crawlers at the edge. On 55 sites with Cloudflare's AI settings, GPTBot and ClaudeBot got a 403 on 20, while ChatGPT's and Claude's search crawlers got through on 54 and 55.
  • Astro won't rank a page on its own. Rankings and AI citations still come from content, links and the metadata you add, which is why our verdict is "a strong starting point", not "an SEO strategy".

Is Astro good for SEO?

Astro is good for SEO because it removes the two technical problems that hurt JavaScript-heavy sites most: content that only exists after scripts run, and slow pages. It renders every component to HTML at build time and strips client-side JavaScript unless you opt in per component (Astro docs, islands). Everything else that ranks a page is outside the framework's reach.

Google is clear about why the first part matters. Its JavaScript SEO guide says server-side or pre-rendering "is still a great idea because it makes your website faster for users and crawlers, and not all bots can run JavaScript" (Google Search Central, updated 4 March 2026). Martin Splitt, a developer advocate on Google Search, went further about content sites: "requiring JavaScript is a drawback. It can break. It can cause problems. It will make things slower" (relayed by Search Engine Journal, 2 May 2025).

Practitioners who moved to Astro say the same thing from the other side. Joost de Valk, who founded Yoast SEO and now runs his own blog on Astro, wrote that "from an SEO perspective, static HTML on a CDN is a better starting point than most CMSes will ever give you" (joost.blog, 12 April 2026). Note the words "starting point". He then built his own head component, schema graph, sitemaps and IndexNow pings by hand.

Our position: Astro is the best default we know for a content-led B2B marketing site that cares about search, provided someone on the team owns the SEO layer. It is not a plugin. If nobody will write canonicals and schema, a CMS with an SEO plugin will beat it.

Skip Astro for SEO reasons if your site is a logged-in app where the public pages are an afterthought, or your marketers need to publish landing pages daily without a developer and won't use a headless CMS. Our Astro vs Webflow comparison covers when a visual builder is the better call.

What does Astro handle for SEO by default, and what do you add?

Astro handles rendering, images and page weight by default. It leaves metadata, sitemaps, crawler rules and international tags to you. That split is where most "is Astro SEO friendly" answers go wrong: several ranking pages and ChatGPT's own answer describe the sitemap as built in, and it isn't.

SEO jobWhat Astro does out of the boxWhat you add
HTML for crawlersPrerenders every page to static HTML (`output: 'static'` is the default)Nothing, unless you use `client:only` or server islands for content
JavaScript weightShips zero client JS for components without a `client:*` directiveKeep interactive islands small
Images`<Image />` from astro:assets resizes, sets width and height to avoid layout shift, lazy-loadsAlt text on every image
Title, meta description, Open GraphNothing. There is no default headOne head component used by every layout
Canonical tagNothing in core. Setting `site` gives Astro the base URLA self-referencing canonical in that head component
SitemapNot built in`@astrojs/sitemap`, which writes `sitemap-index.xml` and can't list on-demand routes
robots.txtNothingA static `public/robots.txt`, or an endpoint
Structured dataNothingJSON-LD per template (Organization, Article, Product, FAQ where true)
Trailing slashes`trailingSlash: 'ignore'`, so both forms can answerPick `always` or `never` and set the host to match
hreflangi18n routing builds locale URLs but writes no hreflang tagsTags in the head, or the sitemap integration's `i18n` option

Sources: Astro's configuration reference, sitemap integration, images guide and i18n guide, checked against the framework source for Astro 7.4.2.

Astro SEO defaults: what Astro handles for you and the SEO jobs you still add yourself
Astro gives you clean HTML, light pages and optimised images; titles, canonicals, schema, sitemaps, crawler rules and hreflang are still your job.

Chris Good, a digital strategist who moved from WordPress to Astro in early 2026, put the trade in one line: "the big battle is really about doing for yourself what plugins were doing for you" (chrisgood.online, 20 August 2026). There is no Yoast for Astro. There are community packages such as `astro-seo` for head tags, and the official sitemap integration, and a well-built layout component does the rest.

Do Astro sites pass Core Web Vitals more often?

Yes. In September 2026, 71.1% of the 47,277 Astro origins in the Chrome UX Report passed all three Core Web Vitals on mobile, according to the HTTP Archive Technology Report. That puts Astro level with Webflow and well ahead of WordPress and Next.js, and the lead over those two holds in every market we pulled.

Mobile origins passing Core Web Vitals, Sep 2026GlobalUnited StatesUnited KingdomIndia
Astro71.1%84.2%77.5%48.7%
Webflow71.3%80.2%79.3%39.1%
WordPress48.3%62.0%66.3%29.0%
Gatsby43.7%45.4%58.5%22.0%
Next.js35.7%51.6%50.5%21.2%
Nuxt28.8%49.1%36.9%17.8%

Source: HTTP Archive Technology Report, API pull on 4 November 2026 (origin-level CrUX field data, September 2026 crawl).

Mobile Core Web Vitals pass rates for Astro, Webflow, WordPress and Next.js in the US, UK, India and globally, Sep 2026
Astro passes Core Web Vitals on mobile more often than Next.js and WordPress in every market, and runs level with Webflow.

Two things stand out. Astro's global pass rate rose from 61.6% in September 2025 to 71.1% a year later, while the number of Astro origins on mobile grew from 17,783 to 49,450 in HTTP Archive's adoption data. So the framework kept its speed while it nearly tripled in size. And India is where the gap is widest: on slower networks and cheaper phones, Astro's 48.7% is almost ten points clear of Webflow and more than double Next.js.

Keep this in proportion. Google says Core Web Vitals "are used by our ranking systems", then adds that good scores don't guarantee top rankings and that it will still show the most relevant page "even if the page experience is sub-par" (Google Search Central, updated 22 September 2026). Roger Montti at Search Engine Journal called it "a minor ranking factor" in his own read of this dataset (27 May 2026). The data is also correlational. Astro sites skew towards small content sites, which flatters any framework. Speed is a tie-breaker and a conversion lever. It won't outrank a better page.

HTTP Archive's median Lighthouse SEO score for Astro sites was 100 on mobile in September 2026, against 96 for Next.js and Webflow and 92 for WordPress. That audit checks basics such as a title, a meta description and crawlable links, so read it as "few Astro sites forget the basics", not as a ranking forecast.

Can AI crawlers read an Astro site?

Yes, and this is Astro's biggest advantage for AI search. The crawlers behind ChatGPT, Claude and Perplexity fetch pages without running JavaScript, so they only see what's in the first HTML response, and Astro puts nearly all of a page there. On the 201 Astro content pages we tested, the median page had 99.6% of its visible text in the raw HTML.

The evidence on AI crawlers is consistent. Vercel and MERJ studied a month of crawler traffic on Vercel's network and found that none of the major AI crawlers rendered JavaScript, including GPTBot, ClaudeBot and PerplexityBot. ChatGPT's and Claude's crawlers did download script files (11.50% and 23.84% of their requests) but never ran them (Vercel, 17 December 2024). In June 2026 Andre Alpar ran a controlled test for Search Engine World with a value that only appeared after JavaScript ran: none of the seven US assistants he tested, Gemini included, reported it on a live fetch, while five of twelve assistants overall did, four of them Chinese plus Mistral (Search Engine World, 9 June 2026).

Gemini is the exception worth knowing. Martin Splitt has said Google's AI crawler "that Gemini uses also renders", using the same Web Rendering Service as Googlebot (Search Engine Roundtable, 5 May 2025). Gianluca Fiorelli's reading, which we share, is that rendering happens when Google indexes a page, not when an assistant fetches it live for a user. Either way, an Astro page doesn't depend on which lane a bot uses.

Bing says the same thing in plain words. Its current Webmaster Guidelines, which now cover Copilot answers, tell sites to avoid "hiding critical content behind client-side rendering" (Bing Webmaster Guidelines, read 4 November 2026).

What a crawler without JavaScript sees from Astro static components, hydrated islands, client:only and server islands
A crawler that doesn't run JavaScript reads static components and hydrated islands in full, but gets nothing from a client:only component and only the fallback from a server island.

When do Astro islands hide content from crawlers?

Astro islands hide content only in two cases: a component marked `client:only`, which skips server rendering entirely, and a server island marked `server:defer`, which sends its fallback content first and fetches the real content with a script (Astro directives reference; server islands guide). Islands with `client:load`, `client:idle` or `client:visible` are rendered to HTML first and then hydrated, so crawlers see their text.

In practice teams use the risky directives sparingly. Of 203 Astro inner pages we fetched, 71 contained islands. Only 7 had a `client:only` island and 1 a server island, and all 7 `client:only` pages still kept 93% to 100% of their text in the raw HTML, because the directive wrapped a widget rather than the article.

The one Astro blog post in our sample that AI crawlers could barely read (110 words of raw HTML against 2,502 rendered) used no island at all. Its article body was injected by a third-party script. That's the pattern to watch on any stack: an embedded CMS widget, a review feed or a pricing table loaded from an API. If it matters for search, render it at build time or on the server.

What did our scan of 207 live Astro sites find?

Astro sites in our sample got the page-level basics right more often than the Next.js and Webflow controls, and got sitemaps wrong more often. We tested 207 Astro sites, 101 Next.js sites and 107 Webflow sites on 4 November 2026, most of them B2B SaaS and Y Combinator marketing sites from our earlier Astro research.

For each site we read robots.txt for 15 crawlers, found the sitemap, checked for llms.txt, and fetched one content page (a blog post or docs page where one existed) without JavaScript and again in headless Chrome. The tools and raw results sit in our research folder, described under Sources and method.

What we checked (one content page per site)AstroNext.jsWebflow
Median share of visible text in the raw HTML99.6%98.6%99.0%
Pages under 80% of text in raw HTML2.0%5.4%2.0%
Canonical tag on the page94.0%66.3%64.7%
Self-referencing canonical89.1%59.8%52.9%
Any JSON-LD78.1%60.9%40.2%
Article-type schema on the page52.2%32.6%24.5%
og:image90.0%73.9%73.5%
Exactly one H194.0%88.0%82.4%
llms.txt at the root (site level)72.9%53.5%43.0%
Sitemap with no lastmod (site level)59.8%12.9%18.1%
Astro SEO scan of 207 sites: canonical, JSON-LD, raw HTML text and sitemap lastmod compared with Next.js and Webflow
Across 207 Astro sites, page-level SEO signals beat the Next.js and Webflow controls, while sitemaps without lastmod dates were far more common.

Read the metadata rows with care, because they flatter Astro for a reason unrelated to Astro. Teams that choose Astro skew technical, and technical teams write canonicals and schema. Nothing in the framework adds them. What the framework does decide is that the text is in the HTML, and that row is a tie across all three stacks. Webflow, Next.js and Astro sites all served static or server-rendered marketing pages in our sample.

Why do most Astro sitemaps have no lastmod?

Most Astro sitemaps have no `lastmod` because `@astrojs/sitemap` sets `changefreq`, `lastmod` and `priority` only as one value for the whole site, and per-page dates need a custom `serialize` function (Astro sitemap docs). Most teams install the integration and stop there. Of the 135 Astro sites serving the integration's `sitemap-index.xml`, 95 (70.4%) had no `lastmod` on any URL, and only 22 had it on nine in ten URLs.

Share of sitemaps with no lastmod dates: official Astro integration 70.4%, Next.js 12.9%, Webflow 18.1%
70.4% of sitemaps from the official Astro integration carry no lastmod dates, against 12.9% of Next.js and 18.1% of Webflow sitemaps.

Does it matter? Google says it uses `lastmod` "if it's consistently and verifiably" accurate (Google Search Central, sitemaps guide), and Bing lists IndexNow first among its discovery signals. For a 40-page site, probably not. For a blog or docs site publishing every week, a sitemap that never says what changed makes recrawling slower than it needs to be. The fix is one `serialize` function that reads each post's updated date from your content collection. Joost de Valk takes `lastmod` from git history instead, which is the most honest source if your posts live in the repo.

Are Astro sites blocking AI crawlers by accident?

Rarely by accident, and almost never the crawlers that drive citations. In our scan, 19 of 195 Astro robots.txt files (9.7%) blocked at least one AI crawler, and every one of them blocked a training crawler such as GPTBot, ClaudeBot or Google-Extended. Only one also blocked the search crawlers that ChatGPT, Claude and Perplexity use to find pages to cite.

The host matters more than the framework. Cloudflare sat in front of 102 of our 207 Astro sites, and since 1 July 2025 it has asked every new domain whether to allow AI crawlers (Cloudflare, 1 July 2025). Its managed robots.txt now adds a `Content-Signal` line, and Cloudflare says customers had switched managed robots.txt on for more than 3.8 million domains when it launched the Content Signals Policy (Cloudflare blog, 24 September 2025). Among Astro sites, 15 of 99 on Cloudflare blocked at least one AI crawler in robots.txt (15.2%), against 3 of 48 on Vercel (6.3%) and 0 of 8 on Netlify.

We then asked the edge directly. For 55 sites that showed Cloudflare's AI settings or refused GPTBot, we requested the homepage once with each crawler's published user-agent string. GPTBot and ClaudeBot got a 403 on 20 sites. OAI-SearchBot and ChatGPT-User were refused on one, Claude-SearchBot on none, PerplexityBot on two, and Googlebot and a normal Chrome browser on none. The common setup, on 15 of those 20 sites, was Cloudflare's managed robots.txt with `search=yes, ai-train=no` plus an edge rule for training crawlers. A caveat: real crawlers are verified by IP address, so a user-agent test shows what a rule keyed on the name does, not every rule a host applies.

Edge responses to AI crawlers on 55 sites: training bots refused on 20, ChatGPT, Claude and Perplexity search bots let through
Edge rules on Cloudflare-hosted sites refused AI training crawlers on 20 of 55 sites while letting ChatGPT, Claude and Perplexity search crawlers through.

Four of those 20 sites allowed GPTBot in robots.txt and refused it at the edge anyway. If you want your content in future model training, check both layers. If you only care about being cited, the default is doing what you'd want. As Gianluca Fiorelli put it, "opting out of training is not opting out of citation" (Advanced Web Ranking, 20 August 2026).

Which AI crawlers should an Astro site allow?

Allow every search and user-fetch crawler, and decide about training crawlers as a business choice. Blocking a search crawler removes you from that engine's answers. Blocking a training crawler doesn't.

Crawler tokenOwnerWhat it doesOur default for a B2B SaaS site
GooglebotGoogleSearch and AI Overviews, renders JavaScriptAllow
Google-ExtendedGoogleRobots token for Gemini training; doesn't affect Search rankingYour call; allowing it costs nothing in Search
BingbotMicrosoftBing and Copilot answersAllow, and ping IndexNow on publish
OAI-SearchBotOpenAIChatGPT search resultsAllow
ChatGPT-UserOpenAIFetches a page when a user asksAllow
GPTBotOpenAIModel trainingYour call
Claude-SearchBot, Claude-UserAnthropicClaude search and user fetchesAllow
ClaudeBotAnthropicModel trainingYour call
PerplexityBotPerplexityPerplexity search; Perplexity says it isn't used for trainingAllow
Applebot-ExtendedAppleTraining opt-out; doesn't stop ApplebotYour call

Sources: OpenAI crawlers, Anthropic crawlers, Perplexity crawlers, Google crawlers, Apple.

On Astro, robots.txt is a file you write in `public/`. If your host also manages it, as Cloudflare can, open the live `/robots.txt` after every deploy. The file you committed and the file crawlers get can differ.

Does Astro help you get cited in ChatGPT, Perplexity and AI Overviews?

Astro helps you get cited by making sure crawlers can read your pages. It doesn't decide whether they choose to quote them. Google says AI Overviews and AI Mode have "no additional technical requirements" beyond being indexed with a snippet (Google Search Central, updated 10 December 2025), so the work that wins citations is the same work that wins rankings.

Google's generative AI guide, updated in July 2026, goes through the usual extras one by one. Google Search ignores llms.txt, and structured data "isn't required for generative AI search", though it still powers rich results (Google Search Central). Ahrefs checked server logs across 137,000 domains: 28% published an llms.txt file, and 97% of those files got zero requests in the study period (Ahrefs, May 2026). John Mueller called the file "comparable to the keywords meta tag" back in April 2025 (Search Engine Journal).

So why do 72.9% of the Astro sites we scanned have one? Partly because it's a two-minute file on a static site, and partly because agent tools do read it: Chrome's Lighthouse added an llms.txt check to its experimental agentic browsing audits (Chrome for Developers, updated 5 May 2026). Our advice: ship one if it's cheap, and never count it as AI-search work.

What does earn citations, in our experience running answer engine optimization for B2B SaaS sites, is pages that answer one question directly near the top, carry specific facts with sources, and come from a brand that other sites already mention. That's the work behind the 132 pages of our client Propel that AI engines now cite. Schema helps machines read who you are. Microsoft Bing's Krishna Madhavan wrote that structured data helps search engines and AI systems understand content, and warned that "AI systems may not render hidden content", so answers shouldn't sit in tabs or accordions (Microsoft Advertising blog, 8 October 2025). Astro gives you full control over that markup. It can't write the answers for you.

Is Astro better than Next.js, WordPress or Webflow for SEO?

For a content-led marketing site, Astro is at least as good as any of them and better than Next.js and WordPress on speed. The differences that decide rankings are in how each stack is set up, not in the stack.

AstroNext.jsWordPressWebflow
HTML by defaultStatic HTML, zero JSServer-rendered HTML plus a React runtimeServer-rendered HTMLStatic HTML
Mobile CWV pass, Sep 2026 (HTTP Archive)71.1%35.7%48.3%71.3%
Text in raw HTML, median content page (our scan)99.6%98.6%not tested99.0%
SitemapIntegration you add; no per-page lastmod by defaultBuilt-in `sitemap.ts` conventionCore sitemap; plugins add moreAutomatic, with lastmod
SEO plugin ecosystemSmall community packagesLibraries, code-firstYoast, Rank MathBuilt-in fields
Who edits SEO fieldsDeveloper, or a headless CMSDeveloper, or a headless CMSMarketersMarketers

Sources: HTTP Archive Technology Report (September 2026), our scan (4 November 2026), Next.js sitemap docs, WordPress 5.5 core sitemaps.

If your team already runs Next.js for the product, our Astro vs Next.js comparison covers when a separate Astro marketing site is worth it. If marketers need to edit Astro pages themselves, the best CMS for Astro post ranks ten options. And we ran the same test for Webflow in our guide to whether Webflow is good for SEO, GEO and AEO.

What are the most common Astro SEO mistakes?

The Astro SEO failures on record are configuration mistakes, not framework limits. Every one we found came down to a URL, a sitemap or a canonical that changed when nobody was looking.

  1. Pointing robots.txt and Search Console at `/sitemap.xml`. The official integration writes `sitemap-index.xml`. Rajiv Pant moved rajiv.com from Hugo to Astro and found that "/sitemap.xml had been returning 404 for three months" (rajiv.com, 5 May 2026). In our scan, 135 Astro sites served the index file. Submit that one.
  2. Breaking canonicals in a refactor. Michael Andreuzza saw Lexington Themes drop from about 1,000 daily visitors to under 500 after canonical and routing mistakes in an Astro-to-Astro rebuild (lexingtonthemes.com, 16 December 2025).
  3. Leaving trailing slashes on `ignore`. Both `/pricing` and `/pricing/` can answer, and for prerendered pages the host decides what happens. Pick one form and set the host to redirect the other with a 301.
  4. Expecting on-demand pages in the sitemap. The integration can't generate entries for dynamic routes rendered on demand (Astro docs). Add them with `customPages` or write the sitemap yourself.
  5. Assuming i18n writes hreflang. Astro's routing builds `/de/` and `/fr/` URLs but no hreflang tags. Google treats hreflang in the head, HTTP headers or the sitemap as equivalent (Google Search Central), so pick one and add it. Only 12.9% of the Astro pages we tested had any hreflang.
  6. Loading key content from a script. See the 110-word blog post above. Build-time data fetching fixes it.
  7. Not checking the live robots.txt. A host-managed file can override the one in your repo.

Moving an existing site? Our playbook on migrating to Astro without losing rankings covers redirects, URL shapes and host defaults in detail.

What is the Astro SEO checklist?

Use this list before launch and after every major release. It's the setup we recommend for every Astro build.

  1. Set `site` in `astro.config` to your canonical origin.
  2. Build one head component: title, meta description, self-referencing canonical, Open Graph and Twitter tags. Every layout uses it.
  3. Install `@astrojs/sitemap`, add per-page `lastmod` through `serialize`, and submit `sitemap-index.xml` in Google Search Console and Bing Webmaster Tools.
  4. Write `public/robots.txt` with a `Sitemap:` line, then check the live file on the deployed host.
  5. Allow search and user-fetch AI crawlers; decide training crawlers on purpose (table above).
  6. Choose `trailingSlash: 'always'` or `'never'` and make the host redirect the other form with a 301.
  7. Render all content on the server or at build. Keep `client:only` and `server:defer` for widgets, never for the article.
  8. Add JSON-LD per template: Organization site-wide, Article or BlogPosting on posts, Product or SoftwareApplication where true, BreadcrumbList.
  9. Use `<Image />` with written alt text on every image.
  10. For multilingual sites, add hreflang in the head or through the sitemap integration's `i18n` option.
  11. Ping IndexNow on publish for Bing and the engines that share it.
  12. Fetch three key pages with JavaScript off and confirm the text is there.

How does the answer change by market?

The verdict holds in every market we looked at, but the size of Astro's speed advantage doesn't. In the United States, 84.2% of Astro origins passed Core Web Vitals on mobile in September 2026, the highest of the six stacks we pulled. In the United Kingdom, Astro (77.5%) and Webflow (79.3%) were close, with Webflow slightly ahead. In India, Astro led at 48.7%, almost ten points ahead of Webflow and more than double Next.js, because a light page matters most on slower mobile networks.

If you sell into India from a US-built site, that India gap is your argument for Astro. The page weight your US team never notices is what an Indian buyer on a mid-range Android phone waits for.

Search engines differ by market too. Google dominates the US, UK and India. Bing powers Copilot answers, and IndexNow, which Bing lists first among discovery signals, also feeds engines such as Naver in South Korea, Seznam in Czechia and Yandex (IndexNow FAQ). For China-facing pages, Alpar's test found that four Chinese assistants did run JavaScript, so the raw-HTML advantage matters less there and local hosting matters more.

What changes next for Astro and AI search?

Astro's direction since its 2021 launch has been server-first HTML, and Cloudflare's ownership since January 2026 hasn't changed that: Astro stays MIT-licensed and deployable on any host, and Cloudflare runs its own docs, website and blog on it (Cloudflare blog, 16 January 2026). Co-creator Fred K. Schott wrote that Astro "was born out of frustration" with websites built as apps and rendered in the browser (Astro blog, 16 January 2026).

Timeline of Astro and AI search from Astro's 2021 launch to Cloudflare's 2026 purchase and the 2027 AIPREF signal
From Astro's launch in 2021 to Cloudflare's purchase in 2026, the AI-search questions moved from rendering to crawler permissions.

What's changing is the permission layer. Cloudflare made AI crawling opt-in for new domains in July 2025 and added content signals in September 2025. Google says it plans to support the AIPREF training signal in early 2027 (Google crawler docs). Our forecast, and it is a forecast: within a year the main AI-search setting teams get wrong won't be rendering, which Astro has solved, but the split between training and search permissions across robots.txt, the host and the CDN. Check them together.

When is Astro the wrong choice for SEO?

Astro is the wrong choice when nobody on the team will own the SEO layer, or when the site's value sits behind a login. Astro's speed and HTML won't save a site whose canonicals, titles and schema nobody maintains, and a WordPress site with Yoast and a careful editor will outrank it.

It's also a weak fit if your marketing team must ship pages daily with no developer and no headless CMS, or if most of your important pages are personalised at request time. And if your current site passes Core Web Vitals, ranks well and gets cited, a framework change is all risk. Tim Leland, who moved his blog from WordPress to Astro, was blunt: "the framework matters far less than keeping your slugs, canonical URLs, sitemap, and overall site structure consistent" (timleland.com, 22 March 2026).

Real results back that up in both directions. Patrick Bushe moved bushe.co to Astro in May 2026 and grew from 75 Google clicks in April to 6,293 in September, and he credits most of it to new pages, not to Astro (bushe.co, 18 October 2026). Chris Humphrey saw Search Console impressions for a recipe site he moved rise from 189 to 573 in the first month after a WordPress-to-Astro move, then cautioned that "a single month proves less than the size of these jumps might suggest" (chrishumphrey.ai, 1 July 2026).

If cost is the deciding question, our breakdown of what an Astro website costs for B2B SaaS compares build and running costs with Webflow.

What should you do next?

If you're choosing a stack, use Astro when search matters and someone owns the SEO layer, and run the twelve-point checklist above before launch. If you already run Astro, fetch your live robots.txt and your sitemap today: those two files decided more in our scan than the framework did.

We build and migrate Astro sites for B2B SaaS teams, with the SEO layer, schema and crawler rules set up from day one. If you want a second pair of eyes on yours, book a scoping call on our Astro development page.

Sources and method

The scan ran on 4 November 2026 from a single connection in India. We built a pool of 433 sites from three earlier Webyansh Astro studies (mostly Y Combinator and B2B SaaS marketing sites), re-verified each stack from its live homepage (Astro generator tag or `/_astro/` assets, Next.js `/_next/` assets, Webflow site attributes) and kept 207 Astro, 101 Next.js and 107 Webflow sites. For each we parsed robots.txt with Python's `urllib.robotparser` for 15 crawler tokens, read Content-Signal lines, found the sitemap from robots.txt or the default paths and counted `lastmod` on the first three child sitemaps, checked `/llms.txt`, and picked one content page (from the sitemap first). We fetched that page without JavaScript and again in headless Chrome, and measured the share of distinct rendered words that also appear in the raw HTML. A follow-up probe requested 55 homepages once per crawler user-agent string. The sample isn't random, one page per site isn't a full crawl, and a user-agent probe isn't a verified-bot test.

Core Web Vitals and Lighthouse figures come from the HTTP Archive Technology Report API (September 2026 crawl, origin-level Chrome UX Report data), pulled on 4 November 2026. Every other figure links to its source where it appears.

  • Is Astro good for SEO?

    Yes. Astro is good for SEO because it serves every page as static HTML with no JavaScript by default, so search engines and AI crawlers read the full content, and its sites pass Core Web Vitals more often than most stacks. You still add titles, canonicals, schema, a sitemap and crawler rules yourself.

  • Can ChatGPT and other AI crawlers read Astro sites?

    Yes. ChatGPT, Claude and Perplexity crawlers fetch HTML without running JavaScript, and Astro puts the content in that HTML. In our November 2026 scan of 201 Astro content pages, the median page had 99.6% of its visible text in the raw HTML. Only client:only components and server islands hide content from them.

  • Does Astro generate a sitemap automatically?

    No. Astro needs the official @astrojs/sitemap integration, which writes sitemap-index.xml rather than sitemap.xml. It sets lastmod only site-wide unless you add a serialize function, and it can't list routes rendered on demand. In our scan, 70.4% of these sitemaps had no lastmod dates at all.

  • Is Astro better than Next.js for SEO?

    For marketing and content sites, usually yes. Both can serve server-rendered HTML, but Astro ships far less JavaScript: in September 2026, 71.1% of Astro origins passed Core Web Vitals on mobile against 35.7% for Next.js, per HTTP Archive. For logged-in apps with heavy interactivity, Next.js remains the better fit.

  • Does Astro need an SEO plugin?

    Astro has no equivalent of Yoast. You add SEO through a head component you write, the official sitemap integration and optional community packages such as astro-seo for meta tags. That gives developers full control, but someone has to own titles, canonicals and schema, or a CMS with an SEO plugin will do better.

Got a Project to discuss?

Book a call

Let’s create something out of this world together.

Have a project in mind? Contact us for expert design and development solutions. Let's discuss how we can help grow your business.

Divyansh Agarwal, founder of Webyansh

Hi, I’m Divyansh - Founder at Webyansh. 
Schedule a call with me to discuss in detail about your project and how we can help your business. You can also request for free custom quote if the scope of work is clear.

WhatsApp Divyansh