How Google and AI crawlers handle AJAX content in 2026

The term "AJAX" predates React, predates SPAs, and predates the App Router. It originally described jQuery-era pages that fetched XML via XMLHttpRequest and updated the DOM without a full page reload. Today the same pattern is everywhere - any time a component fetches data on mount, lazy-loads a section on scroll, or hydrates content from an API after page load, that's AJAX in the modern sense.

The SEO problem is unchanged. The initial HTML response that crawlers receive on first contact contains a shell with placeholders ("Loading..." or skeleton states). The actual content arrives later, after JavaScript has executed and the API has responded. Whether that content is ever indexed depends on which crawler is visiting and whether it has a JavaScript rendering pipeline.

The categories of AJAX-loaded content that most often go missing from search:

• Product listings, search results, catalogue pages populated from a /api/products endpoint after page load.
• Article body text on blog or news platforms where the CMS data is fetched client-side.
• Reviews, comments, ratings that load lazily when users scroll into view.
• Filter and facet results that re-fetch on selection without changing the URL or HTML.
• Infinite scroll pages that load additional items only when the user scrolls.
• Dynamic meta tags set by client-side libraries like React Helmet after data has arrived.

The crawler matrix in 2026:

• Googlebot renders JavaScript - including XHR/fetch calls - in a headless Chromium engine. Google has stated since 2019 that the median time-to-render is around five seconds, though for low-priority pages or constrained crawl budgets the render step can be deferred or skipped. The earlier "two waves of indexing" framing is no longer how Google describes the pipeline. Practically: AJAX content is usually indexed, sometimes with a meaningful delay.
• Bingbot uses an evergreen Chromium-based renderer (similar to Googlebot's) per Microsoft's official position, but field reports consistently show Bing's rendering is less reliable on complex SPA patterns - particularly content gated behind scroll or interaction. Bing's index also powers DuckDuckGo (one of several sources), Yahoo, and Ecosia, so Bingbot coverage matters for roughly the share of search those engines represent.
• GPTBot, ClaudeBot do not execute JavaScript per public testing and Vercel's tens-of-millions-of-fetches analysis. PerplexityBot's JavaScript handling has been reported as inconsistent. AJAX-loaded content is largely invisible to these crawlers; if your /pricing page fetches plan data via /api/plans, ChatGPT is likely to report that pricing information isn't available on the page.
• Social media bots (facebookexternalhit, LinkedInBot, Twitterbot) parse the raw HTML for Open Graph tags only. If your og:title and og:image are set client-side via React Helmet after data fetches, social previews collapse to bare URLs.
• Internal site search and SEO crawlers (Screaming Frog default mode, Sitebulb, Ahrefs spider) read raw HTML by default. JavaScript rendering is opt-in and slow, so audits often miss AJAX content unless explicitly configured.

The practical implication: even if Googlebot eventually indexes your AJAX content, AI assistants, social platforms, and many SEO tools may not - so AJAX content is more invisible than it looks if you only check Google.

Failure 1: Empty initial HTML. The page returns a shell with skeleton loaders and no real content. Crawlers without JavaScript rendering see nothing. The fix is to put critical content in the server response - either via SSR, static generation, or dynamic rendering for bots.

Failure 2: Hash-fragment routing. Older AJAX patterns used URL hashes (e.g. example.com/#/products) for client-side routes. Crawlers historically ignored everything after the # because hashes are not part of the HTTP request. This pattern is largely dead but still appears in legacy applications. The fix is to migrate to the HTML5 History API (pushState) so each route has a real URL the server responds to.

Failure 3: Client-side meta tag injection. Libraries like React Helmet, Vue Meta, and similar set <title>, meta description, and Open Graph tags after the data arrives. Social bots and AI crawlers read the initial HTML and never see the updated tags. The fix is to render meta tags server-side or to ship them as static defaults that the client can refine for human visitors.

Failure 4: Interaction-gated content. Tabs that fetch content on click, accordions that expand to reveal text, search results that only appear after a query. Crawlers don't click, scroll, or type. Any content gated behind user interaction is invisible by default. The fix is to expose the content in the initial HTML (collapsed but parseable) or to provide a non-AJAX URL that returns the full content.

Server-side rendering (SSR): render the page on the server for every request. Frameworks like Next.js, Nuxt, Remix, and SvelteKit make this straightforward. Pros: every visitor (human or bot) sees fully rendered HTML on first byte. Cons: increases server load, requires a Node.js runtime in production, and may need a refactor if your app currently fetches data only client-side.

Static site generation (SSG): pre-render pages at build time. Excellent for content that changes infrequently - product pages with stable inventory, blog posts, marketing content. Cons: not viable for highly dynamic content like personalised feeds, real-time inventory, or per-user dashboards. Tools: Next.js, Astro, Eleventy, Hugo.

Dynamic rendering: serve fully rendered HTML to bots while humans get the normal client-side application. The renderer detects bot user-agents and delivers a pre-rendered cached version. Pros: no code changes to your application, works with any framework, covers AI crawlers and social bots in addition to Google. Cons: you need a rendering service or middleware, and Google's official position has shifted since 2022 - their docs now describe dynamic rendering as a workaround rather than a long-term recommendation, while explicitly noting it isn't cloaking when bots get the same content humans eventually see. Still useful when SSR/SSG aren't realistic.

Progressive enhancement: render the core content server-side as plain HTML, then layer JavaScript interactivity on top. The page works without JS at all. Pros: simplest model, robust SEO, works in adverse network conditions. Cons: limits how interactive the app can be without significant duplication of effort. Practical for content sites; impractical for true SaaS apps.

For most teams running a React, Vue, or Angular SPA where rewriting in Next.js or Nuxt isn't on the cards, dynamic rendering is a pragmatic short-to-medium-term option. PageGlass implements it at the DNS layer - a CNAME record routes bot traffic through a pre-rendering edge service. Human visitors still get the SPA. Search engines, AI crawlers, and social bots get rendered HTML. See AJAX SEO with PageGlass for how it handles AJAX-loaded pages.

Not every async pattern hurts SEO. Three things often get flagged that are actually fine:

• Lazy-loaded images. As long as the <img> tag is in the HTML with a real src or data-src plus width/height attributes, crawlers index the image. The browser deferring its load is a performance optimisation, not an SEO issue.
• Deferred analytics and tracking scripts. Loading Google Analytics, Posthog, or Hotjar after the main content has rendered does not affect what crawlers see. The HTML they need is already there.
• Hydration after first paint. SSR frameworks render HTML on the server, ship it to the browser, and then hydrate (attach event handlers) client-side. The hydration step happens after the first paint but the content is already in the HTML when the crawler reads it. This is a "rendering" pattern, not an "AJAX" pattern.

The reliable test: visit your page, view source (Ctrl+U or Cmd+Option+U in Chrome), and search the raw HTML for the content you care about. If it's there in the initial response, crawlers will see it. If it's missing and only appears in the rendered DOM (Ctrl+Shift+I), you have an AJAX SEO problem.

Concrete test steps:

1. View page source (Ctrl+U). Search the raw HTML for the page title, h1, primary content, and meta description. Anything missing is invisible to non-rendering crawlers.
2. Run our What Bots See tool to fetch your URL with a Googlebot user-agent and see the exact HTML Google indexes.
3. Check AI crawler visibility with our ChatGPT (GPTBot) simulator, Claude (ClaudeBot) simulator, and Perplexity simulator. These are the bots that don't execute JavaScript at all.
4. Test social media previews with our OG checker - this fetches your page exactly as Facebook, LinkedIn, and Twitter scrape it.
5. Run a full audit via our SEO audit tool which compares raw HTML against rendered output and flags every gap.
6. List your key pages for AI assistants with the llms.txt generator. It is a proposed convention, not a ranking factor, and it does nothing for a page a bot cannot read.
7. Use Google Search Console URL Inspection for the official Google view. The "Tested page" tab shows the rendered HTML; "More info > HTTP response" shows what Googlebot received.

If your AJAX content is missing from the raw HTML but present in the rendered DOM, that's the gap to close.