fix(seo): restore redirects for deleted v6 docs and repair metadata regressions - #8144
Open
ankur-arch wants to merge 4 commits into
Open
fix(seo): restore redirects for deleted v6 docs and repair metadata regressions#8144ankur-arch wants to merge 4 commits into
ankur-arch wants to merge 4 commits into
Conversation
…egressions The May 5 v6 docs consolidation (309fe27) deleted ~150 indexed /docs/v6/** pages but only added redirects for the ORM section, leaving the rest 404ing during a period of declining search traffic. Add 125 redirect rules mapping every deleted v6 URL to its current equivalent (exact page where one exists, section index otherwise), verified with scripts/audit-redirects.mjs against the live route set. Also fix three smaller regressions found in the same investigation: - /compute shipped without a canonical URL (only page not using createPageMetadata) - every BlogPosting emitted a publisher logo pointing at the nonexistent /logo.png; use /images/logo.svg like the site Organization schema - the agentic-software-development series rename shipped without a redirect, 404ing the old sitemap-listed URL Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (12)
WalkthroughThe pull request updates production URL fallbacks, canonical metadata, redirects, markdown rewrites, and robots routes across the blog, docs, and site applications. ChangesCross-app routing and metadata
Estimated code review effort: 3 (Moderate) | ~20 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Comment |
|
The latest updates on your projects. Learn more about Argos notifications ↗︎
|
…estore blog markdown renditions Follow-ups from validating the traffic investigation against Search Console. The docs and blog robots.ts files render under each zone's basePath, so their Disallow rules never reached the live www.prisma.io/robots.txt. Port the parameter and og/api rules into the site robots.ts, which is the only robots.txt crawlers see. The ?tag= rule is deliberately not ported because blog tag pages 301 into /blog?tag= URLs. Search Console shows blog.prisma.io origin-host URLs (including ?tag=/?page= variants) indexed with impressions and zero clicks, and neither origin host serves a robots.txt at all. Add a disallow-all robots.txt on docs.prisma.io and blog.prisma.io via basePath-free rewrites; www traffic never reaches these routes. Blog markdown renditions 404 on www because the site's bare /:path*.mdx rewrite captures /blog/*.mdx before the blog zone forward and lands in the site's stub llms.mdx route (verified in production). Forward /blog/*.mdx and /blog/*.md to the blog zone, and teach the blog app the .md suffix that docs already advertises in its Link headers. Also add the production-host guard to docs and blog getBaseUrl() so canonicals, sitemaps, and JSON-LD ids can never fall back to a Vercel deployment hostname, matching apps/site. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Next.js only allows basePath:false rewrites for external destinations, so the previous rewrite failed the docs and blog builds with "Invalid rewrite found". Use a basePath:false redirect instead, which both configs already rely on for their root redirects. Google follows robots.txt redirects and treats the target as the host's robots file. Verified in dev: /robots.txt returns 307 to /docs/robots-origin.txt and the target serves the disallow-all body. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ankur-arch
marked this pull request as ready for review
August 12, 2026 08:04
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
While investigating the reported traffic decline, we found five concrete technical SEO problems across the docs, blog, and main site. This PR fixes those problems.
The analytics context in brief: the decline shown in PostHog is larger than the real decline. PostHog became consent-gated on June 22 (#7971), so it now only counts visitors who accept analytics cookies.
Search Console, which is not affected by cookie consent, still shows a real organic decline. But rankings have broadly remained stable or improved. The remaining decline appears to come mostly from fewer search impressions and fewer clicks per impression.
This PR addresses the technical issues discovered during the investigation. It does not attempt to solve changes in search demand or search-result click behavior.
What this PR changes
1. Redirect deleted Prisma v6 documentation URLs
Problem
The May 5 v6 consolidation (#7864) removed several v6 URL trees: guides, Prisma Postgres, Accelerate, Platform, AI, and the quickstarts. Redirects were only created for the ORM section. The other URLs have returned 404 since.
Why it matters
Search Console shows some of these URLs still received search traffic, roughly 600 clicks per month in March. A redirect sends that traffic and any accumulated link value to a live page. A 404 discards both.
Change
Add 125 redirects to
apps/docs/vercel.json. Each old URL maps to its exact replacement page where one exists, otherwise to the closest relevant section.Verification
apps/docs/scripts/audit-redirects.mjspasses: 1,422 redirect rules checked against 648 live documentation routes, no broken destinations, no redirect loops, no chains returning to/docs/v6.2. Move robots rules to the robots.txt crawlers actually read
Problem
Docs and blog are separate Next.js apps served under
www.prisma.iothrough path prefixes (basePath). Each defines its ownrobots.ts, but because of the prefix those files are served at/docs/robots.txtand/blog/robots.txt. Crawlers only request/robots.txtat the root, which is served byapps/site. So the rules defined in docs and blog have never affected crawlers in production.Why it matters
Those rules block crawling of URLs that should not be indexed: OG image endpoints, API routes, and parameterized listing URLs such as
/blog?page=2.Change
Port the rules into
apps/site/src/app/robots.ts, the file behind the real/robots.txt. The?tag=rule is intentionally not ported: blog tag pages currently redirect into/blog?tag=URLs, and blocking those would block a redirect destination. The right crawl behavior for tag URLs remains a follow-up.Verification
The rule list is carried over unchanged apart from the
?tag=exclusion. Only the serving location changes.3. Prevent the internal origin hosts from being indexed
Problem
The multi-zone setup uses
docs.prisma.ioandblog.prisma.iointernally as origins behindwww.prisma.io. Those hosts can also be requested directly, and they serve full copies of the same content. Neither serves a robots.txt today (both return 404 in production). Search Console shows someblog.prisma.ioURLs indexed, including?tag=and?page=variants, with impressions and zero clicks.Why it matters
Duplicate copies of pages compete with the canonical
www.prisma.ioversions in search results.Change
Serve a robots.txt on each origin host that disallows all crawling:
apps/docs/src/app/robots-origin.txt/route.tsapps/blog/src/app/robots-origin.txt/route.tsEach app redirects
/robots.txtto its route using abasePath: falseredirect. That option matches the URL without the app's path prefix, so the redirect answers atdocs.prisma.io/robots.txtrather thandocs.prisma.io/docs/robots.txt. Google follows robots.txt redirects and treats the target as the host's robots file.A redirect is used rather than a rewrite because Next.js only allows
basePath: falserewrites for external destinations. The first version of this change used a rewrite and failed the docs and blog builds; the redirect form is the same mechanism both apps already use for their root redirects.Requests that come through
www.prisma.ioare unaffected: www serves its own/robots.txtfromapps/site, and the zone routing only forwards/docs/*and/blog/*paths to the origin apps. Redirecting the whole origin hosts to www would be the stronger fix, but the zone routing itself sends requests to those hosts, so that is not an option.Verification
Confirmed both origin hosts return 404 for
/robots.txtin production today. Verified in a local dev server that/robots.txtreturns a 307 to the robots route and the target serves the disallow-all body.4. Fix markdown versions of blog posts on www
Problem
Pages are also exposed as markdown (
.mdand.mdxsuffixes) for AI agents and other machine consumers, and the docs HTTP headers advertise this. For blog posts this only works on the internal origin host. In production,blog.prisma.io/blog/<post>.mdxreturns 200 whilewww.prisma.io/blog/<post>.mdxreturns 404.The cause is rewrite ordering in
apps/site/next.config.mjs. Rewrites run in order, and the site's generic/:path*.mdxrule matches/blog/*.mdxbefore the rule that forwards blog paths to the blog app. The generic rule sends the request to the site's ownllms.mdxhandler, which is a stub that always returns 404.Change
In
apps/site/next.config.mjs, add forwarding rules for/blog/*.mdxand/blog/*.mdahead of the generic rule, following the same pattern as the existing docs forwarding rule next to them. Also add.mdhandling to the blog app so both suffixes work, matching docs.Verification
Production probes documented the broken behavior (200 on the origin host, 404 on www for the same post).
node --checkpasses on the modified config.5. Fix metadata regressions
Four small, independent fixes:
/computecanonical URL. A canonical URL tells search engines which URL is the authoritative version of a page./computeis the only site page that bypassescreatePageMetadata()and it shipped without one. Addedalternates.canonical.BlogPostingreferenced/logo.png, which does not exist. It now uses/images/logo.svg, the asset the Organization schema already uses.agentic-software-developmenttoagentic-engineeringwithout a redirect, so the old URL returned 404. Added the redirect.getBaseUrl(). IfNEXT_PUBLIC_PRISMA_URLwere ever unset in production, they would fall back to the Vercel deployment hostname and stamp the wrong host into that metadata. They now fall back tohttps://www.prisma.ioin production, matchingapps/site. This is a defensive guard. There is no evidence that wrong hosts are currently being generated.Traffic investigation: what we actually know
The short answer to "is traffic really down from 350k to 90k": no. The 350k and 90k numbers come from two different measurement regimes, so they are not comparable. Roughly, real traffic went from about 380k monthly visitors in March to an estimated 210k in July. That is still a real decline, and Search Console confirms it, but it is about half the size the dashboard suggests.
The June drop in PostHog is a tracking change (confirmed)
Since June 22 (#7971), PostHog only loads after a visitor accepts analytics cookies through CookieYes. Visitors who do not consent are no longer counted.
Every signal that cookie consent cannot affect stayed roughly flat across that date. Comparing the six weekdays before and after the change (June 16 to 21 vs June 23 to 28):
www.prisma.ioconsole.prisma.ioconsole:user_signed_up)* The before-window includes a June 17 signup spike; the weekly signup rate stays in its normal range before and after.
Only the consent-gated metric dropped sharply. The drop also happened simultaneously across every acquisition channel including direct visits, which no search or ranking change can produce. Based on the difference between PostHog and Search Console trends, the dashboard overstates the decline by roughly 2.5x.
Search traffic is genuinely down
Search Console is unaffected by cookie consent. Comparing March to July:
Google is still showing Prisma pages frequently, and their average ranking has not broadly worsened. The larger fall in clicks cannot be explained primarily by rankings.
What appears to be driving it
Fewer clicks for the same visibility.
/docs/orm/prisma-migrate/getting-startedheld position 4.6 with slightly higher impressions, yet clicks fell 69% (CTR 2.1% to 0.6%). The page kept appearing in roughly the same place in Google, but fewer people clicked it. The timing overlaps with Google's May 21 to June 2 core update and broader AI Overview expansion, but Search Console alone cannot establish the cause.Fewer searches for Prisma terms. Queries such as "prisma", "prisma seed", and "prisma migrate" received 30 to 58% fewer impressions while Prisma continued to rank near the top for them. Fewer searches are happening for those terms, rather than Prisma ranking lower. Developers shifting from search to AI assistants and summer seasonality (2025 shows a similar March-to-summer dip) are plausible contributing factors, not established causes.
Validation
Investigation
Code
/docs/v6node --checkpasses for all modifiednext.config.mjsfiles@prisma/eclipseerrors in unrelated filesOut of scope
Known issues found during the investigation, intentionally not included in this PR:
llms-full.txtbut nollms.txtindexllms.mdxroute is still a stub, so site pages have no markdown versions?tag=URLs needs a product/SEO decisionSummary by CodeRabbit