fix: ask the config service whether a site exists, not head.html

[benpeter]

Description

before

• /head.html decided whether a site exists, so a site with no head.html on the ref was answered
  404: Unable to retrieve AEM branch
• an unpreviewed ref, a preview host answering 500 and a site behind Helix authentication came
  back as that same 404
• a throw from an unreachable preview host reached the browser as a bare 500
• getPageTemplate swallowed a config store failure and answered the default template over a
  document that exists
• the store came from admin.hlx.page/ping

now

• one read of config.aem.page at scope=pipeline answers existence, head.html and the store.
  404 means no such site, any other non-200 is a 503
• contentSource.url names the store. a config that names no store reads as legacy, and names the
  site in a warning
• head.html comes off that same read, so a site behind Helix authentication renders too
• UpstreamError and reach() in src/utils/upstream.js: a read with no answer to use is a 503.
  the upstream is named in the body and in x-error, and the author is told to retry or ask an
  admin
• a save asks that same one read. for a site the config service does not know, it is refused 404
  where it used to go to da-admin
• admin.hlx.page is gone. the last call was the site token exchange in src/routes/cookie.js,
  which hardcoded the host and now posts to ${env.AEM_API}/auth/adobe/exchange
• dev/lookup-shim.js and npm run dev:lookups stand in for the config service locally
• [env.dev] now declares HLX_CONFIG_SERVICE_TOKEN, without which npm run dev 401s against
  config.aem.page. cloudflare/wrangler-action v3 to v4, since v3's wrangler ignores secrets.
  same two lines as chore(deps): update cloudflare/wrangler-action action to v4 #214

Related Issue

fixes #258. follows the review on #263.

Testing

live against config.aem.page, with the pre-change branch on a second port. the 404 body for a site
that does not exist changed on purpose, from Not found: Unable to retrieve AEM branch to
404: Site not found.

• a site whose config has contentSource: one read, routed to
  https://api.aem.live/{org}/sites/{site}/source/index.html. a legacy site: routed to
  https://admin.da.live/source/{org}/{site}/.... a site the config service does not know: 404
• a site whose config is still cached without contentSource: routed to da-admin, with
  named no content source, reading {org}/{site} as legacy in the log
• a write to a source-bus site: 405 before any store is asked. a write to a site the config service
  does not know: 404 in plain text, also before any store. a write to a legacy site that exists:
  routed to https://admin.da.live/source/{org}/{site}/...
• a save while config.aem.page cannot be reached: 503, no store asked, nothing written
• /gimme_cookie on a site behind Helix authentication: 200 with an auth_token and a 24 hour
  site_token. on a site that needs no site token, the auth cookie alone. api.aem.live and
  admin.hlx.page, asked in the same second, mint a byte-identical token
• an anonymous request and an IMS bearer are both 401
• not live: the legacy store end to end, since da-admin-local was not running, and the editor
  config 503, which has unit tests only

dropping the second lookup from a write changed no routing. both trees ran 13 behaviours per scope
against a scripted config service, and of 169 outcome pairs none writes to a different store.

Risks

HLX_CONFIG_SERVICE_TOKEN has to be set on da-ue and da-ue-stage before the first deploy. the
deploy fails red without it.

[tripodsan]

I think it was once decided that we don't need to support the case, where a DA project uses the new helix6 API for preview/publish, but still uses the DA storage.

so, in order to simplify things, we don't need to read /ping at all.

1. read config.aem.page/..?scope=pipeline (we need to add the content source, though)
2. the config contains the head.html
3. check the content-source if to use source bus (and api.aem.live).

[mhaack]

Q for @tripodsan: not sure if it is too early for this PR but would it be ok to already only use https://api.aem.live and no admin.hlx.page any more?

[tripodsan]

Q for @tripodsan: not sure if it is too early for this PR but would it be ok to already only use https://api.aem.live and no admin.hlx.page any more?

with #270 (comment), you don't need admin.hlx.page at all

(exchange token should also work: https://api.aem.live/auth/adobe/exchange)

[tripodsan]

config service pipeline scope now returns the contentSource (if it doesn't work for you, trigger a cache purge by changing the site config).

eg:

https://config.aem.page/main--da-hlx6--rofe/config.json?scope=pipeline

...
"contentSource": {
  "type": "markup",
  "url": "https://api.aem.live/rofe/sites/da-hlx6/source"
},
...

[benpeter]

tested against stage (in sync with headhtml / 8a315e8)
against {ref}--{site}--{org}.stage-ue.da.live

reads

• anonymous request, any site: 401 with the da:401 shell
• legacy site with content in stage DA: 200, stage DA document
• source-bus site: 200, from the source bus
• site unknown to config service: 404 404: Site not found.
• HEAD on known site: 200
• HEAD on unknown site: 404

writes

• to a source-bus site: 405 in plain text with Allow: GET, HEAD, OPTIONS
• site unknown to config service: 404
• legacy site: written through to stage DA

auth

• /gimme_cookie from trusted origin: 200, auth_token
• /gimme_cookie from untrusted origin: 403
• GET with that cookie, no auth header: 200
• production IMS bearer against the stage legacy site: 401