Skip to content

Select JSON-LD metadata across scripts and support non-article types - #1030

Draft
vvinayakkk wants to merge 1 commit into
mozilla:mainfrom
vvinayakkk:jsonld-metadata-selection
Draft

vvinayakkk wants to merge 1 commit into
mozilla:mainfrom
vvinayakkk:jsonld-metadata-selection

Conversation

@vvinayakkk

@vvinayakkk vvinayakkk commented Sep 10, 2026

Copy link
Copy Markdown

Recipe pages can expose useful metadata in JSON-LD that Readability currently skips. A string @type: "Recipe" is outside the article-type filter, and an array-valued @type throws when the existing parser calls .match() on it. Simply broadening the filter also allows an earlier video or publisher block to prevent extraction of the main content's metadata.

This change accepts non-empty string types, including members of @type arrays, and separates candidate collection from selection. It collects Schema.org entities across scripts, root arrays, and @graph arrays, then selects:

  1. The first dated entity whose mainEntityOfPage matches the document or canonical URL.
  2. Otherwise, the first dated entity.
  3. If there are no dated entities, the first entity with a matching page association.
  4. Otherwise, no JSON-LD metadata; existing HTML fallbacks apply.

A date means a non-empty string datePublished. Page references support strings, @id objects, url objects when @id is absent, and arrays. Relative URLs resolve against the document base URI; fragments are ignored. Types are not ranked and candidate titles are not compared with the HTML title.

The existing extraction of title, author, description, publisher, and publication date from the selected entity is retained, including its existing name-versus-headline check. HTML metadata fallback precedence and content extraction are unchanged. Metadata is not merged across JSON-LD entities.

Examples

  • Allrecipes banana bread has a root array containing @type: ["Recipe", "NewsArticle"] and an @id page association. The selected publication date is 2000-07-11T13:40:38-04:00.
  • Good Food banana bread has an undated VideoObject before a dated Recipe in a separate script. The selection returns recipe metadata and 2025-03-31T18:10:12+01:00, rather than stopping at the video.
  • In an @graph containing an undated Organization and a dated Recipe, the organization does not prevent selection of the recipe.

The checked-in real-page fixtures preserve reduced JSON-LD selection fields and order with synthetic body text. They exercise public parse() with both jsdom and JSDOMParser and run offline.

Behavior changes and limitations

This deliberately changes selection policy beyond array-type support. An undated NewsArticle without a matching mainEntityOfPage no longer supplies its JSON-LD title, author, or description. HTML fallbacks may provide less complete metadata. Please review whether this tradeoff is appropriate for Readability.

First-dated fallback is deterministic, not a guarantee of relevance: a dated video before a dated article can win if neither has a distinguishing page association. A dated candidate also takes precedence over an undated page-associated candidate. Tests document these cases. The implementation is not a general JSON-LD processor: it does not resolve graph references, reverse mainEntity, remote contexts, or arbitrary nested entities, and does not validate dates.

Validation

Reproducible coverage: selection tests, real-page layout tests, and captured selection fields. Run npm ci, npm test, and npm run lint from the repository root.

  • npm test: 2069 passing on Node v26.5.0.
  • npm run lint: passed.
  • git diff --check: passed.
  • 32 focused selection cases, four real-page layout tests, and three page fixtures.
  • Re-ran public parse() on previously downloaded full HTML for Allrecipes, Good Food, and InStyle; expected publication dates returned with non-empty extracted content. These are saved-page checks, not CI network tests.
  • No pre-existing upstream expected fixture outputs changed.

Related: #1013 already addresses array-valued @type independently. This proposal additionally broadens accepted types and changes cross-entity selection.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant