✨ graphql service endpoint for catalogd#2100
Conversation
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
✅ Deploy Preview for olmv1 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2100 +/- ##
==========================================
- Coverage 68.24% 67.30% -0.95%
==========================================
Files 145 148 +3
Lines 10698 11263 +565
==========================================
+ Hits 7301 7580 +279
- Misses 2869 3129 +260
- Partials 528 554 +26
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
closing this as stale - please reopen if needed |
grokspawn
force-pushed
the
catd-graphql-play
branch
from
c7edbd2 to
559b18f
Compare
There was a problem hiding this comment.
Pull Request Overview
This pull request refactors the catalogd storage layer to introduce GraphQL support for querying catalog data. The changes extract HTTP handler logic into a separate server package, introduce a service layer for GraphQL schema management with caching, and replace direct struct initialization with a constructor pattern for LocalDirV1.
- Introduces a new GraphQL endpoint at
/api/v1/graphqlwith dynamic schema generation - Refactors HTTP handlers into a dedicated
serverpackage with cleaner separation of concerns - Adds a GraphQL service layer with schema caching to improve performance
- Updates
LocalDirV1to use a constructor pattern (NewLocalDirV1) for proper initialization
Reviewed Changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/catalogd/storage/localdir.go | Refactored to use constructor pattern, added GraphQL service integration, moved HTTP handlers to server package |
| internal/catalogd/storage/localdir_test.go | Updated all test instantiations to use new NewLocalDirV1 constructor |
| internal/catalogd/storage/http_preconditions_check.go | Removed (moved to server package with simplified implementation) |
| internal/catalogd/server/handlers.go | New file implementing HTTP handlers extracted from storage layer |
| internal/catalogd/server/http_helpers.go | New file with simplified HTTP precondition checking |
| internal/catalogd/service/graphql_service.go | New GraphQL service with caching for schema generation |
| internal/catalogd/graphql/graphql.go | New dynamic GraphQL schema generation implementation |
| internal/catalogd/graphql/graphql_test.go | Tests for GraphQL schema discovery |
| internal/catalogd/graphql/discovery_test.go | Additional comprehensive tests for schema discovery edge cases |
| internal/catalogd/graphql/sample-queries.txt | Documentation of sample GraphQL queries |
| internal/catalogd/graphql/README.md | Documentation for GraphQL integration |
| cmd/catalogd/main.go | Updated to use NewLocalDirV1 constructor |
| go.mod, go.sum | Added graphql-go/graphql dependency |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
grokspawn
force-pushed
the
catd-graphql-play
branch
from
b9a05a8 to
58fbad3
Compare
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 15 out of 16 changed files in this pull request and generated 12 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
openshift-merge-robot
added
the
needs-rebase
|
closing as stale |
grokspawn
added
the
lifecycle/frozen
grokspawn
force-pushed
the
catd-graphql-play
branch
from
346cf0d to
61a60be
Compare
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
openshift-merge-robot
removed
the
needs-rebase
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 18 out of 19 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
grokspawn
force-pushed
the
catd-graphql-play
branch
from
2752f1f to
e358467
Compare
grokspawn
force-pushed
the
catd-graphql-play
branch
from
e358467 to
0d2d4e4
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 20 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (2)
internal/shared/util/tlsprofiles/mozilla_data.go:17
internal/shared/util/tlsprofiles/mozilla_data.goappears to have been truncated to ~17 lines (missing closing braces and theintermediateTLSProfile/oldTLSProfiledefinitions). This will not compile and also breaks TLS profile selection at runtime. Re-runhack/tools/update-tls-profiles.sh(and ensure the generated file is fully committed and gofmt’d) so all profiles are present and themodernTLSProfileliteral is complete.
import (
"crypto/tls"
)
var modernTLSProfile = tlsProfile{
ciphers: cipherSlice{
cipherNums: []uint16{
tls.TLS_AES_128_GCM_SHA256,
tls.TLS_AES_256_GCM_SHA384,
tls.TLS_CHACHA20_POLY1305_SHA256,
internal/catalogd/storage/localdir_test.go:252
- The new
/api/v1/graphqlHTTP route isn’t covered by the existing storage/server handler tests (e.g.,TestLocalDirServerHandlerandTestMetasEndpointcover/alland/metasonly). Add an HTTP-level test that exercises the GraphQL endpoint (method handling, invalid body/query, and a basic successful query) to prevent regressions in request parsing and wiring.
func TestLocalDirServerHandler(t *testing.T) {
store := NewLocalDirV1(t.TempDir(), &url.URL{Path: urlPrefix}, MetasHandlerDisabled, GraphQLQueriesDisabled)
if store.Store(context.Background(), "test-catalog", createTestFS(t)) != nil {
t.Fatal("failed to store test catalog and start server")
}
testServer := httptest.NewServer(store.StorageServerHandler())
defer testServer.Close()
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 20 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
grokspawn
force-pushed
the
catd-graphql-play
branch
from
a1b458e to
4cb4c1e
Compare
openshift-ci
Bot
added
the
needs-rebase
Signed-off-by: grokspawn <jordan@nimblewidget.com>
Signed-off-by: grokspawn <jordan@nimblewidget.com>
grokspawn
force-pushed
the
catd-graphql-play
branch
from
4cb4c1e to
0b5dbfb
Compare
openshift-ci
Bot
removed
the
needs-rebase
grokspawn
changed the title
grokspawn
changed the title
grokspawn
force-pushed
the
catd-graphql-play
branch
from
0b5dbfb to
626107c
Compare
grokspawn
force-pushed
the
catd-graphql-play
branch
from
626107c to
53e8200
Compare
openshift-ci
Bot
removed
the
do-not-merge/work-in-progress
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 20 out of 21 changed files in this pull request and generated 5 comments.
Comments suppressed due to low confidence (1)
internal/catalogd/graphql/graphql.go:710
- The sample query emitted by
PrintCatalogSummaryusesbundles(...)/packages(...), but the generated GraphQL fields are schema-derived (e.g.olm.bundle→olmbundles). This output is likely to confuse users debugging catalogs. Consider emitting the sanitized schema-derived field names instead (or omitting this sample block).
if _, ok := catalogSchema.Schemas[declcfg.SchemaBundle]; ok {
fmt.Printf(" bundles(limit: 5) { name package }\n")
}
if _, ok := catalogSchema.Schemas[declcfg.SchemaPackage]; ok {
fmt.Printf(" packages(limit: 5) { name }\n")
| case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64: | ||
| baseType = graphql.Int | ||
| case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64: | ||
| baseType = graphql.Int | ||
| case reflect.Float32, reflect.Float64: |
| // Parse arguments | ||
| limit, _ := p.Args["limit"].(int) | ||
| offset, _ := p.Args["offset"].(int) | ||
|
|
| // Execute the query | ||
| result := graphql.Do(graphql.Params{ | ||
| Schema: dynamicSchema.Schema, | ||
| RequestString: query, | ||
| }) |
| // Collect all metas from the catalog filesystem | ||
| // WalkMetasFS walks the filesystem concurrently, so we need to protect the metas slice and error | ||
| err := declcfg.WalkMetasFS(context.Background(), catalogFS, func(path string, meta *declcfg.Meta, err error) error { | ||
| metasMux.Lock() | ||
| defer metasMux.Unlock() |
| fmt.Printf("\nGraphQL endpoints available:\n") | ||
| for schemaName := range catalogSchema.Schemas { | ||
| fmt.Printf(" - %ss\n", strings.ToLower(schemaName)) | ||
| } |
Signed-off-by: Jordan <jordan@nimblewidget.com>
grokspawn
force-pushed
the
catd-graphql-play
branch
from
53e8200 to
a4eaf83
Compare
| // Schema build failed - rollback by removing the catalog directory | ||
| // to maintain consistency (don't persist catalog without valid schema) |
There was a problem hiding this comment.
But isn't this only the graphql schema? What if the other schemas are OK? Do we want to fail in this case?
| if r.Method == http.MethodPost { | ||
| // Check if this is the GraphQL endpoint - must end with exactly "/api/v1/graphql" | ||
| if !strings.HasSuffix(r.URL.Path, "/api/v1/graphql") { | ||
| http.Error(w, http.StatusText(http.StatusMethodNotAllowed), http.StatusMethodNotAllowed) | ||
| return | ||
| } | ||
| } else if !allowedMethodSet.Has(r.Method) { | ||
| http.Error(w, http.StatusText(http.StatusMethodNotAllowed), http.StatusMethodNotAllowed) | ||
| return | ||
| } |
There was a problem hiding this comment.
The structure of this check means that allowedMethodSet doesn't need to have http.MethodPost in it. So maybe keep the method check first, and then special-case Post (although, not as efficient). But this seems fragile, unfortunately, you can't register a POST-only handler.
Should the string "/api/v1/graphql" be made into a constant? And it used elsewhere as needed (or broken up)?
| catalogSchema := dynamicSchema.CatalogSchema | ||
|
|
||
| // Print comprehensive summary | ||
| fmt.Printf("Dynamic GraphQL schema generation complete.\n") |
There was a problem hiding this comment.
Println or Print for here and other areas where you aren't formatting a string?
|
|
||
| # Get first 10 packages | ||
| { | ||
| packages(limit: 10) { |
There was a problem hiding this comment.
olmpackages? And other keywords are missing olm prefix (i.e. bundles and channels).
openshift-ci
Bot
added
the
needs-rebase
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Description
This PR includes two significant changes:
GraphQL data is served using the existing HTTP(S) service endpoint via a specified API path, which is only enabled when the featuregate is enabled (defaulted to off for 'stable' build; enabled for 'experimental' build).
This facilitates an approach where clients can request only the schema of interest for a query.
The previous architecture for service+storage looked like:
The new architecture splits between
in-progress RFC here
Reviewer Checklist