Multiple service resources in a ServiceBinding injected as a single logical service

Hello folks!

May I propose that we allow the ServiceBinding resource to have multiple service objects, which would be projected into an application as a single logical service "account-database" ?

apiVersion: service.binding/v1alpha2
kind: ServiceBinding
metadata:
  name: account-database
spec:
  type: postgresdb
  name: account-database
  type: database 
  provider: 

  application:
    apiVersion: apps/v1
    kind:       Deployment
    name:       online-banking

  services:
    -  apiVersion: com.example/v1alpha1
       kind:       DatabaseInstance 
       name:     dbInstanceRef  # provides uri  
       prefix: ..

    - apiVersion: com.example/v1alpha1
       kind:       DatabaseCredentials
       name:     dbCredentials  # provides username and password
       prefix: ...

status:
  conditions:
  - type:   Ready
    status: 'True'

The injection could still happen under one logical service given that the intent of the list of services isn't to have multiple distinct logical services, rather the intent is to construct a single binding secret from multiple service objects.

$SERVICE_BINDING_ROOT
├── account-database
│   ├── type
│   ├── provider
│   ├── uri
│   ├── username
│   └── password

As already proposed in this specification, if the user wishes to add another backing service, the user may create a new ServiceBinding for a "different-service" and that would provide the following projection:

$SERVICE_BINDING_ROOT
├── account-database
│   ├── type
│   ├── provider
│   ├── uri
│   ├── username
│   └── password
├── a-different-service
│   ├── type
│   ├── provider
│   ├── uri
│   ├── username
│   └── password

The above proposal would enable one to use ServiceBinding without forcing the backing service ( or the application ?) to undergo changes.

This could be an extension with clarity about how the .spec.mappings works.

Multiple Services

Developers may require applications to connect with multiple services. This extension allows a ServiceBinding resource to specify more than one service.

If the .spec.services section is defined, it MUST be an array of services.

The .spec.mappings MAY define new entries with the name of the service as the key prefix.

Here is an example:

 mappings:
  - name:  accountServiceUri
    value: https://{{ urlquery .service1.username }}:{{ urlquery .service2.password }}@{{ .service3.host }}:{{ .prod-account-service.port }}/{{ .some-other-service.path }}

Baiju, as per your proposal, would an implementor need to support both spec.service and spec.services then?

@sbose78

@sbose78 the best way to experiment with this behavior, and get the semantics right, is to create a new resource whose sole purpose is to expose a synthetic provisioned service from multiple resources. If that new resource exposes the Provisioned Service duck-type, then a ServiceBinding can consume it natively.

In general, I believe the ServiceBinding resource already does too much, and should be simpler. More advanced (and specialized) behavior can be layered on top by other resources. There's certainly a balance to strike between many focused resource and a single über binding resource.

This could be an extension...

@baijum Spec extensions cannot change the shape of the ServiceBinding resource without breaking conformance with the core spec. They can add new behavior within the schema of the current resource, but not change that schema.

@scothis

Baiju, as per your proposal, would an implementor need to support both spec.service and spec.services then?

Yes, that was the idea.

But based on the feedback from @scothis, changing the ServiceBinding schema is not good. So, I came up with an alternate proposal that requires an additional CRD.

Multiservice

Developers will require applications to connect with more than one service. Creating distinct ServiceBinding resources may not be desirable for specific reasons:

ServiceBinding reconciler can inject all secrets at once and reduce potential conflicts
Application might be expecting combined secrets derived from more than one services
Reduce the proliferation of ServiceBinding resources
Less number of ServiceBinding resources might be more convenient

This extension provides a Provisioned Service called Multiservice to specify more than one service.

When the .spec.service.kind attribute is Multiservice and .spec.service.apiVersion is service.binding/v1alpha2, the ServiceBinding recociler MUST wait for Ready condition status to become True.

Resource Type Schema

apiVersion: service.binding/v1alpha2
kind: Multiservice
metadata:
  name:                 # string
  generation:           # int64, defined by the Kubernetes control plane
  ...
spec:
  services:             # []Service containing at least one entry
  - apiVersion:         # string
    kind:               # string
    name:               # string
status:
  binding:              # LocalObjectReference,
    name:               # string
  conditions:           # []Condition containing at least one entry for `Ready`
  - type:               # string
    status:             # string
    lastTransitionTime: # Time
    reason:             # string
    message:            # string
  observedGeneration:   # int64

Multiservice Example Resource

apiVersion: service.binding/v1alpha2
kind: Multiservice
metadata:
  name: combined-service
spec:
  services:
  - apiVersion: services.example.com/v1alpha1
    kind: Database
    name: database-service
  - apiVersion: services.example.com/v1alpha1
    kind: Cache
    name: cache-service
status:
  binding:
    name: combined-service-secret
  conditions:
  - type:   Ready
    status: 'True'

Service Binding with Multiservice Example

apiVersion: service.binding/v1alpha2
kind: ServiceBinding
metadata:
  name: account-service
spec:
  application:
    apiVersion: apps/v1
    kind:       Deployment
    name:       online-banking

  service:
    apiVersion: service.binding/v1alpha2
    kind:       Multiservice
    name:       combined-service

status:
  conditions:
  - type:   Ready
    status: 'True'

@sbose78

hey @sbose78 - I really like the scope you defined by:

The injection could still happen under one logical service given that the intent of the list of services isn't to have multiple distinct logical services, rather the intent is to construct a single binding secret from multiple service objects.

@baijum - with that in mind perhaps the name should be something like ComposedService, to avoid misuse of combining distinct services.

ComposedService

I changed it to Multiservice. How does that sound?

Is the intent of the Multiservice to bind multiple distinct services to an application at once, or to aggregate multiple service into a single projection? The OP proposal is for the later.

Assuming the former is the intent, I have a few thoughts:

.status.binding should be removed on Multiservice since there isn't a single secret.
what should the type, provider, env and mappings fields on the ServiceBinding do when the service is a Multiservice?
what should a custom ServiceBindingProjection look like for a Multiservice?

If the later is the intent:

how are values from the existing services merged?

@scothis

ComposedService

I changed it to Multiservice. How does that sound?

After thinking some more on this and seeing the previous comment from @scothis I ComposedService makes more sense.

But I don't have a clear answer to this question he raised:

how are values from the existing services merged?

If we just join all the services' secret keys, there is a chance for name collision. Another option is to use some prefix, probably the name of the service, or introduce another attribute as @sbose78 mentioned in the description.

Discussed this during interlock. There are still open questions about conflicting key/value pairs in the binding secrets.
One option is to use prefix as suggested in the issue description. Another option is to use the first index service as is and use a prefix field for the other services.

Labelling as RC3 for now.

Ah, missed responding to Scott. Sorry!

We could support three scenarios

Merging will be "deterministic" if keys are unique
Merging will not be "deterministic" it one or more keys are not unique. ( The user owns the risk )
Use of prefix. This has worked pretty well, we have users using this in the service binding operator implementation, though our API would love a lot of cleanup in this respect.

hi folks. One idea would be to include a spec.service.additionalResources element inside ServiceBinding, where it is an array of additional related resources. So for Strimzi as an example, perhaps KafkaCluster is the ref from spec.service, and KafkaUser is the [0] ref inside spec.service.additionalResources.

This way we don't need the additional CRD, and helps users understand that this is just for related CRs, not unrelated services (which should be a separate ServiceBinding CR).

Based on the proposal in #145, I think we can create ComposedService to compose multiple services into a single logical service.

apiVersion: mapping.service.binding/v1alpha2
kind: ComposedService
metadata:
  name: composed-service
spec:
  services:
  - apiVersion: com.example/v1alpha1
    kind:       AccountService
    name:       prod-account-service
  - apiVersion: v1
    kind:       Secret
    name:       credentials
  mappings:
  - name: username
    service: credentials
    key: username
  - name: password
    service: credentials
    key: password
  - name: host
    service: prod-account-service
    key: host
  - name: port
    service: prod-account-service
    key: port
status:
  binding:
    name: prod-account-composed-service

As you can see above, these are well-defined mappings without any ambiguity. This extension will help us during the early days where users do not want to create a Direct Secret Reference with fixed values. Once the world is full of Provisioned Service resources, probably the importance of this extension will go away!

I like the ComposedService example, but we should strive to avoid the mappings. I'd like to see clear precedence rules (either first wins, or last wins) to make the order of services significant, and reserve mappings for exceptions.

these are well-defined mappings without any ambiguity

service in the mappings is ambiguous. What if the two services you're trying to compose look like this?

  services:
  - apiVersion: com.example/v1alpha1
    kind:       AccountService
    name:       prod-account-service
  - apiVersion: v1
    kind:       Secret
    name:       prod-account-service

With name and key it's not immediately clear which is the name of the value from the existing service and which is the name of the value in the composed service.

I like the idea of defining a set of precedence rules, but I wonder if this imposes restrictions on the implementations to use libraries that guarantee the order of array elements?

I wonder if this imposes restrictions on the implementations to use libraries that guarantee the order of array elements?

Structurally, arrays have an order by definition. Sometimes the semantic use of an array ignores the ordering, but that's always defined at the semantic layer. A library that don't respect the order of an array sounds like a chaos generator.

Based on the recent discussion, I wrote this draft extension.

Composed Service

There are scenarios where an appropriate resource conforming to the Provisioned Service duck-type does not exist, and using a Direct Secret Reference with fixed values is not desirable. In some cases, it is possible to extract all the required binding values from multiple Provisioned Services (including GoV1Template and JSONPath mappings), and Secret resources. This extension allows composing multiple Provisioned Service-able resources and constructs a Provisioned Service that can be used in the Service Binding as a service.

The Composed Service describes composing multiple Provisioned Service-able resources to form a coherent Provisioned Service that can be used for Service Binding. It MUST be codified as a concrete resource type conforming to the Provisioned Service duck-type with version mapping.service.binding/v1alpha2 and kind ComposedService. For portability, the schema MUST comply to the exemplar CRD found [here][composed-service-schema].

A Composed Service resource MUST define a .spec.services which is an array of ObjectReference-like declarations to Provisioned Service-able resources. A Composed Service MAY define a .spec.mappings which is an array of Mapping. A Mapping object must define name, service, and key entries. The key of a Mapping MUST refer to a binding Secret key name of the given Provisioned Service-able object referred to in the service field.

Resource Type Schema

apiVersion: mapping.service.binding/v1alpha2
kind: ComposedService
metadata:
  name:             # string
spec:
  services:         # []Service (Povisioned Service-able resource)
  - apiVersion:     # string
    kind:           # string
    name:           # string
  mappings:         # []Mapping, optional
  - name:           # string
    service:        # string
    key:            # string
status:
  binding:          # LocalObjectReference
    name:           # string

Minimal Example Resource

apiVersion: mapping.service.binding/v1alpha2
kind: ComposedService
metadata:
  name: composed-service
spec:
  services:
  - apiVersion: com.example/v1alpha1
    kind:       AccountService
    name:       prod-account-service
  - apiVersion: v1
    kind:       Secret
    name:       credentials
  mappings:
  - name: username
    service: credentials
    key: username
  - name: password
    service: prod-account-service
    key: passwd
status:
  binding:
    name: prod-account-composed-service

Reconciler Implementation

A reconciler implementation MUST merge the Secret key-value pairs from all the listed Povisioned Service-able resources. If there is a conflict in key names, the last defined service key MUST be selected. If .spec.mappings is defined, the key-value pairs MUST override the previously constructed key-value pairs.

@arthurdm

If the intention is to aggregate bindings from provisioned services only, how about that kind reflect that, i.e. to name the resource something like ComposedProvisionedService or AggregatedProvisionedService? Also services field might be called provisionedServices instead? Speaking of that field, should not we define its minimal length, for example it must be o size two or more?

A few additional questions/concerns:

What is the default behavior if .spec.mappings is not defined?
What if we does not know what bindings are exposed by services, but we want to expose them all anyway?
What happens if a key cannot be found?
What if referred service is not provisioned?

Otherwise, the idea looks good.

There is however a usecase that is not covered by the proposed resources and I really like @arthurdm idea proposed earlier in the thread:

hi folks. One idea would be to include a spec.service.additionalResources element inside ServiceBinding, where it is an array of additional related resources. So for Strimzi as an example, perhaps KafkaCluster is the ref from spec.service, and KafkaUser is the [0] ref inside spec.service.additionalResources.

Namely, we see quite often operators (e.g. Crunchy Postgres) that provision write credentials (username/password) in a separate secret not linked to servcice CR anyhow. Ofcourse, we should help them to make migrate to the provision service approach, but until then, it would be really helpful to have a way to define several resources related to the service and create single binding secret eventually. For example:

service:
  apiVersion: "foo.bar/v1"
  kind: "Database"
  name: "db1"
  additionalResources:
     - apiVersion: v1
       lond: Secret
       name: db1-credentials

.service.additionalResources will be optional, so no changes required for existing bindings.

what's the proposed behavior for the additionalResources?

Composed Service

I have implemented something similar here:
https://github.com/kubepreset/composite-service
(see the test case for examples)

Composed Service

I have implemented something similar here: https://github.com/kubepreset/composite-service (see the test case for examples)

I made this project private as I couldn't continue to work on it.

If anyone looking at this, this is my proposed resource looks like:

apiVersion: extensions.servicebinding.io/v1beta1
kind: CompositeService
metadata:
  name: database-composite-service
spec:
  type: awesome-db
  provider: managed
  dumpConfigMaps:
  - name: cm1
    exclude:
    - key1
    - key2
  - name: cm2
    exclude:
    - key1
    - key2
  dumpSecrets:
  - name: secret1
    exclude:
    - key1
    - key2
  - name: secret2
    exclude:
    - key1
    - key2
  resources:
  - apiVersion: app1.example.org/v1alpha1
    id: back1
    kind: Database
    name: back1
  - apiVersion: v1
    id: firstSecret
    kind: Secret
    sourceID: back1
    targetPath: '{.spec.username}-{.status.credentials.name}'
  - apiVersion: v1
    id: secondSecret
    kind: Secret
    name: secret2
  paths:
  - name: username
    path: '{.spec.username}'
    resouceID: back1
  - ignore: true
    name: port
    path: '{.spec.port}'
    resouceID: back1
  - base64: true
    name: password
    path: '{.data.password}'
    resouceID: firstSecret
  - base64: true
    name: host
    path: '{.data.host}'
    resouceID: secondSecret
  templates:
  - name: url
    template: http://{{ .username }}:{{ .password }}@{{ .host }}:{{ .port }}
status:
  binding:
    name: database-composite-service-vjt64
  conditions:
  - lastTransitionTime: "2021-07-24T06:10:01Z"
    message: Secret resource created
    reason: SecretCreated
    status: "True"
    type: Ready
  observedGeneration: 1

Navigation Menu

Repository navigation

Multiple service resources in a `ServiceBinding` injected as a single logical service #120

Multiple Services

Multiservice

Resource Type Schema

Multiservice Example Resource

Service Binding with Multiservice Example

Composed Service

Resource Type Schema

Minimal Example Resource

Reconciler Implementation

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Multiple service resources in a ServiceBinding injected as a single logical service #120

Description

Activity

baijum commented on Oct 8, 2020

Multiple Services

sbose78 commented on Oct 8, 2020

scothis commented on Oct 8, 2020

baijum commented on Oct 9, 2020

Multiservice

Resource Type Schema

Multiservice Example Resource

Service Binding with Multiservice Example

arthurdm commented on Oct 9, 2020

baijum commented on Oct 9, 2020

scothis commented on Oct 9, 2020

baijum commented on Oct 13, 2020

navidsh commented on Oct 13, 2020

navidsh commented on Oct 13, 2020

sbose78 commented on Oct 13, 2020

arthurdm commented on Oct 28, 2020

baijum commented on Jul 7, 2021

scothis commented on Jul 7, 2021

arthurdm commented on Jul 7, 2021

scothis commented on Jul 7, 2021

baijum commented on Jul 8, 2021

Composed Service

Resource Type Schema

Minimal Example Resource

Reconciler Implementation

pedjak commented on Jul 8, 2021

scothis commented on Jul 8, 2021

baijum commented on Jul 18, 2021

baijum commented on Jul 25, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Multiple service resources in a `ServiceBinding` injected as a single logical service #120