Skip to content

Commit 5d88a77

Browse files
azaddhirajkumarazaddhirajkumar
authored
Update docs for Couchbase (#316)
* Updated docs for Couchbase * addressed comments and some minor changes
1 parent e01fdad commit 5d88a77

1 file changed

Lines changed: 160 additions & 49 deletions

File tree

‎semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/couchbase-connector.md‎

Lines changed: 160 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ zone_pivot_groups: programming-languages
55
author: azaddhirajkumar
66
ms.topic: conceptual
77
ms.author: westey
8-
ms.date: 01/14/2025
8+
ms.date: 11/03/2025
99
ms.service: semantic-kernel
1010
---
1111

@@ -21,20 +21,19 @@ ms.service: semantic-kernel
2121
The Couchbase Vector Store connector can be used to access and manage data in Couchbase. The connector has the
2222
following characteristics.
2323

24-
| Feature Area | Support |
25-
|---------------------------------------|-------------------------------------------------------------------------------------------------------------------|
26-
| Collection maps to | Couchbase collection |
27-
| Supported key property types | string |
28-
| Supported data property types | All types that are supported by System.Text.Json (either built-in or by using a custom converter) |
29-
| Supported vector property types | <ul><li>ReadOnlyMemory\<float\></li></ul> |
30-
| Supported index types | N/A |
31-
| Supported distance functions | <ul><li>CosineSimilarity</li><li>DotProductSimilarity</li><li>EuclideanDistance</li></ul> |
32-
| Supported filter clauses | <ul><li>AnyTagEqualTo</li><li>EqualTo</li></ul> |
33-
| Supports multiple vectors in a record | Yes |
34-
| IsFilterable supported? | No |
35-
| IsFullTextSearchable supported? | No |
36-
| StoragePropertyName supported? | No, use `JsonSerializerOptions` and `JsonPropertyNameAttribute` instead. [See here for more info.](#data-mapping) |
37-
| HybridSearch supported? | No |
24+
| Feature Area | Support |
25+
| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
26+
| Collection maps to | Couchbase collection + index |
27+
| Supported key property types | <ul><li>`string`</li></ul> |
28+
| Supported data property types | All types that are supported by `System.Text.Json` (either built-in or by using a custom converter) |
29+
| Supported vector property types | <ul><li>`ReadOnlyMemory<float>`</li><li>`Embedding<float>`</li><li>`float[]`</li></ul> |
30+
| Supported distance functions | <ul><li>CosineSimilarity</li><li>DotProductSimilarity</li><li>EuclideanDistance</li></ul> |
31+
| Supported filter clauses | <ul><li>`AnyTagEqualTo`</li><li>`EqualTo`</li></ul> |
32+
| Supports multiple vectors in a record | Yes |
33+
| IsIndexed supported? | Yes |
34+
| IsFullTextIndexed supported? | Yes |
35+
| StoragePropertyName supported? | No, use `JsonSerializerOptions` and `JsonPropertyNameAttribute` instead. [See here for more info.](#data-mapping) |
36+
| HybridSearch supported? | Yes |
3837

3938
## Getting Started
4039

@@ -44,14 +43,15 @@ Add the Couchbase Vector Store connector NuGet package to your project.
4443
dotnet add package CouchbaseConnector.SemanticKernel --prerelease
4544
```
4645

47-
You can add the vector store to the dependency injection container available on the `KernelBuilder` or to
48-
the `IServiceCollection` dependency injection container using extension methods provided by Semantic Kernel.
46+
You can add the vector store to the dependency injection container available on the `KernelBuilder` or to the `IServiceCollection` dependency injection container using extension methods provided by Semantic Kernel.
4947

5048
```csharp
5149
using Microsoft.SemanticKernel;
50+
using Couchbase.SemanticKernel;
5251

5352
// Using Kernel Builder.
54-
var kernelBuilder = Kernel.CreateBuilder()
53+
var kernelBuilder = Kernel
54+
.CreateBuilder()
5555
.AddCouchbaseVectorStore(
5656
connectionString: "couchbases://your-cluster-address",
5757
username: "username",
@@ -61,7 +61,7 @@ var kernelBuilder = Kernel.CreateBuilder()
6161
```
6262

6363
```csharp
64-
using Microsoft.Extensions.DependencyInjection;
64+
using Couchbase.SemanticKernel;
6565

6666
// Using IServiceCollection with ASP.NET Core.
6767
var builder = WebApplication.CreateBuilder(args);
@@ -73,6 +73,52 @@ builder.Services.AddCouchbaseVectorStore(
7373
scopeName: "scope-name");
7474
```
7575

76+
**Configuring Index Type**
77+
78+
The vector store defaults to using Hyperscale indexes. You can specify a different index type by passing `CouchbaseVectorStoreOptions`:
79+
80+
```csharp
81+
using Couchbase.SemanticKernel;
82+
83+
var builder = WebApplication.CreateBuilder(args);
84+
85+
// Option 1: Use Hyperscale index
86+
builder.Services.AddCouchbaseVectorStore(
87+
connectionString: "couchbases://your-cluster-address",
88+
username: "username",
89+
password: "password",
90+
bucketName: "bucket-name",
91+
scopeName: "scope-name",
92+
options: new CouchbaseVectorStoreOptions
93+
{
94+
IndexType = CouchbaseIndexType.Hyperscale
95+
});
96+
97+
// Option 2: Use Composite index
98+
builder.Services.AddCouchbaseVectorStore(
99+
connectionString: "couchbases://your-cluster-address",
100+
username: "username",
101+
password: "password",
102+
bucketName: "bucket-name",
103+
scopeName: "scope-name",
104+
options: new CouchbaseVectorStoreOptions
105+
{
106+
IndexType = CouchbaseIndexType.Composite
107+
});
108+
109+
// Option 3: Use Search vector index
110+
builder.Services.AddCouchbaseVectorStore(
111+
connectionString: "couchbases://your-cluster-address",
112+
username: "username",
113+
password: "password",
114+
bucketName: "bucket-name",
115+
scopeName: "scope-name",
116+
options: new CouchbaseVectorStoreOptions
117+
{
118+
IndexType = CouchbaseIndexType.Search
119+
});
120+
```
121+
76122
Extension methods that take no parameters are also provided. These require an instance of the `IScope` class to be
77123
separately registered with the dependency injection container.
78124

@@ -109,7 +155,6 @@ kernelBuilder.Services.AddCouchbaseVectorStore();
109155

110156
```csharp
111157
using Microsoft.Extensions.DependencyInjection;
112-
using Microsoft.SemanticKernel;
113158
using Couchbase.KeyValue;
114159
using Couchbase;
115160

@@ -154,7 +199,6 @@ var clusterOptions = new ClusterOptions
154199
};
155200

156201
var cluster = await Cluster.ConnectAsync(clusterOptions);
157-
158202
var bucket = await cluster.BucketAsync("bucket-name");
159203
var scope = bucket.Scope("scope-name");
160204

@@ -163,80 +207,147 @@ var vectorStore = new CouchbaseVectorStore(scope);
163207

164208
It is possible to construct a direct reference to a named collection.
165209

210+
### Using Query Collection (Hyperscale or Composite Index)
211+
212+
For high-performance vector search with Hyperscale indexes:
213+
166214
```csharp
215+
using Couchbase.SemanticKernel;
167216
using Couchbase;
168217
using Couchbase.KeyValue;
169-
using Couchbase.SemanticKernel;
170218

171-
var clusterOptions = new ClusterOptions
172-
{
173-
ConnectionString = "couchbases://your-cluster-address",
174-
UserName = "username",
175-
Password = "password"
176-
};
219+
var cluster = await Cluster.ConnectAsync(clusterOptions);
220+
var bucket = await cluster.BucketAsync("bucket-name");
221+
var scope = bucket.Scope("scope-name");
222+
223+
// Using Hyperscale index (default)
224+
var collection = new CouchbaseQueryCollection<string, Hotel>(
225+
scope,
226+
"skhotels",
227+
indexType: CouchbaseIndexType.Hyperscale);
228+
229+
// Or using Composite index
230+
var collectionComposite = new CouchbaseQueryCollection<string, Hotel>(
231+
scope,
232+
"skhotels",
233+
indexType: CouchbaseIndexType.Composite);
234+
```
235+
236+
### Using Search Collection (Seach Vector Index)
237+
238+
For hybrid search scenarios combining full-text search:
239+
240+
```csharp
241+
using Couchbase.SemanticKernel;
242+
using Couchbase;
243+
using Couchbase.KeyValue;
177244

178245
var cluster = await Cluster.ConnectAsync(clusterOptions);
179246
var bucket = await cluster.BucketAsync("bucket-name");
180247
var scope = bucket.Scope("scope-name");
181248

182-
var collection = new CouchbaseFtsVectorStoreRecordCollection<Hotel>(
249+
var collection = new CouchbaseSearchCollection<string, Hotel>(
183250
scope,
184-
"hotelCollection");
251+
"skhotels");
185252
```
253+
254+
### Index Type Comparison
255+
256+
Couchbase offers three types of indexes for vector search:
257+
258+
**Hyperscale Vector Indexes**
259+
260+
- Best for pure vector searches - content discovery, recommendations, semantic search
261+
- High performance with low memory footprint - designed to scale to billions of vectors
262+
- Optimized for concurrent operations - supports simultaneous searches and inserts
263+
- **Use when:** You primarily perform vector-only queries without complex scalar filtering
264+
- **Ideal for:** Large-scale semantic search, recommendation systems, content discovery
265+
- **Requires:** Couchbase Server 8.0+ or Capella
266+
267+
**Composite Vector Indexes**
268+
269+
- Best for filtered vector searches - combines vector search with scalar value filtering
270+
- Efficient pre-filtering - scalar attributes reduce the vector comparison scope
271+
- **Use when:** Your queries combine vector similarity with scalar filters that eliminate large portions of data
272+
- **Ideal for:** Compliance-based filtering, user-specific searches, time-bounded queries
273+
- **Requires:** Couchbase Server 8.0+ or Capella
274+
275+
**Search Vector Indexes**
276+
277+
- Best for hybrid searches combining full-text search with vector similarity
278+
- Allows semantic search alongside traditional keyword matching
279+
- Supports geospatial searches in addition to vector and text
280+
- **Use when:** You need to combine traditional keyword search with vector similarity search in the same query
281+
- **Ideal for:** E-commerce product search, travel recommendations, content discovery with multiple search criteria
282+
- **Requires:** Couchbase Server 7.6+ or Capella
283+
284+
**Choosing the Right Index Type:**
285+
286+
- Start with **Hyperscale Index** for pure vector searches and large datasets (scales to billions)
287+
- Choose **Composite Index** when scalar filters significantly reduce your search space (works well for tens of millions to billions of vectors)
288+
- Use **Search Vector Index** for hybrid search combining text and vectors
289+
290+
291+
[Detailed comparison of vector index types](https://docs.couchbase.com/server/current/vector-index/use-vector-indexes.html)
186292
## Data mapping
187293

188-
The Couchbase connector uses `System.Text.Json.JsonSerializer` for data mapping. Properties in the data model are serialized into a JSON object and mapped to Couchbase storage.
294+
The Couchbase connector will use `System.Text.Json.JsonSerializer` to do mapping. Properties in the data model are serialized into a JSON object and stored as the document value in Couchbase.
295+
296+
Usage of the `JsonPropertyNameAttribute` is supported if a different storage name to the data model property name is required. It is also possible to use a custom `JsonSerializerOptions` instance with a customized property naming policy.
189297

190-
Use the `JsonPropertyName` attribute to map a property to a different name in Couchbase storage. Alternatively, you can configure `JsonSerializerOptions` for advanced customization.
191298
```csharp
192299
using Couchbase.SemanticKernel;
193300
using Couchbase.KeyValue;
194301
using System.Text.Json;
195302

196303
var jsonSerializerOptions = new JsonSerializerOptions
197304
{
198-
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
305+
PropertyNamingPolicy = JsonNamingPolicy.SnakeCaseUpper
199306
};
200307

201-
var options = new CouchbaseFtsVectorStoreRecordCollectionOptions<Hotel>
308+
var options = new CouchbaseQueryCollectionOptions
202309
{
203310
JsonSerializerOptions = jsonSerializerOptions
204311
};
205312

206-
var collection = new CouchbaseFtsVectorStoreRecordCollection<Hotel>(scope, "hotels", options);
313+
var collection = new CouchbaseQueryCollection<string, Hotel>(scope, "skhotelsjson", options);
207314
```
208-
Using the above custom `JsonSerializerOptions` which is using `CamelCase`, the following data model will be mapped to the below json.
315+
316+
Since a naming policy of snake case upper was chosen, here is an example of how this data type will be stored in Couchbase. Also note the use of `JsonPropertyNameAttribute` on the `Description` property to further customize the storage naming.
209317

210318
```csharp
211319
using System.Text.Json.Serialization;
212320
using Microsoft.Extensions.VectorData;
213321

214322
public class Hotel
215323
{
216-
[JsonPropertyName("hotelId")]
217-
[VectorStoreRecordKey]
324+
[VectorStoreKey]
218325
public string HotelId { get; set; }
219326

220-
[JsonPropertyName("hotelName")]
221-
[VectorStoreRecordData]
327+
[VectorStoreData(IsIndexed = true)]
222328
public string HotelName { get; set; }
223329

224-
[JsonPropertyName("description")]
225-
[VectorStoreRecordData]
330+
[JsonPropertyName("HOTEL_DESCRIPTION")]
331+
[VectorStoreData(IsFullTextIndexed = true)]
226332
public string Description { get; set; }
227333

228-
[JsonPropertyName("descriptionEmbedding")]
229-
[VectorStoreRecordVector(Dimensions: 4, DistanceFunction.DotProductSimilarity)]
230-
public ReadOnlyMemory<float> DescriptionEmbedding { get; set; }
334+
[VectorStoreVector(Dimensions: 4, DistanceFunction.CosineSimilarity)]
335+
public ReadOnlyMemory<float>? DescriptionEmbedding { get; set; }
231336
}
232337
```
233338

234339
```json
235340
{
236-
"hotelId": "h1",
237-
"hotelName": "Hotel Happy",
238-
"description": "A place where everyone can be happy",
239-
"descriptionEmbedding": [0.9, 0.1, 0.1, 0.1]
341+
"_id" : "h1",
342+
"HOTEL_ID" : "h1",
343+
"HOTEL_NAME" : "Hotel Happy",
344+
"HOTEL_DESCRIPTION" : "A place where everyone can be happy.",
345+
"DESCRIPTION_EMBEDDING" : [
346+
0.9,
347+
0.1,
348+
0.1,
349+
0.1
350+
]
240351
}
241352
```
242353

0 commit comments

Comments
 (0)