diff --git a/docs/vector_search/configure-mongot.md b/docs/vector_search/configure-mongot.md new file mode 100644 index 000000000..c48de5820 --- /dev/null +++ b/docs/vector_search/configure-mongot.md @@ -0,0 +1,29 @@ +# Configure mongot + +After installing `mongot`, configure it to connect to the Percona Server for MongoDB replica set, store search indexes, and accept search requests from `mongod`. + +Vector Search requires configuration on both sides: + +- `mongot` must be able to connect to the replica set and synchronize data. +- `mongod` must know where to send search queries and search-index management requests. + +!!! note + The examples in this section use a single-node test environment with `mongod` listening on port **27017** and `mongot` listening on port **27028**. Replace the host names, ports, paths, and credentials with values that are applicable for your deployment. + +## Before you begin + +- PSMDB is running as an initialized replica set. +- Authentication is enabled. +- The `mongot` binary is installed. +- The user running `mongot` can write to the search data and log directories. +- The host running `mongot` can connect to the replica set. +- The required ports are available. + +## Procedure + + + + + + + diff --git a/docs/vector_search/create-search-index.md b/docs/vector_search/create-search-index.md new file mode 100644 index 000000000..c44486c7b --- /dev/null +++ b/docs/vector_search/create-search-index.md @@ -0,0 +1,167 @@ +# Create Index + +You can use the `createSearchIndex()` method to create a single Search or Vector Search index on a collection, or the `createSearchIndexes()` method to create multiple indexes simultaneously. + +## Create single Search Index + +Follow these steps to create a single search index: +{.power-number} + +1. Insert test data from mongosh: + + ```javascript + use test + db.docs.insertMany([ + { text: "MongoDB search is powerful" }, + { text: "Vector search is the future" }, + { text: "Full text search with mongot" } + ]) + + { + acknowledged: true, + insertedIds: { + '0': ObjectId('69ebae2599c54be2ea44ba89'), + '1': ObjectId('69ebae2599c54be2ea44ba8a'), + '2': ObjectId('69ebae2599c54be2ea44ba8b') + } + } + ``` + +2. Create a **single Search Index**: + + ```javascript + db.docs.createSearchIndex({ + name: "search_idx", + definition: { + mappings: { + dynamic: true + } + } + }) + + search_idx + ``` + +3. Check the status. + + ```javascript + db.docs.getSearchIndexes() + [ + { + id: '69ebae2a651bce4d10f57bdc', + name: 'search_idx', + status: 'READY', + queryable: true, + latestDefinitionVersion: { version: 0, createdAt: ISODate('2026-04-24T17:53:46.000Z') }, + latestDefinition: { mappings: { dynamic: true } }, + statusDetail: [ + { + hostname: '69eb5fc573906b6bfb8cefe7', + status: 'READY', + queryable: true, + mainIndex: { + status: 'READY', + queryable: true, + definitionVersion: { version: 0, createdAt: ISODate('2026-04-24T17:53:46.000Z') }, + definition: { mappings: { dynamic: true, fields: {} } } + } + } + ] + } + ] + ``` + +## Create multiple Search Indexes + +The following example creates two Search indexes: + +```javascript +db.docs.createSearchIndexes([ + { + name: "search_idx", + definition: { + mappings: { + dynamic: true + } + } + }, + { + name: "text_search_idx", + definition: { + mappings: { + dynamic: false, + fields: { + text: { + type: "string" + } + } + } + } + } +]) +[ 'search_idx', 'text_search_idx' ] +``` +This creates: + +- `search_idx`, which dynamically indexes supported fields. +- `text_search_idx`, which indexes only the text field. + +## Create Search and Vector Search Indexes together + +You can include Search and Vector Search index definitions in the same command. + +```javascript +db.docs.createSearchIndexes([ + { + name: "search_idx", + type: "search", + definition: { + mappings: { + dynamic: false, + fields: { + text: { + type: "string" + } + } + } + } + }, + { + name: "vector_idx", + type: "vectorSearch", + definition: { + fields: [ + { + type: "vector", + path: "embedding", + numDimensions: 3, + similarity: "cosine" + } + ] + } + } +]) +[ 'search_idx', 'vector_idx' ] +``` + +## Supported field types + +The following field types are supported: + +| Field type | Supported source data| Use| +| -----------| ----------------------| --| +| `boolean` | Boolean | Supports exact matching and filtering on `true` and `false` values. | +| `date` | BSON Date | Supports exact matches, range and proximity queries, sorting, and faceting. | +| `document` | Object or subdocument | Defines mappings for fields within a nested object. | +| `embeddedDocuments` | Array of objects | Indexes objects within an array so that each embedded object can be evaluated independently. | +| `number` | `int32`, `int64`, or `double` | Supports exact matches, numeric range and proximity queries, sorting, and faceting. | +| `objectId` | `ObjectId` | Supports matching, filtering, and range operations on `ObjectId` values. | +| `string` | String | Analyzes text for full-text operators such as `text`, `phrase`, `regex`, and `wildcard`. | + +For detailed inforamtion on supported feild types, see MongoDB documentation](https://www.mongodb.com/docs/search/index/define-field-mappings/#mongodb-search-field-types) + + +[Update a search index :material-arrow-right:](../update-search-index.md){.md-button} + +[Delete a search index :material-arrow-right:](../delete-search-index.md){.md-button} + diff --git a/docs/vector_search/delete-search-index.md b/docs/vector_search/delete-search-index.md new file mode 100644 index 000000000..c8fa60951 --- /dev/null +++ b/docs/vector_search/delete-search-index.md @@ -0,0 +1,14 @@ +# Delete Index + +Use the `dropSearchIndex()` method to delete a search or vector search index. The method works the same way for both index types: + +```javascript +db.products.dropSearchIndex("products_text_idx"); +``` + +Deleting an index is irreversible. To restore search functionality, create the index again with `createSearchIndex()`, and wait for the initial sync to +complete before running queries against it. + +!!! warning + `$search` and `$vectorSearch` queries that reference an index that does not exist return an empty result set rather than an error. If your application starts receiving empty search results after an index change, verify that the index exists and check its status with `getSearchIndexes()`. + diff --git a/docs/vector_search/install-mongot.md b/docs/vector_search/install-mongot.md new file mode 100644 index 000000000..95151ae1d --- /dev/null +++ b/docs/vector_search/install-mongot.md @@ -0,0 +1,492 @@ +# Install mongot + +Before deploying Vector Search for Percona Server for MongoDB, ensure that you have: + +- Percona Server for MongoDB 8.3 or later installed. +- Administrative privileges to install and configure `mongot`. +- A supported Linux operating system. + +Additional requirements depend on the installation method: + +- **Tarballs** - An initiated replica set with keyfile access control or sharded deployment. `mongosh` installed on the host. + +- **Docker** - Docker engine with Compose v2 installed on the host. + +For more information, see [Vector Search compatibility](vector-search-compatibility.md). + +## Procedure + +=== "Tarballs" + + ### Install `mongot` from tarballs + + Follow these steps to install `mongot` from a tarball: + {.power-number} + + 1. Download the `mongot` tarball. + + === "ARM64" + + Download the [ARM64 tarball :octicons-link-external-16:](https://downloads.mongodb.org/mongodb-search-community/0.53.0/mongot_community_0.53.0_linux_aarch64.tgz){:target="_blank"}. + + === "AMD64 (x86_64)" + + Download the [AMD64 tarball :octicons-link-external-16:](https://downloads.mongodb.org/mongodb-search-community/0.53.0/mongot_community_0.53.0_linux_x86_64.tgz){:target="_blank"}. + + 2. Extract the tarball. + + === "ARM64" + + ```sh + tar -zxvf mongot_community_0.53.0_linux_aarch64.tgz + ``` + + === "AMD64 (x86_64)" + + ```sh + tar -zxvf mongot_community_0.53.0_linux_x86_64.tgz + ``` + + The extracted archive contains the `mongot` binary, the sample configuration file, the `mongot` launcher script, and the MongoDB Search and Vector Search license files. + + 3. Configure `mongod` to communicate with `mongot`. + + Configure the following `mongod` startup parameters. + + | Parameter | Description | + |-----------|-------------| + | `searchIndexManagementHostAndPort` | Specifies the host and port of the `mongot` service used for search index management operations. | + | `mongotHost` | Specifies the host and port of the `mongot` service used to process search queries. This value must match `searchIndexManagementHostAndPort`. | + | `skipAuthenticationToSearchIndexManagementServer` | Enables or disables authentication between `mongod` and `mongot` for search index management operations. | + | `useGrpcForSearch` | Enables or disables gRPC communication between `mongod` and `mongot`. | + + ??? example "Example: `mongod` configuration" + + ```yaml + setParameter: + searchIndexManagementHostAndPort: localhost:27028 + mongotHost: localhost:27028 + skipAuthenticationToSearchIndexManagementServer: false + useGrpcForSearch: true + ``` + + These are startup parameters. Restart `mongod` for the changes to take effect. + + ```sh + sudo systemctl restart mongod + ``` + + 4. Create a user for the `mongot` process. + + `mongot` connects to your Percona Server for MongoDB deployment by using a user with the `searchCoordinator` role. + + 1. Connect to `mongosh` as an administrator. + + ```sh + mongosh --port 27017 -u -p + ``` + + 2. Switch to the `admin` database. + + ```javascript + use admin + ``` + + 3. Create the `mongot` user. + + Replace: + + - `` with the username for the `mongot` user. + - `` with the password that you save in the password file in the next step. + + Then run: + + ```javascript + db.createUser({ + user: "", + pwd: "", + roles: ["searchCoordinator"] + }) + ``` + 5. Create the password file. + + ```sh + sudo mkdir -p /etc/mongot + echo -n "" | sudo tee /etc/mongot/mongot.passwd > /dev/null + sudo chmod 600 /etc/mongot/mongot.passwd + sudo chown mongod:mongod /etc/mongot/mongot.passwd + ``` + + 6. Prepare the required directories. + + ```sh + sudo mkdir -p /var/lib/mongot /etc/mongot /opt/mongot + sudo chown -R mongod:mongod /var/lib/mongot /etc/mongot /opt/mongot + sudo chmod 750 /etc/mongot + ``` + + 7. Create the `mongot` configuration file. + + The tarball includes a sample configuration file named `config.default.yml`. Modify it as needed for your deployment. + + ??? example "Example: `config.default.yml`" + + ```yaml + syncSource: + replicaSet: + hostAndPort: localhost:27017 + username: + passwordFile: /etc/mongot/mongot.passwd + tls: false + + storage: + dataPath: /var/lib/mongot + + server: + grpc: + address: localhost:27028 + tls: + mode: disabled + + metrics: + enabled: true + address: localhost:9946 + + healthCheck: + address: localhost:8080 + + logging: + verbosity: INFO + ``` + + !!! note + + The user running `mongot` must have write access to the directory specified by `storage.dataPath`. + + For the complete list of configuration options, see the upstream [mongot configuration options :octicons-link-external-16:](https://www.mongodb.com/docs/manual/reference/configuration-options/#std-label-mongot-configuration-options){:target="_blank"} documentation. + + 8. Copy the extracted files to the installation directory. + + ```sh + sudo cp -a mongot-community/* /opt/mongot/ + sudo chown -R mongod:mongod /opt/mongot + ``` + + 9. Create the `systemd` service. + + Create the following file: + + ```text + /etc/systemd/system/mongot.service + ``` + + ```ini + [Unit] + Description=MongoDB Search (mongot) + Documentation=https://www.mongodb.com/docs/manual/reference/configuration-options/#std-label-mongot-configuration-options + After=network.target + + [Service] + User=mongod + Group=mongod + Type=simple + ExecStart=/opt/mongot/mongot --config /etc/mongot/config.yml + Restart=on-failure + RestartSec=5 + LimitNOFILE=64000 + TimeoutStartSec=30 + TimeoutStopSec=30 + SuccessExitStatus=143 + + [Install] + WantedBy=multi-user.target + ``` + + 10. Set the required file permissions and SELinux contexts. + + ```sh + sudo chown mongod:mongod /etc/mongot/config.yml + sudo chmod 600 /etc/mongot/config.yml + sudo restorecon -Rv /opt/mongot + ``` + + 11. Enable and start `mongot`. + + ```sh + sudo systemctl daemon-reload + sudo systemctl enable mongot + sudo systemctl start mongot + ``` + + 12. Verify that `mongot` is running. + + ```sh + curl localhost:8080/health + ``` + + Example output: + + ```json + {"status":"SERVING"} + ``` + +=== "Docker" + + ### Install Vector Search with Docker + + !!! info "Important" + - `mongot` synchronizes data from `mongod` and requires a PSMDB replica set. Standalone deployments are not supported. + + Follow these steps to install and configure `mongot` using Docker: + {.power-number} + + 1. Pull the `mongod` Docker image. + + ```bash + docker pull percona/percona-server-mongodb: + ``` + + 2. Pull the `mongot` Docker image. + + ```bash + docker pull percona/percona-server-mongodb-mongot: + ``` + + 3. Verify the downloaded images. + + ```bash + docker image ls | grep percona-server-mongodb + ``` + + 4. Create a Docker network. + + The `mongod` and `mongot` containers must run on the same Docker network so that they can communicate using their container names. Use the same `` value when you start both containers. + + ```bash + docker network create + ``` + Replace `` with a name for the network, for example `psmdb-search`. + + 5. Create the password file for the `mongot` user. + + `mongot` reads the password for its database user from a file. Create the file and restrict its permissions. The `-n` option prevents a trailing newline from being written to the file, which would otherwise become part of the password: + + + ```bash + echo -n "" > mongot-password.txt + chmod 400 mongot-password.txt + ``` + + Replace `` with a password of your choice. You use the same password when you create the `mongot` database user in step 9. + + 6. Create the `mongod` configuration file. + + Save the following configuration as `mongod.conf`. + + ```yaml + net: + port: 27017 + bindIpAll: true + + replication: + replSetName: rs0 + + setParameter: + searchIndexManagementHostAndPort: mongot:27028 + mongotHost: mongot:27028 + skipAuthenticationToSearchIndexManagementServer: false + useGrpcForSearch: true + searchTLSMode: disabled + ``` + + The `mongot:27028` address consists of the Docker container name and the gRPC port exposed by the `mongot` container. Specify the same value for both `searchIndexManagementHostAndPort` and `mongotHost`. + + 7. Start `mongod`. + + Replace: + + - `` with the path to the MongoDB data directory. + - `` with the path to the `mongod.conf` file. + - `` with the Docker network created in step 4. + + ```bash + docker run --rm \ + --name mongod \ + -v :/etc/mongod.conf:ro \ + -v :/data/db \ + -p 27017:27017 \ + --network \ + percona/percona-server-mongodb: \ + --config /etc/mongod.conf + ``` + + The container name `mongod` becomes the hostname used by `mongot` to connect to the database. If you change the container name, update the `syncSource.replicaSet.hostAndPort` value in the `mongot` configuration file. + + To view the server logs: + + ```bash + docker logs -f mongod + ``` + + 8. Initiate the replica set. + + Connect to the running container: + + ```bash + docker exec -it mongod mongosh --port 27017 + ``` + + Initialize the replica set: + + ```javascript + rs.initiate() + ``` + + Wait until the node becomes the primary replica set member before continuing. + + 9. Create the `mongot` user. + + `mongot` requires a database user with the `searchCoordinator` role. + + The `searchCoordinator` role grants `readAnyDatabase` privileges and write access to the internal `__mdb_internal_search` database, which `mongot` uses to store search index metadata. + + 1. Switch to the `admin` database. + + ```javascript + use admin + ``` + + 2. Create the user. + + Replace: + + - `` with the username for the `mongot` user. + - `` with the password stored in `mongot-password.txt`. + + ```javascript + db.createUser({ + user: "", + pwd: "", + roles: ["searchCoordinator"] + }) + ``` + + 10. Create the `mongot` configuration file. + + !!! info "Important" + Set `syncSource.replicaSet.scramAuth.username` to the user created in the previous step, and `syncSource.replicaSet.scramAuth.passwordFile` to the password file created in step 5. + + For more information about the available configuration options, see the upstream documentation for [mongot configuration options :octicons-link-external-16:](https://www.mongodb.com/docs/manual/reference/configuration-options/#std-label-mongot-configuration-options){:target="_blank"}. + + ??? example "Example: `mongot.conf`" + + ```yaml + syncSource: + replicaSet: + hostAndPort: "mongod:27017" + scramAuth: + username: "mongotUser" + passwordFile: "/passwordFile" + authSource: "admin" + tls: + enabled: false + replicationReader: + readPreference: "secondaryPreferred" + + storage: + dataPath: "/data/mongot" + + server: + grpc: + address: "mongot:27028" + tls: + mode: "disabled" + + metrics: + enabled: true + address: "mongot:9946" + + healthCheck: + address: "mongot:8080" + + logging: + verbosity: INFO + ``` + + Save the configuration file as `mongot.conf`. + + The `mongod` and `mongot` containers must run on the same Docker network. + + Ensure that: + + - `scramAuth.username` matches the user created in step 9. + - `passwordFile` points to the password file created in step 5. + + The `mongot` container reads its configuration from `/mongot-community/config.default.yml`, which is mounted in the next step. + + 11. Start the `mongot` container. + + Replace: + + - `` with the directory used to store search index data. + - `` with the path to the `mongot.conf` file. + - `` with the path to the password file. + - `` with the Docker network created in step 4. + + ```bash + docker run --rm \ + --name mongot \ + -v :/data/mongot \ + -v :/mongot-community/config.default.yml:ro \ + -v :/passwordFile:ro \ + --network \ + percona/percona-server-mongodb-mongot: + ``` + + To view the `mongot` logs: + + ```bash + docker logs -f mongot + ``` + + 12. Verify the health of the `mongot` process. + + Send a request to the readiness endpoint. + + ```bash + docker exec mongot -- curl localhost:8080/health + ``` + + If `mongot` starts successfully, the endpoint returns: + + ```text + SERVING + ``` + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/vector_search/overview-search.md b/docs/vector_search/overview-search.md new file mode 100644 index 000000000..819672737 --- /dev/null +++ b/docs/vector_search/overview-search.md @@ -0,0 +1,52 @@ +# Search overview + +Starting with Percona Server for MongoDB (PSMDB) 8.3, you can use **Full-text Search** and **Vector Search** with `mongot`, a dedicated search service that runs alongside `mongod`. + +You can create search indexes on your collections and use aggregation pipeline stages such as `$search`, `$searchMeta`, and `$vectorSearch` to retrieve relevant results. + +`mongot` is available via binary distributions (tarballs) and Docker images. + +## What is mongot? + +[mongot :octicons-link-external-16:](https://www.mongodb.com/docs/manual/tutorial/mongot-sizing/advanced-guidance/architecture/){:target="_blank"} is a companion process that builds and maintains search indexes for your MongoDB collections. While `mongod` stores and manages your application data, `mongot` creates optimized search indexes and processes search queries. + +The two services communicate internally during query execution: + +- `mongod` stores documents and handles database operations. +- `mongot` maintains search indexes. +- Search queries are processed by `mongot`, while `mongod` retrieves the matching documents and returns the results to the client. + +This architecture separates search workloads from core database operations while keeping the indexed data synchronized with the database. + + +## Search types + +Percona Server for MongoDB supports the following search types. Choose the type that matches your query patterns: + +| **Search type** | **Query stages**| **Index type** | **Use for**| +|------------------|-----------------|----------------|------------| +| Full-text search | `$search`, `$searchMeta` | `search`| Relevance-ranked text queries, autocomplete, faceting, and highlighting | +| Vector search | `$vectorSearch`| `vectorSearch` | Semantic similarity queries using machine learning embeddings| + + +### Full-text search + +Full-text Search lets you search text stored in one or more fields using relevance-based ranking. It supports capabilities such as: + +- Keyword and phrase searches +- Boolean operators +- Fuzzy matching +- Wildcard searches +- Field-specific searches +- Relevance scoring + +Use cases include: + +- Product catalog search +- Documentation search +- Blog and article search +- Customer support knowledge bases + + +For information about the search architecture and the eventual consistency model, see [Architecture](vector-search-architecture.md). + diff --git a/docs/vector_search/query-search.md b/docs/vector_search/query-search.md new file mode 100644 index 000000000..9aad518be --- /dev/null +++ b/docs/vector_search/query-search.md @@ -0,0 +1,104 @@ +# Query with $search + +The `$search` aggregation stage performs full-text search on fields covered by a search index. It returns documents ordered by relevance, with the most relevant document returned first. + +## Before you begin + +- `mongot` is running and connected to your Percona Search for MongoDB deployment. +- A search index exists for the collection. +- The index includes the fields you want to search. + +## Syntax for $search + +```javascript +db..aggregate([ + { + $search: { + index: "", + : { + + } + } + } +]) +``` + +The `$search `stage accepts the following primary fields: + +| Field| Required | Description| +|------| ---------|-------------| +| `index` | No | Name of the search index. If omitted, `$search` uses the index named `default`. | +| `` | Yes | Operator that defines the search criteria, such as `text`, `phrase`, `autocomplete`, or `compound`. | +| `highlight`| No | Returns matching terms in their original context. | +| `count` | No | Returns the number of matching documents.| +| `sort`| No| Specifies the order of the results.| +| `scoreDetails` | No | Returns details about how the relevance score was calculated. | + +## Search a text field + +The following example searches the `title` field in the `movies` collection for documents containing the term database: + +```javascript + { + $search: { + index: "movies_search", + text: { + query: "database", + path: "title" + } + } + } +]) +``` + +The `text` operator analyzes the query text and compares it with the indexed values in the specified field. + +To search more than one field, provide an array of field names: + +```javascript + { + $search: { + index: "movies_search", + text: { + query: "database", + path: ["title", "description"] + } + } + } +]) +``` + +## Return the relevance score + +Each result has a relevance score that indicates how closely it matches the query. Add a `$project` stage and the `searchScore` metadata expression to include the score in the output: + +```javascript +db.movies.aggregate([ + { + $search: { + index: "movies_search", + text: { + query: "database", + path: ["title", "description"] + } + } + }, + { + $project: { + _id: 0, + title: 1, + description: 1, + score: { + $meta: "searchScore" + } + } + }, + { + $limit: 10 + } +]) +``` + +This query returns the ten most relevant documents and includes the relevance score for each result. + +For the complete syntax and available options, see the [upstream MongoDB documentation :octicons-link-external-16:](https://www.mongodb.com/docs/search/query/aggregation-stages/search/){:target="_blank"} for the $search aggregation stage. \ No newline at end of file diff --git a/docs/vector_search/search-index-overview.md b/docs/vector_search/search-index-overview.md new file mode 100644 index 000000000..e5e4af57f --- /dev/null +++ b/docs/vector_search/search-index-overview.md @@ -0,0 +1,40 @@ +# Search Indexes + +A search index is a data structure that maps the terms in your documents to the documents that contain them. Instead of scanning every document in a collection, a search query looks up terms in the index and retrieves only the matching documents, along with metadata such as term positions and relevance data. + +- Search Indexes are maintained by the `mongot` process using Apache Lucene. +- Search queries use the aggregation pipeline stages `$search` and `$searchMeta`. +- When you create a search index, Percona Search for MongoDB transforms your data into a sequence of tokens or terms. + +## How Search Indexes work + +When you create a search index on a collection, `mongot` does the following: +{.power-number} + +1. Performs an initial sync, reading the collection data from mongod and building the Lucene index. +2. Opens a change stream on the collection to watch for inserts, updates, and deletes. +3. Applies those changes to the index continuously, keeping it in sync with the collection. + +## Types of Search Indexes + +Percona Search for MongoDB supports two index types: + +- Search indexes power full-text search with the `$search` and `$searchMeta` stages. They support text analyzers, relevance-based scoring, autocomplete, faceting, and highlighting. +- Vector search indexes power semantic and similarity search with the `$vectorSearch` stage. They index vector embeddings that you store in your documents and support [approximate nearest neighbor (ANN) :octicons-link-external-16:](https://www.mongodb.com/resources/basics/ann-search){:target="_blank"} search. + +For more information, see [how vector search works](). + +## Field mappings + +A search index definition specifies which fields to index and how to index them. You can choose between two mapping strategies: + +- [Dynamic mapping :octicons-link-external-16:](https://www.mongodb.com/docs/search/index/define-field-mappings/#dynamic-mappings){:target="_blank"} indexes all fields of supported types automatically, including fields added to documents later. Fields can be indexed based on the default set of types or by configuring a `typeSet`. + +- [Static mapping :octicons-link-external-16:](https://www.mongodb.com/docs/search/index/define-field-mappings/#static-mappings){:target="_blank"} indexes only the fields you explicitly define. To use static mappings to configure index options for only some fields, set `mappings.dynamic` to false and specify the field name, [data type :octicons-link-external-16:](https://www.mongodb.com/docs/search/index/define-field-mappings/#mongodb-search-field-types){:target="_blank"}, and other configuration options for each field that you want to index. You can specify the fields in any order. + + +[Create a search index :material-arrow-right:](../create-search-index.md){.md-button} + +[Update a search index :material-arrow-right:](../cupdate-search-index.md){.md-button} + +[Delete a search index :material-arrow-right:](../delete-search-index.md){.md-button} \ No newline at end of file diff --git a/docs/vector_search/update-search-index.md b/docs/vector_search/update-search-index.md new file mode 100644 index 000000000..27db0fe1d --- /dev/null +++ b/docs/vector_search/update-search-index.md @@ -0,0 +1,67 @@ +# Update Index + +You can use the `updateSearchIndex()` method to update a Search or Vector Search index. + +!!! warning + `updateSearchIndex()` replaces the index definition; it does not merge your changes into the existing one. Always submit the complete definition, including the fields you are not changing. Any field you omit is removed from the index. + +You cannot rename an index with this method. To rename an index, drop it and create a new one with the desired name. + +After you update an index, mongot rebuilds it in the background. The index continues to serve queries with the old definition until the rebuild completes. Use `getSearchIndexes()` to check the rebuild status. + +## Update single Search Index + +Suppose the `products_text_idx `index currently includes the `name` and `description` fields. The following operation updates the index to include the `category` field as well: + +```javascript +db.products.updateSearchIndex( + "products_text_idx", + { + mappings: { + dynamic: false, + fields: { + name: { + type: "string", + analyzer: "lucene.standard" + }, + description: { + type: "string", + analyzer: "lucene.standard" + }, + category: { + type: "string", + analyzer: "lucene.standard" + } + } + } + } +); +``` + +## Update a Vector Search index + +Suppose `products_vector_idx` currently indexes the `embedding` field. The following operation adds `category` as a filter field: + +```javascript +db.products.updateSearchIndex( + "products_vector_idx", + { + fields: [ + { + type: "vector", + path: "embedding", + numDimensions: 768, + similarity: "cosine" + }, + { + type: "filter", + path: "category" + } + ] + } +); +``` + +[Delete a search index :material-arrow-right:](../delete-search-index.md){.md-button} + + diff --git a/docs/vector_search/vector-search-architecture.md b/docs/vector_search/vector-search-architecture.md new file mode 100644 index 000000000..35438edaa --- /dev/null +++ b/docs/vector_search/vector-search-architecture.md @@ -0,0 +1,30 @@ +# Architecture + +`mongod` and `mongot` work together to provide full-text and vector search capabilities. While `mongod` continues to store and manage your application data, `mongot` is responsible for building search indexes and executing search queries. The two processes run alongside each other and communicate internally to keep search indexes synchronized with the underlying data. + +## Components + +| **Component** | **Description** | +|-----------|-------------| +| `Application` | Sends vector search requests using the `$vectorSearch` aggregation stage. | +| `mongod` | Stores application data, manages vector search indexes, forwards vector search requests to `mongot`, retrieves matching documents, and returns the final results. | +| `mongos` | Routes vector search requests to the appropriate shard in sharded deployments. | +| `mongot` | Builds and maintains vector indexes, synchronizes them with data stored in `mongod`, performs nearest-neighbor searches, and returns matching document identifiers with similarity scores. | +| Vector indexes | Specialized indexes maintained by `mongot` to efficiently perform semantic similarity searches over vector embeddings. | + +## How vector search works + +The following steps describe how a vector search request is processed: +{.power-number} + +1. The application converts the search query into a vector embedding using an embedding model. +2. The application submits the vector search request to `mongod` or `mongos` using the `$vectorSearch` aggregation stage. +3. `mongod` forwards the vector search portion of the request to `mongot`. +4. `mongot` searches the vector index for the nearest matching embeddings. The vector indexes are continuously synchronized with the data stored in `mongod`, ensuring that search results reflect the latest changes. +5. `mongot` returns the matching document identifiers and similarity scores to `mongod`. +6. `mongod` retrieves the corresponding documents from the database, applies any remaining aggregation pipeline stages, and returns the final results to the application. + +## Data synchronization +`mongot` does not store the primary copy of your data. Instead, it maintains vector indexes that are synchronized with the collections stored in `mongod`. Whenever documents are inserted, updated, or deleted, the corresponding vector indexes are updated automatically. This synchronization ensures that vector search queries operate on current data without requiring manual index maintenance. + + diff --git a/docs/vector_search/vector-search-compatibility.md b/docs/vector_search/vector-search-compatibility.md new file mode 100644 index 000000000..a8246a461 --- /dev/null +++ b/docs/vector_search/vector-search-compatibility.md @@ -0,0 +1,29 @@ +# System requirements and compatibility + +Before deploying Vector Search in Percona Server for MongoDB (PSMDB), ensure that your environment meets the software, hardware, and deployment requirements for `mongot`. + +!!! info "Important" + The requirements listed below provide the minimum supported configuration. Production environments that handle large collections or high search traffic typically require additional CPU, memory, and storage resources. + +## Version compatibility + +`mongot` requires Percona Server for MongoDB (PSMDB) version 8.3 or later. Ensure that the `mongot` build you deploy is compatible with the PSMDB version in your environment. + +## Operating system support + +`mongot` is supported on Linux systems running one of the following processor architectures: + +- x86_64 (64-bit x86) +- ARM64 (AArch64) + +Verify that your operating system and architecture are supported by the current release of Percona Server for MongoDB before deployment. + +## Memory requirements + +A minimum of 4 GB of RAM is required to run `mongot`. + +Search indexes are maintained in memory whenever possible to improve query performance. Deployments with large collections, high-dimensional vectors, or multiple search indexes may require significantly more memory than the minimum requirement. + +## Storage recommendations + +Although `mongot` can run on any supported storage device, solid-state drives (SSDs) are strongly recommended for production deployments. diff --git a/mkdocs-base.yml b/mkdocs-base.yml index 460093777..6a9bcb3ec 100644 --- a/mkdocs-base.yml +++ b/mkdocs-base.yml @@ -200,6 +200,21 @@ nav: - Feature comparison with MongoDB: comparison.md - Storage: - "Percona Memory Engine": "inmemory.md" + - Search: + - vector_search/overview-search.md + - vector_search/vector-search-architecture.md + - vector_search/vector-search-compatibility.md + - vector_search/install-mongot.md + - Search indexes: + - "Search index overview": vector_search/search-index-overview.md + - "Create a search index": vector_search/create-search-index.md + - "Update search index": vector_search/update-search-index.md + - "Delete search index": vector_search/delete-search-index.md + - Run search queries: + - Query with $search: vector_search/query-search.md + - Retrieve metadata with $searchMeta: vector_search/query-searchmeta.md + - Operators: vector_search/operators.md # phase 2 candidate + - Scoring: vector_search/scoring.md - Backup: - "Hot Backup": "hot-backup.md" - backup-cursor.md