Provision OpenSearch for SeekrFlow

OpenSearch stores the search indexes SeekrFlow uses for file search and retrieval-augmented generation. On Amazon Web Services (AWS), you provide the instance. On-premises, the chart deploys it in the cluster.

Amazon Web Services (AWS)
On-premises

Create the instance

Create an Amazon OpenSearch Service domain. See Creating and managing Amazon OpenSearch Service domains. Use the settings in this table. For security and backups, follow your organization’s requirements.

Setting	What SeekrFlow needs
Engine version	OpenSearch 2.17 or later.
Storage	Start with 500 GB.
Network	Virtual private cloud (VPC) access, in the VPC your cluster runs in.
Security group	Inbound traffic on port 443 from the cluster, such as from the cluster security group.
Access	SeekrFlow’s AWS Identity and Access Management (IAM) role, with `es:ESHttp` on the domain. You grant it in the role’s permission policy in Set up cloud identity. The principal that registers the search pipeline also needs `es:ESHttp`.

You can also run OpenSearch yourself, in your cluster with the official Helm chart or on your own compute.

Check the instance

Amazon OpenSearch Service accepts only requests signed by a principal its access policy allows. From a machine that can reach the domain, such as a bastion host in your VPC, use a signing client such as awscurl:

awscurl --service es --region <region> "https://<opensearch-host>/_cluster/health?pretty"

A status of green or yellow is fine for deployment.

Register the hybrid search pipeline

Hybrid search needs a search pipeline named hybrid_pipeline. Register it once. The request is safe to repeat.

awscurl --service es --region <region> -X PUT "https://<opensearch-host>/_search/pipeline/hybrid_pipeline" \
  -H "Content-Type: application/json" \
  -d '{"phase_results_processors":[{"normalization-processor":{"normalization":{"technique":"min_max"},"combination":{"technique":"arithmetic_mean","parameters":{"weights":[0.25,0.75]}}}}]}'

Record the values

Note these values. You will enter them when you create the values file.

Value	Used for	Example or format
Hostname	`global.defaults.opensearch.host`	Your domain endpoint.
Port	`global.defaults.opensearch.port`	`443` for Amazon OpenSearch Service, `9200` for OpenSearch you run yourself.
Scheme	`global.defaults.opensearch.scheme`	`"https://"`
Whether the chart deploys OpenSearch	`opensearch.enabled`	`false`. `sfe bootstrap` sets it to `true`, which you correct in Create the values file.

Set up the in-cluster instance

The chart deploys OpenSearch in your cluster and registers the hybrid search pipeline for you. You set it up through the values file and one secret.The chart creates three OpenSearch pods, each with an 8 GiB persistent volume. Your cluster needs a default storage class that provisions volumes automatically. Size the volumes for the amount of content you plan to ingest.OpenSearch takes its admin password from the opensearch-credentials secret the first time it starts. The chart doesn’t generate this secret. Generate a password, or let sfe secrets create --auto-generate generate it when you create SeekrFlow’s secrets.

Keep this password for the life of the deployment. OpenSearch sets its admin password only once, at first start. If the secret changes later, the password no longer matches and every request to OpenSearch fails.

Record the values

Note these values. You will enter them when you create the values file and secrets.

Value	Used for	Example or format
Whether the chart deploys OpenSearch	`opensearch.enabled`	`true`
Admin user	`opensearch-credentials` secret: `OPENSEARCH_USER`	`admin`. OpenSearch requires this name.
Admin password	`opensearch-credentials` secret: `OPENSEARCH_PASS`	The password you generated, or the one `sfe secrets create --auto-generate` creates.
Scheme	`global.defaults.opensearch.scheme`	`"https://"`

SeekrFlow Self-Managed

Plan a SeekrFlow deployment

Provision SeekrFlow dependencies

Prepare to install SeekrFlow

Deploy SeekrFlow

Operate SeekrFlow

Reference

Provision OpenSearch for SeekrFlow

Create the instance

Check the instance

Register the hybrid search pipeline

Record the values

Set up the in-cluster instance

Record the values

​Create the instance

​Check the instance

​Register the hybrid search pipeline

​Record the values

​Set up the in-cluster instance

​Record the values

Create the instance

Check the instance

Register the hybrid search pipeline

Record the values

Set up the in-cluster instance

Record the values