Provision storage for SeekrFlow

SeekrFlow uses two kinds of storage. Object storage holds training datasets, model artifacts, and workflow outputs. Shared file storage holds model weights, which every GPU node mounts so that each copy of a model server reads the same files.

Amazon Web Services (AWS)
On-premises

Create object storage

Create an Amazon Simple Storage Service (Amazon S3) bucket. Start with 1 TB, and apply the security and backup settings your organization requires. SeekrFlow reaches the bucket through cloud identity, not stored credentials. See Set up cloud identity.

Create shared file storage

Create an Amazon Elastic File System (Amazon EFS) file system. See Creating EFS file systems. Use the settings in this table. For security and backups, follow your organization’s requirements.

Setting	What SeekrFlow needs
Performance	General Purpose performance mode, Elastic throughput, and the Standard storage class.
Virtual private cloud (VPC)	The VPC your cluster runs in.
Mount targets	One in every Availability Zone where GPU nodes run.
Security group	Inbound NFS traffic, TCP port 2049, from the nodes, such as from the cluster security group.

Install the Amazon EFS CSI driver

Pods mount the file system through the Amazon EFS Container Storage Interface (CSI) driver, an Amazon Elastic Kubernetes Service (Amazon EKS) add-on. The driver must be running before you install SeekrFlow. See Use elastic file system storage with Amazon EFS.

Setting	What SeekrFlow needs
Add-on	`aws-efs-csi-driver`.
AWS Identity and Access Management (IAM) role	A role with the `AmazonEFSCSIDriverPolicy` managed policy, associated through Amazon EKS Pod Identity with namespace `kube-system` and service account `efs-csi-controller-sa`. SeekrFlow’s storage class has the driver create EFS access points, which requires this role.
AWS GovCloud (US) with FIPS-enabled nodes	Add-on configuration `{"useFIPS":true}`.

To check that the driver is running, run:

kubectl get pods -n kube-system -l app=efs-csi-controller

The output lists two running pods.

Test the file system

The driver check doesn’t confirm that pods can mount the file system. To test the role, mount targets, and security group before you install, create a temporary volume and write to it. Replace <file-system-id> with your file system ID.

kubectl apply -f - <<'EOF'
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata: {name: efs-test}
provisioner: efs.csi.aws.com
parameters: {provisioningMode: efs-ap, fileSystemId: <file-system-id>, directoryPerms: "700"}
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata: {name: efs-test}
spec: {accessModes: [ReadWriteMany], storageClassName: efs-test, resources: {requests: {storage: 5Gi}}}
---
apiVersion: v1
kind: Pod
metadata: {name: efs-test}
spec:
  restartPolicy: Never
  containers: [{name: test, image: busybox, command: ["sh", "-c", "echo storage-ok > /data/t && cat /data/t"], volumeMounts: [{name: data, mountPath: /data}]}]
  volumes: [{name: data, persistentVolumeClaim: {claimName: efs-test}}]
EOF
kubectl wait --for=jsonpath='{.status.phase}'=Succeeded pod/efs-test --timeout=180s
kubectl logs efs-test

The output is storage-ok. Then delete the test resources:

kubectl delete pod/efs-test pvc/efs-test storageclass/efs-test

Record the values

Note these values. You will enter them when you create the values file.

Value	Used for	Example or format
Bucket name	`global.fileStorePath`	`seekrflow-data`, or `seekrflow-data/prod` with a path. Use the bare bucket name, without `s3://`.
Shared file storage type	`global.SharedModelCache.type`	`"efs"`
File system ID	`global.SharedModelCache.efs.fileSystemId`	`fs-0123456789abcdef0`

Create object storage

Provide an S3-compatible object store, such as MinIO, and create a bucket. Start with 1 TB, and apply the security and backup settings your organization requires. SeekrFlow reaches the store with access keys stored in the appliance-storage-creds secret.

Create shared file storage

Provide shared file storage that every GPU node can mount. Start with 1 TB. SeekrFlow supports two options:

Network File System (NFS) – An NFS server reachable from the cluster. Install the NFS CSI driver:
helm repo add csi-driver-nfs \ https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/charts helm install csi-driver-nfs csi-driver-nfs/csi-driver-nfs --namespace kube-system
To check that it’s running, run kubectl get pods -n kube-system -l app=csi-nfs-controller. If the cluster has no internet access, mirror the driver’s container images into your own registry before you install it.
Longhorn – A cluster with the Longhorn CSI driver already installed. SeekrFlow can use Longhorn’s existing storage class.

The CSI driver must be running before you install SeekrFlow.

Test the file system

If you use NFS, test that pods can mount it before you install. Create a temporary volume and write to it. Replace <nfs-server> and <exported-path> with your NFS server and exported path.

kubectl apply -f - <<'EOF'
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata: {name: nfs-test}
provisioner: nfs.csi.k8s.io
parameters: {server: <nfs-server>, share: <exported-path>}
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata: {name: nfs-test}
spec: {accessModes: [ReadWriteMany], storageClassName: nfs-test, resources: {requests: {storage: 5Gi}}}
---
apiVersion: v1
kind: Pod
metadata: {name: nfs-test}
spec:
  restartPolicy: Never
  containers: [{name: test, image: busybox, command: ["sh", "-c", "echo storage-ok > /data/t && cat /data/t"], volumeMounts: [{name: data, mountPath: /data}]}]
  volumes: [{name: data, persistentVolumeClaim: {claimName: nfs-test}}]
EOF
kubectl wait --for=jsonpath='{.status.phase}'=Succeeded pod/nfs-test --timeout=180s
kubectl logs nfs-test

The output is storage-ok. Then delete the test resources:

kubectl delete pod/nfs-test pvc/nfs-test storageclass/nfs-test

Record the values

Note these values. You will enter them when you create the values file and secrets.

Value	Used for	Example or format
Bucket name	`global.fileStorePath`	`seekrflow-data`, or `seekrflow-data/prod` with a path. Use the bare bucket name, without a scheme such as `s3://`.
Object store endpoint	`global.cloudProvider.on-prem.storageEndpoint`	`http://minio.seekrflow.svc:9000`
Object store access keys	`appliance-storage-creds` secret: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`	Your MinIO access key and secret key.
Shared file storage type	`global.SharedModelCache.type`	`"nfs"` or `"longhorn"`
NFS server and path	`global.SharedModelCache.nfs.server`, `global.SharedModelCache.nfs.path`	Your NFS server hostname or IP address, and the exported path.
Existing Longhorn storage class	`global.SharedModelCache.createStorageClass`, `global.SharedModelCache.storageClassName`	`false` and `longhorn`

SeekrFlow Self-Managed

Plan a SeekrFlow deployment

Provision SeekrFlow dependencies

Prepare to install SeekrFlow

Deploy SeekrFlow

Operate SeekrFlow

Reference

Create object storage

Create shared file storage

Install the Amazon EFS CSI driver

Test the file system

Record the values

Create object storage

Create shared file storage

Test the file system

Record the values

​Create object storage

​Create shared file storage

​Install the Amazon EFS CSI driver

​Test the file system

​Record the values

​Create object storage

​Create shared file storage

​Test the file system

​Record the values

Create object storage

Create shared file storage

Install the Amazon EFS CSI driver

Test the file system

Record the values

Create object storage

Create shared file storage

Test the file system

Record the values