> ## Documentation Index > Fetch the complete documentation index at: https://docs.seekr.com/llms.txt > Use this file to discover all available pages before exploring further. # Provision the Kubernetes cluster for SeekrFlow > Create the Kubernetes cluster SeekrFlow runs on, add its nodes, and prepare the GPU nodes. SeekrFlow runs on a Kubernetes cluster you create and manage. Create it with your provider's standard process, and make sure it meets the requirements in [Review SeekrFlow requirements](/self-managed/flow/plan/requirements). ## Create the cluster Create an Amazon Elastic Kubernetes Service (Amazon EKS) cluster with the AWS Management Console, `eksctl`, or the AWS Command Line Interface (AWS CLI). See [Create an Amazon EKS cluster](https://docs.aws.amazon.com/eks/latest/userguide/create-cluster.html). Use the settings in this table, and leave the rest at their defaults. | Setting | What SeekrFlow needs | | - | - | | Name and AWS Region | Any name. Any commercial Region, or `us-gov-west-1` or `us-gov-east-1` for AWS GovCloud (US). You enter both in the values file later. | | EKS Auto Mode | Off. SeekrFlow runs on managed node groups. Auto Mode replaces and consolidates nodes on its own schedule, which can interrupt model serving and fine-tuning and release GPU instances that are hard to replace. | | Kubernetes version | One of the three most recent Kubernetes minor versions. Versions in Amazon EKS extended support aren't supported. | | Cluster IAM role | An AWS Identity and Access Management (IAM) role that Amazon EKS can assume, with the `AmazonEKSClusterPolicy` managed policy attached. If you already have one, such as `AmazonEKSClusterRole`, use it. Otherwise, create one. See [Amazon EKS cluster IAM role](https://docs.aws.amazon.com/eks/latest/userguide/cluster-iam-role.html). | | Virtual private cloud (VPC) and subnets | Any layout your organization uses. Amazon EKS requires subnets in at least two Availability Zones. The nodes must reach the container registry, object storage, PostgreSQL, OpenSearch, and any cloud model services you use. | | Endpoint access | Any. If you select **Private**, run `kubectl` and `sfe` from a machine inside your network, such as a bastion host. | | Add-ons | The Amazon EKS Pod Identity Agent, which SeekrFlow uses to reach your cloud services. See [Set up the Amazon EKS Pod Identity Agent](https://docs.aws.amazon.com/eks/latest/userguide/pod-id-agent-setup.html). Also keep Amazon VPC CNI, CoreDNS, and kube-proxy, which the cluster needs, and Metrics Server, which SeekrFlow needs to scale its services. SeekrFlow doesn't need the other add-ons the console preselects, such as External DNS. Remove any you don't plan to configure. | Use any conformant Kubernetes cluster, meaning one that meets the requirements of the [Kubernetes conformance tests](https://github.com/cncf/k8s-conformance). The cluster must also run [Metrics Server](https://github.com/kubernetes-sigs/metrics-server), which SeekrFlow needs to scale its services. ## Add worker and GPU nodes Add two sets of nodes to the cluster: * **Worker nodes** – Run the platform services. Provision the virtual CPUs (vCPUs) and memory from [Size your deployment](/self-managed/flow/plan/size-deployment). Enabling the Valkey caches takes at least three worker nodes. * **GPU nodes** – Run model servers, fine-tuning, and explainability workloads. Use an instance type or server that meets the [GPU requirements](/self-managed/flow/plan/requirements#gpu-nodes), with the GPU count from [Size your deployment](/self-managed/flow/plan/size-deployment). Each GPU node needs enough GPUs and memory for the largest model it hosts, because a model can't span nodes. Both must be nodes you manage. Serverless compute, such as AWS Fargate, isn't supported. Create two managed node groups, one for worker nodes and one for GPU nodes. See [Create a managed node group for your cluster](https://docs.aws.amazon.com/eks/latest/userguide/create-managed-node-group.html). Use the settings in this table, and leave the rest at their defaults. | Setting | Worker node group | GPU node group | | - | - | - | | Node IAM role | A role with the `AmazonEKSWorkerNodePolicy`, `AmazonEC2ContainerRegistryReadOnly`, and `AmazonEKS_CNI_Policy` managed policies attached. If you already have one, use it. Otherwise, create one. See [Amazon EKS node IAM role](https://docs.aws.amazon.com/eks/latest/userguide/create-node-role.html). | The same role. | | Amazon Machine Image (AMI) type | A standard x86\_64 AMI type, such as Amazon Linux 2023 (x86\_64) Standard. | An NVIDIA AMI type, such as Amazon Linux 2023 NVIDIA or Bottlerocket NVIDIA. On Kubernetes 1.34 and later, these include driver 580 or newer. Earlier Kubernetes versions get an older driver. If your compliance requirements call for FIPS mode on the nodes, use an NVIDIA AMI with FIPS mode enabled, such as Amazon Linux 2023 NVIDIA. See [Use EKS-optimized accelerated AMIs for GPU instances](https://docs.aws.amazon.com/eks/latest/userguide/ml-eks-optimized-ami.html). | | Instance type | Enough instances for the vCPUs and memory in your sizing, and at least three if you enable the Valkey caches. | An instance family with a supported GPU, such as G5 (A10G), G6 (L4), G6e (L40S), P4d (A100), or P5 (H100). Check that your Region offers it and that your quota covers it. | | Root volume | At least 100 GiB. | At least 200 GiB, plus temporary space for fine-tuning jobs. See [Size your deployment](/self-managed/flow/plan/size-deployment). | | Kubernetes taints | None. | Optional. To keep platform services off GPU nodes, add the [taint](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) `nvidia.com/gpu=true:NoSchedule`. Model pods on Amazon EKS accept this taint automatically. | | Subnets | The subnets you chose for the cluster. | The same subnets. | Join the worker and GPU servers to the cluster as nodes, using your Kubernetes distribution's process. On each GPU node, install an NVIDIA driver that meets the [driver requirement](/self-managed/flow/plan/requirements#gpu-nodes). To keep platform services off GPU nodes, [taint](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) the GPU nodes with `nvidia.com/gpu=true:NoSchedule`. Each model then needs a matching toleration in `FlowModel.models..tolerations`. See [Run models on reserved GPU nodes](/self-managed/flow/prepare/models#run-models-on-reserved-gpu-nodes). ## Check the cluster Run these checks after the nodes are ready. Each one tells you a value to record. Add the cluster to your `kubectl` configuration: ```bash theme={null} aws eks update-kubeconfig --name --region ``` Then confirm every node shows `Ready`: ```bash theme={null} kubectl get nodes ``` To check that Metrics Server is running, run: ```bash theme={null} kubectl top nodes ``` If the command prints CPU and memory for each node, Metrics Server is running. If it returns an error, add the Metrics Server add-on to the cluster. It takes about a minute to start reporting. The NVIDIA device plugin makes each node's GPUs available to Kubernetes. To check whether it's running, run: ```bash theme={null} kubectl get daemonset --all-namespaces | grep nvidia-device-plugin ``` If the command prints nothing, the plugin isn't running. Install it before you install SeekrFlow, which doesn't install it for you: ```bash theme={null} helm repo add nvdp https://nvidia.github.io/k8s-device-plugin helm install nvidia-device-plugin nvdp/nvidia-device-plugin --namespace kube-system ``` For other installation options, see the [NVIDIA device plugin](https://github.com/NVIDIA/k8s-device-plugin) documentation. The Bottlerocket NVIDIA AMIs include the plugin. The Amazon Linux 2023 NVIDIA AMIs don't. Kubernetes needs a label to identify which nodes are GPU nodes. Any key-value pair works as long as it matches what is on your nodes. To see the labels on your nodes, run: ```bash theme={null} kubectl get nodes --show-labels ``` The Amazon Linux 2023 NVIDIA AMIs add `nvidia.com/gpu.present: "true"`. If all your GPU nodes share one instance type, you can also use `node.kubernetes.io/instance-type`, such as `node.kubernetes.io/instance-type: g5.xlarge`. Multi-node fine-tuning needs the JobSet controller. SeekrFlow can install it during configuration, but a cluster must not run two. To check whether one is installed, run: ```bash theme={null} kubectl get crd jobsets.jobset.x-k8s.io ``` If the command returns `NotFound`, JobSet isn't installed. Record `true` for `jobset.enabled` only if you need multi-node fine-tuning. Otherwise, record `false`. If the command returns the resource, JobSet is already installed. Record `false`. Set your `kubectl` context to the cluster, using your Kubernetes distribution's process. Then confirm every node shows `Ready`: ```bash theme={null} kubectl get nodes ``` To check that Metrics Server is running, run: ```bash theme={null} kubectl top nodes ``` If the command prints CPU and memory for each node, Metrics Server is running. If it returns an error, install [Metrics Server](https://github.com/kubernetes-sigs/metrics-server). It takes about a minute to start reporting. Run `nvidia-smi` on each GPU node. The **Driver Version** it reports must be 580 or later. The NVIDIA device plugin makes each node's GPUs available to Kubernetes. To check whether it's running, run: ```bash theme={null} kubectl get daemonset --all-namespaces | grep nvidia-device-plugin ``` If the command prints nothing, the plugin isn't running. Install it before you install SeekrFlow, which doesn't install it for you: ```bash theme={null} helm repo add nvdp https://nvidia.github.io/k8s-device-plugin helm install nvidia-device-plugin nvdp/nvidia-device-plugin --namespace kube-system ``` For other installation options, see the [NVIDIA device plugin](https://github.com/NVIDIA/k8s-device-plugin) documentation. Kubernetes needs a label to identify which nodes are GPU nodes. Any key-value pair works as long as it matches what is on your nodes. To see the labels on your nodes, run: ```bash theme={null} kubectl get nodes --show-labels ``` Use whatever label your GPU nodes carry. Multi-node fine-tuning needs the JobSet controller. SeekrFlow can install it during configuration, but a cluster must not run two. To check whether one is installed, run: ```bash theme={null} kubectl get crd jobsets.jobset.x-k8s.io ``` If the command returns `NotFound`, JobSet isn't installed. Record `true` for `jobset.enabled` only if you need multi-node fine-tuning. Otherwise, record `false`. If the command returns the resource, JobSet is already installed. Record `false`. ## Create the namespaces SeekrFlow installs into the `seekrflow` namespace and runs training and ingestion jobs in the `argo-workflows` namespace. Create both before you continue. Later steps add network policies, secrets, and cloud identity settings to them. ```bash theme={null} kubectl create namespace seekrflow kubectl create namespace argo-workflows ``` If a namespace already exists, `kubectl` returns an `AlreadyExists` error. You can continue. ## Record the values Note these values. You will enter them when you create the values file. | Value | Used for | Example or format | | - | - | - | | Cluster name | `global.cloudProvider.clusterName` | `seekrflow-prod` | | AWS Region, AWS only | `global.cloudProvider.aws.region` | `us-east-1` | | Kubernetes context | Your `sfe` profile | The `kubectl` context for this cluster. | | GPU node label | `global.defaults.gpuNodeLabel` | `nvidia.com/gpu.present: "true"` | | Whether a JobSet controller is running | `jobset.enabled` | `true` only if you need multi-node fine-tuning and the cluster has no JobSet controller. | This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.