Add and manage datasets

Your dataset library holds the data you test models on. Add data from a file, from a dataset that already exists in Seekr such as one generated by AI-Ready Data, or from HuggingFace, then confirm how its columns map onto the message templates so evaluators and examinations know what to send a model and how to score the response. Each row supplies a user question, and optionally a system prompt, retrieved context, prior conversation, and an expected answer. You can build your dataset library before adding any models or evaluators.

Prepare your dataset

Each row in your dataset is one test case. Its columns supply the messages SeekrGuard sends to a model and the reference material evaluators score the response against.

Map columns to message templates

When you add a dataset, you map its columns onto SeekrGuard’s message templates. Include a column for each field you need:

Field	Contents
System Message	The system or instruction prompt (optional)
User Message	The user’s question or query
Context	Background or retrieved material for RAG scenarios (optional)
Chat History	Prior turns of a multi-turn conversation (optional)
Assistant Message	The expected or reference answer (optional)

Which fields a dataset contains determines which evaluators it can run. For example, an evaluator that requires an expected answer can only run on a dataset that maps Assistant Message. See Run examinations.

Add columns for tool evaluation

The tool evaluators, such as Tool Name Match and Tool Selection, score the tool calls a model makes. To run them, add two columns to your dataset file. One lists the tools the model should call, and the other defines the tools it can choose from.

Column	Accepted headers
Expected tool calls	`expected_tool_calls` or `Expected Tool Calls`
Tool definitions	`tool_definitions` or `Tool Definitions`

Headers must match exactly, including case. SeekrGuard treats any other spelling, such as expectedToolCalls, as an ordinary column.

Format expected tool calls

List the tools the model should call, separated by commas:

search_knowledge_base,get_order_status

Write plain tool names, without brackets or quotes.
Use names that appear in the same row’s tool definitions.
On a row that isn’t testing tool use, leave the cell empty. Evaluators that need expected tool calls skip that row.

Format tool definitions

Write a JSON array of tool definitions, one per tool, and repeat the full array on every row. Each definition has a type of function and a function object with the tool’s name, description, and parameters:

[
  {
    "type": "function",
    "function": {
      "name": "get_order_status",
      "description": "Look up the shipping status of a customer order by its order number.",
      "parameters": {
        "type": "object",
        "properties": {
          "order_number": {
            "type": "string",
            "description": "The order number, such as A-10492"
          }
        },
        "required": ["order_number"]
      }
    }
  }
]

Use a valid JSON array, even for a single tool.
In parameters, declare the required fields and a type for each property. Tool Arg Schema Validation checks the model’s arguments against them.
Give each tool a specific description. The model reads it to decide which tool to call.

Format the context column

Tools don’t run during an examination. SeekrGuard reads each tool’s result from the row’s context column instead. Format the context as a JSON object keyed by tool name:

{
  "search_knowledge_base": {
    "content": "Orders ship within two business days of payment."
  },
  "get_order_status": {
    "content": "Order A-10492 shipped on May 3 and is in transit."
  }
}

Add a dataset

Add Dataset

On the Datasets page, click Add Dataset.

Upload Dataset

Enter a Name and Description, then choose a tab under Data Source.

Upload File
Seekr Dataset
HuggingFace

Drag and drop a CSV or Parquet file onto the upload area, or click the area to choose a file from your computer.

Enter a dataset name in the format organization/dataset-name, such as truthfulqa/truthful_qa.
Click Check Dataset.
Choose a Configuration. Each one shows its row count.

Click Next.

Configure Templates

Your source columns appear as chips under Available Columns. Drag them onto the message templates.

A dataset generated by AI-Ready Data arrives with System Message, User Message, and Assistant Message already mapped. Review the Live Preview and continue.

To add placeholders by typing instead of dragging, click Edit on a message template.
A message template is a pattern applied to every row. A placeholder, written {{{column_name}}}, marks where that column’s value goes when the row is sent to a model.
The Live Preview panel shows the messages a row will send to the model. Switch rows with the Example list.

If your file has tool columns, a message confirms that SeekrGuard detected them. If the message doesn’t appear, check the column headers.Click Next.

Preview & Save Dataset

Review the Dataset Preview, Template Configuration, and Sample Data Preview, then click Save Dataset. Any template you left unmapped reads not set in Template Configuration.

Manage your datasets

The Datasets list shows one row per dataset, with Name, Description, Num. Input Samples, Num. Output Samples, Last Updated, Visibility, and Actions. Each dataset’s Visibility appears as a chip: private, public, or collaborative. See Visibility. Above the table, filter by Name, reorder the list with Sort By, and limit it to your own datasets with Only mine. Each row’s Actions column offers:

View Dataset – open the dataset’s detail page to inspect its inputs and outputs.
Delete Dataset – available only if you own the dataset or you are an admin. The dataset is archived rather than permanently destroyed.

SeekrGuard Overview

SeekrGuard App

Reference

Add and manage datasets

Prepare your dataset

Map columns to message templates

Add columns for tool evaluation

Format expected tool calls

Format tool definitions

Format the context column

Add a dataset

Manage your datasets

​Prepare your dataset

​Map columns to message templates

​Add columns for tool evaluation

​Format expected tool calls

​Format tool definitions

​Format the context column

​Add a dataset

​Manage your datasets

Prepare your dataset

Map columns to message templates

Add columns for tool evaluation

Format expected tool calls

Format tool definitions

Format the context column

Add a dataset

Manage your datasets