> ## Documentation Index
> Fetch the complete documentation index at: https://docs.seekr.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Question-answering bot with confidence scoring
> Build a Q&A bot with SeekrFlow that returns a confidence score alongside each answer.
export const DownloadNotebook = ({href, filename}) =>
;
};
This recipe builds a question-answering bot with SeekrFlow that pairs every
answer with a confidence score, a signal for how much to trust the reply. It
calls a chat model directly through SeekrFlow's serverless inference and parses
the rating out of the response, with no extra frameworks.
## What you'll build
A Q\&A bot that:
1. Answers questions about any topic.
2. Returns a confidence score with each answer.
3. Flags low-confidence answers for follow-up.
## Prerequisites
* A SeekrFlow API key, set as the `SEEKR_API_KEY` environment variable
* Python 3.8 or later
* The SeekrFlow SDK: `pip install seekrai`
Each question you ask is billed as serverless inference, charged by the token.
## Build it
Create `qa_bot.py` with the imports, configuration, and client.
```python qa_bot.py theme={null}
import os
import re
from seekrai import SeekrFlow
API_KEY = os.environ["SEEKR_API_KEY"]
API_URL = "https://flow.seekr.com/v1/"
MODEL_ID = "meta-llama/Meta-Llama-3-8B-Instruct"
client = SeekrFlow(api_key=API_KEY, base_url=API_URL)
```
Write a prompt that asks the model for an answer followed by a confidence rating
on its own line. Parsing depends on that `Confidence:` marker. Keep it in the
instructions.
```python qa_bot.py theme={null}
TEMPLATE = """Question: {question}
Answer the question above. Then, on a new line starting with 'Confidence:', rate
your confidence from 1 (complete guess) to 10 (absolutely certain), and briefly
explain the rating.
"""
```
Write a helper that splits the answer from the confidence rating and its
explanation.
```python qa_bot.py theme={null}
def parse_response(response: str) -> dict:
"""Extract the answer, confidence score, and explanation from a reply."""
answer, confidence, explanation = response, "Not provided", ""
if "Confidence:" in response:
answer, _, confidence_part = response.partition("Confidence:")
answer = answer.strip()
confidence_part = confidence_part.strip()
rating_match = re.search(r"(\d+)(/10)?", confidence_part)
if rating_match:
confidence = f"{rating_match.group(1)}/10"
explanation = re.sub(r"^\d+(/10)?", "", confidence_part).strip()
return {"answer": answer, "confidence": confidence, "explanation": explanation}
```
Wrap the model call and parser in a function that prints the answer, its
confidence, and a follow-up hint when confidence is low. A low temperature keeps
answers more deterministic.
```python qa_bot.py theme={null}
def ask_question(question: str) -> dict:
"""Send a question to the model and print the parsed result."""
response = client.chat.completions.create(
model=MODEL_ID,
messages=[{"role": "user", "content": TEMPLATE.format(question=question)}],
temperature=0.3,
)
parsed = parse_response(response.choices[0].message.content)
print(f"Question: {question}\n")
print(f"Answer: {parsed['answer']}\n")
print(f"Confidence: {parsed['confidence']}")
if parsed["explanation"]:
print(f"Explanation: {parsed['explanation']}")
if parsed["confidence"] in ("1/10", "2/10", "3/10"):
print("\nLow confidence. Consider rephrasing or asking for more specifics.")
return parsed
```
Run a few questions of varying difficulty to see the confidence score change.
```python qa_bot.py theme={null}
questions = [
"What is the capital of France?",
"How do quantum computers work?",
"When was the book 'Trilby' by George du Maurier published?",
]
for q in questions:
ask_question(q)
print("-" * 50)
```
Run the finished script:
```bash theme={null}
python qa_bot.py
```
Easy, well-known questions should come back with high confidence; obscure ones
should score lower and trigger the follow-up hint.
## Next steps
* **Keep context across turns.** Hold a running `messages` list and include prior
turns in each call to answer follow-up questions in context.
* **Tune the threshold.** Adjust which confidence scores trigger the follow-up
hint to match how cautious you want the bot to be.
* **Ground the answers.** Combine this with the [New hire onboarding agent with citations](/flow/recipes/new-hire-onboarding-agent-with-citations) so answers come from your own documents.