> ## Documentation Index
> Fetch the complete documentation index at: https://docs.seekr.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Release notes 2024
export const SupportedOn = ({ui = false, api = true, sdk = true}) => ;
> Improvements & Bug Fixes
### Tooltip for Icon Buttons in Model Library
We’ve improved the user experience in the Model Library. Icon buttons in base model cards now display a hover tooltip with descriptive text, ensuring users clearly understand the available actions.
### Faster Response Rendering in Sandbox
The simulated typing animation for sandbox responses now matches the speed of token count updates. This enhancement delivers quicker rendering, offering a more seamless and accurate representation of response generation.
### Run Description Added to Run Summary
The run summary now displays the description entered during the run creation wizard, enhancing clarity and alignment with the design specifications.
### Minor Bug Fixes and Stability Improvements
This release includes several minor bug fixes and performance enhancements across SeekrFlow™ to ensure a smoother and more reliable user experience.
November’s release introduces new features designed to give users greater control and efficiency over their AI workflows—from optimizing model outputs to managing large-scale deployments. For more details, read our full release blog.
> New Features
### Sandbox Input Parameters
We have added new input parameters to the Sandbox environment: Temperature, Top P, and Max Tokens. These options give more control over inference outputs, allowing users tailor responses for specific tasks and use cases.
### Enhanced Inference Engine for Faster Inference
With the integration of vLLM, inference speeds have dramatically improved, making your AI workflows faster and more efficient. This integration ensures that even complex models deliver faster results, helping users achieve more in less time.
We conducted performance testing to compare TGI and vLLM on Intel Gaudi2 accelerators.
* **TGI**: tgi-gaudi v2.0.4
* **vLLM**: vllm-fork v0.5.3.post1-Gaudi-1.17.0
On average, the enhanced inference engine performed 32% faster when compared to TGI for the RAG experiments (full interaction traces, 10 concurrent requests) using Meta-Llama-3.1-8B-Instruct
Significant latency improvements in load testing with 100 concurrent users:
* **Meta-Llama-3-8B-Instruct**: \~45% faster
* **Meta-Llama-3.1-8B-Instruct**: \~39% faster
### Enhanced OpenAI compatibility features
Our inference engine now seamlessly integrates with OpenAI’s ecosystem, expanding workflow capabilities and enhancing usability.
* **Log Probabilities**: New support for log\_probs and top\_logprobs, providing insights into model decision-making, aiding debugging, and improving output accuracy.
* **Dynamic Tool Calling**: Custom functions can now be automatically invoked by the model based on context, streamlining business logic integration.
### Try it yourself!
The example code below shows you how to leverage the OpenAI client and SeekrFlow's inference engine to create a custom unit conversion tool that can be configured dynamically.
#### Create the client and make an API request
```python Python expandable theme={null}
import os
import openai
# Set the API key
os.environ["OPENAI_API_KEY"] = "Paste your API key here"
# Create the OpenAI client and retrieve the API key.
client = openai.OpenAI(
base_url="https://flow.seekr.com/v1/inference",
api_key=os.environ.get("OPENAI_API_KEY")
)
# Send a request to the OpenAI API to leverage the specified Llama model as a unit conversion tool.
response = client.chat.completions.create(
model="meta-llama/Llama-3.1-8B-Instruct",
stream=False,
messages=[{
"role": "user",
"content": "Convert from 5 kilometers to miles"
}],
max_tokens=100,
tools=[{
"type": "function",
"function": {
"name": "convert_units",
"description": "Convert between different units of measurement",
"parameters": {
"type": "object",
"properties": {
"value": {"type": "number"},
"from_unit": {"type": "string"},
"to_unit": {"type": "string"}
},
"required": ["value", "from_unit", "to_unit"]
}
}
}]
)
```
#### Register a function from JSONNext, define and register a Python function from JSON data.
```python Python theme={null}
# Parse json and register
def register_from_json(json_obj):
code = f"def {json_obj['name']}({', '.join(json_obj['args'])}):\n{json_obj['docstring']}\n{json_obj['code']}"
print(code)
namespace = {}
exec(code, namespace)
return namespace[json_obj["name"]]
```
#### Run the unit conversion toolThis function executes the tool call, given an LLM response object.
```python Python theme={null}
# Execute our tool
def execute_tool_call(resp):
tool_call = resp.choices[0].message.tool_calls[0]
func_name = tool_call.function.name
args = tool_call.function.arguments
func = globals().get(func_name)
if not func:
raise ValueError(f"Function {func_name} not found")
if isinstance(args, str):
import json
args = json.loads(args)
return func(**args)
execute_tool_call(response)
```
#### Sample output
This is the output expected in response to the request made earlier to convert 5 kilometers to miles.
```python Python theme={null}
3.106855
```
### Federated Login with Intel
Intel® Tiber™ AI Cloud users can now access SeekrFlow™ with a new federated login feature
First-time users: Start by using your Intel® Tiber™ AI Cloud credentials, which will auto-populate the sign-up form for quick and easy access to SeekrFlow.
Returning users: Simply log in with your Intel® Tiber™ AI Cloud credentials for direct access to SeekrFlow.
This integration simplifies user management and access for those connected to Intel® Tiber™ AI Cloud.
> Improvements & Bug Fixes
### Streaming in Sandbox
We have enabled streaming for chat responses in the Sandbox, delivering results incrementally so users can utilize results without a delay.
### Clear and Restart Button Fix
We have resolved an issue where the “Clear and Restart” button in Sandbox didn’t function. A dialog box now appears, confirming that chat history will be cleared, allowing you to start fresh.
### Dataset Directory Update
Uploaded file improvements in the Create Run Modal
* Successfully uploaded files immediately appear in the dataset directory.
* Switching to the directory view auto-selects the newly uploaded file.
* Radio buttons now remain active, ensuring smooth file selection.
> UI/UX Enhancements
We have made several updates to improve the user experience and provide clearer guidance across the platform
* **Sandbox**: Updated language to better support the new model parameter settings for improved clarity.
* **Deployment Dashboard**: Enhanced explanations of cost transparency features for better understanding of resource usage.
* **Projects**: Cancellation dialogs now show detailed cost information related to token usage, offering users more visibility into their resource consumption.
These updates aim to make SeekrFlow’s interface more intuitive and user-friendly, enhancing navigation and overall clarity.
> UI/UX Enhancements
### Quickstart Card Reorganization
We have reorganized the Quickstart cards to improve navigation and help users find essential features and tools faster, making it easier to get started and accelerate the time to deployment.
### New Icons and Tooltips
New icons and tooltips have been introduced across the platform. These provide quick access to relevant documentation without interrupting your workflow, ensuring that users can reference helpful resources whenever needed.
### Enhanced User Guidance
We have updated the user guidance system to provide clearer instructions throughout the process, with a focus on alignment and data creation. Users will now find an embedded documentation tab within the UI, offering step-by-step instructions and best practices for principle alignment and data creation. These updates are designed to support users of all technical backgrounds, helping them navigate the platform with confidence and accelerate their AI model deployment.
### The New SeekrFlow™ Self-Service Enterprise AI Platform
SeekrFlow™ is now available as a complete, self-service platform that empowers enterprises to train, validate, deploy, and scale trusted AI applications with ease.
This includes:
* **Lifecycle Management**: Manage the entire AI lifecycle through a single API call, SDK, or no-code interface.
* **Principle Alignment**: An intelligent agent that simplifies the process of aligning AI models to domain-specific knowledge, such as company policies, industry regulation,s or brand guidelines. This feature enhances the accuracy and relevance of base model responses by 3x and 6x respectively, at 90% reduced cost compared to traditional data preparation methods.
* **Five-step Deployment**: Simplify and accelerate the deployment process with an intuitive, guided workflow.
* **Model Validation Tools**: Ensure model accuracy with advanced validation features, including side-by-side comparisons and token-level confidence scoring.
* **Real-time Monitoring**: Monitor model performance and production health to maintain reliability and optimize outcomes.
* **Flexible Cloud and Hardware Deployment**: Deploy models seamlessly on all leading cloud providers and hardware platforms, offering the flexibility to scale based on your needs.
For more details, read the full blog [here](https://www.seekr.com/the-new-seekrflow-self-service-enterprise-ai-platform/).
