Race Console Guide

This page provides a guide to the Race entity in the Namirasoft Inference Console. It defines the concepts and configuration fields used when creating and managing Races. Use this guide to understand the purpose and behavior of each setting available during Race configuration.

What Is a Race?

A Race in Namirasoft Inference sends a request to all of its targets at the same time and uses the first response that arrives. Each target is an inferencer, which can be a Model or any other inferencer.

Any application can call a Race through the Namirasoft Inference API, in the same way as a Model. When a request arrives, the Race calls every target at once and returns the fastest response. Namirasoft applications such as Namirasoft Expert and Namirasoft Job Arranger also use it.

The Challenge a Race Solves

Response times vary between models and change with load, so choosing one model in advance does not guarantee the fastest reply for every request.

Common challenges include:

• Variable response times: The fastest model for one request may not be the fastest for the next. Speed shifts with load, time of day, and the request itself.
• Waiting on a single model: Sending to one model means waiting for that model, even when another could have replied sooner. The user waits on a choice made in advance, not on the fastest option available.
• Latency sensitive work: Some requests need the quickest possible reply, whichever model provides it. No single pick can promise that every time.

How Namirasoft Inference Solves the Problem

A Race sends each request to all of its targets at once and returns the first response to arrive. This gives you the fastest available reply for every request, without deciding in advance which model will be quickest, and without changing the applications that use the Race.

Overview of Race Fields and Options

Below is a detailed explanation of the fields available when creating or managing a Race. Understanding these fields helps ensure your Race is configured correctly for your requirements.

• ID (String): This is a unique identifier automatically assigned to the Race when it is created. The system uses it to track, reference, and manage this specific Race. This value is auto-generated and cannot be modified.

• User ID (Namirasoft Account’s ID): This is the unique identifier of the Namirasoft Account user who owns this Race. It is used internally for permission control, audit logging, and access management.

• Workspace ID (Namirasoft Workspace’s ID): This is the identifier of the workspace this Race belongs to, as defined in Namirasoft Workspace. A workspace is a shared organizational space where teams group their configurations, projects, and members.

• Name (String): This is a label used to identify this Race in the console. A good name clearly describes its purpose, for example: “Fastest Reply” or “Low Latency Pool”.

• Description (String): This is a note that describes the Race and its purpose. It is for your reference and does not affect routing.

• Targets (List): These are the inferencers this Race calls at the same time. Add one or more targets, each with the following:
  • Type (Enum): This is the kind of target, one of Model, Load Balancer, Failover, Race, Router, or Gateway. Because a target can itself be another execution flow, Races can be combined with other inferencers.
  • Target: This is the specific inferencer, of the chosen Type, that receives the request. Every target is called at the same time, and the first to respond wins.

• Budget (Budget’s ID): This determines whether spending through the Race is subject to a defined limit. A Budget can set spending ceilings per run, chat, day, week, or month. The same Budget can be used by several inferencers.

• Memory (Memory’s ID): This determines whether the Race keeps conversation context across requests. When a Memory is connected, relevant conversation history is stored and provided as context when the Race handles later requests.

• Wiper (Wiper’s ID): This determines whether stored data associated with the Race is cleaned up according to a defined policy. A Wiper can remove data based on time or count limits.

• Log Group (Log Group’s ID): This determines where the Race’s execution logs are recorded. When a Log Group, defined in Namirasoft Log, is connected, request activity and errors are recorded there for debugging and review.

• Created At (DateTime): This is the date and time when this Race was created. This value is automatically generated and cannot be modified.

• Updated At (DateTime): This is the date and time when this Race configuration was last modified. This value is updated automatically whenever any field is changed.