call
Go to Console

Failover Console Guide

This page provides a guide to the Failover entity in the Namirasoft Inference Console. It defines the concepts and configuration fields used when creating and managing Failovers. Use this guide to understand the purpose and behavior of each setting available during Failover configuration.

What Is a Failover?

A Failover in Namirasoft Inference tries its targets in a set order, so requests keep moving. It sends a request to the first target, and if that target does not respond within its timeout, it moves on to the next target, and so on, until one responds. Each target is an inferencer, which can be a Model or any other inferencer.

Any application can call a Failover through the Namirasoft Inference API, in the same way as a Model. When a request arrives, the Failover works through its targets in order until it receives a response. Namirasoft applications such as Namirasoft Expert and Namirasoft Job Arranger also use it.

The Challenge a Failover Solves

When an application depends on a single model, a slow or unavailable provider leaves requests without a response and no defined path to continue.

Common challenges include:

  • Single point of dependence: With only one model, there is no alternative to turn to when a provider is slow or unavailable. Every request shares the fate of that one provider.
  • Provider interruptions: A provider outage or a delayed response can hold up requests with no automatic next step. Users wait on a timeout that has no plan behind it.
  • Manual recovery: Switching to another model during an interruption would otherwise require changing each application. By the time the change ships, the interruption is usually over.

How Namirasoft Inference Solves the Problem

A Failover lets you define an ordered list of targets and a timeout for each. Requests try the first target, and when a target returns an error or does not respond in time, the Failover automatically continues with the next available target. This keeps requests moving during a provider interruption, without changing the applications that use the Failover.

Overview of Failover Fields and Options

Below is a detailed explanation of the fields available when creating or managing a Failover. Understanding these fields helps ensure your Failover is configured correctly for your requirements.

  • ID (String): This is a unique identifier automatically assigned to the Failover when it is created. The system uses it to track, reference, and manage this specific Failover. This value is auto-generated and cannot be modified.
  • User ID (Namirasoft Account’s ID): This is the unique identifier of the Namirasoft Account user who owns this Failover. It is used internally for permission control, audit logging, and access management.
  • Workspace ID (Namirasoft Workspace’s ID): This is the identifier of the workspace this Failover belongs to, as defined in Namirasoft Workspace. A workspace is a shared organizational space where teams group their configurations, projects, and members.
  • Name (String): This is a label used to identify this Failover in the console. A good name clearly describes its purpose, for example: “Primary with Backups” or “High Availability Chain”.
  • Description (String): This is a note that describes the Failover and its purpose. It is for your reference and does not affect routing.
  • Targets (List): These are the inferencers this Failover tries, in order. Add one or more targets, each with the following:
    • Type (Enum): This is the kind of target, one of Model, Load Balancer, Failover, Race, Router, or Gateway. Because a target can itself be another execution flow, Failovers can be combined with other inferencers.
    • Target: This is the specific inferencer, of the chosen Type, that receives the request when it is this target’s turn.
    • Timeout (Integer, milliseconds): This sets the maximum time, in milliseconds, to wait for this target to respond before moving on to the next target.
  • Budget (Budget’s ID): This determines whether spending through the Failover is subject to a defined limit. A Budget can set spending ceilings per run, chat, day, week, or month. The same Budget can be used by several inferencers.
  • Memory (Memory’s ID): This determines whether the Failover keeps conversation context across requests. When a Memory is connected, relevant conversation history is stored and provided as context when the Failover handles later requests.
  • Wiper (Wiper’s ID): This determines whether stored data associated with the Failover is cleaned up according to a defined policy. A Wiper can remove data based on time or count limits.
  • Log Group (Log Group’s ID): This determines where the Failover’s execution logs are recorded. When a Log Group, defined in Namirasoft Log, is connected, request activity and errors are recorded there for debugging and review.
  • Created At (DateTime): This is the date and time when this Failover was created. This value is automatically generated and cannot be modified.
  • Updated At (DateTime): This is the date and time when this Failover configuration was last modified. This value is updated automatically whenever any field is changed.

Ready to Configure Your Failover or Want to Explore the Console Guide?

Get Started Explore Console Guide