Load Balancer Console Guide

This page provides a guide to the Load Balancer entity in the Namirasoft Inference Console. It defines the concepts and configuration fields used when creating and managing Load Balancers. Use this guide to understand the purpose and behavior of each setting available during Load Balancer configuration.

What Is a Load Balancer?

A Load Balancer in Namirasoft Inference distributes requests across several targets, so that no single target carries all the load. Each target is an inferencer, which can be a Model or another routing configuration, and each is given a weight that sets how much traffic it receives.

 

A Load Balancer is used by supported Namirasoft applications, such as Namirasoft Expert and Namirasoft Job Arranger, in the same way as a Model. When a supported application sends a request, the Load Balancer selects one of its targets using the configured algorithm and the target weights.

The Challenge a Load Balancer Solves

Sending every request to a single model concentrates load in one place, can reach a provider’s usage limits, and leaves no room to balance cost and speed across models.

 

Common challenges include:

 

• Concentrated load: A single model handles all traffic, with no way to spread requests across several models.
• Provider limits: High volume on one model can reach the usage limits of a single provider.
• Cost and speed trade-offs: Without weighting, there is no simple way to send more traffic to a cheaper or faster model.

How Namirasoft Inference Solves the Problem

A Load Balancer lets you group several targets and share requests across them. You give each target a weight and choose an algorithm that decides how the next request is routed. You can send more traffic to a faster or cheaper model, or spread traffic evenly to stay within each provider’s limits, all without changing the applications that use the Load Balancer.

Overview of Load Balancer Fields and Options

Below is a detailed explanation of the fields available when creating or managing a Load Balancer. Understanding these fields helps ensure your Load Balancer is configured correctly for your requirements.

• ID (String): This is a unique identifier automatically assigned to the Load Balancer when it is created. The system uses it to track, reference, and manage this specific Load Balancer. This value is auto-generated and cannot be modified.

• User ID (Namirasoft Account’s ID): This is the unique identifier of the Namirasoft Account user who owns this Load Balancer. It is used internally for permission control, audit logging, and access management.

• Workspace ID (Namirasoft Workspace’s ID): This is the identifier of the workspace this Load Balancer belongs to, as defined in Namirasoft Workspace. A workspace is a shared organizational space where teams group their configurations, projects, and members.

• Name (String): This is a label used to identify this Load Balancer in the console. A good name clearly describes its purpose, for example: “Primary Balancer” or “Cost Optimized Pool”.

• Algorithm (Enum): This decides how the next request is chosen among the targets.
  • Random: A target is chosen at random, in proportion to its weight.
  • Round Robin: Requests rotate through the targets in turn, adjusted by weight.
  • Least Response Time: The request is sent to the target that is currently responding fastest.

• Description (String): A note describing this Load Balancer and its purpose. It is for your reference and does not affect routing.

• Targets (List): The inferencers this Load Balancer distributes requests across. Add one or more targets, each with the following:
  • Type (Enum): The kind of target, one of Model, Load Balancer, Failover, or Race. Because a target can itself be a routing configuration, Load Balancers can be combined with other strategies.
  • Target: The specific inferencer, of the chosen Type, that receives requests.
  • Weight (Number): The relative share of traffic this target receives. A higher weight means more requests are routed to it.

• Created At (DateTime): This is the date and time when this Load Balancer was created. This value is automatically generated and cannot be modified.

• Updated At (DateTime): This is the date and time when this Load Balancer configuration was last modified. This value is updated automatically whenever any field is changed.