Failover
Continue execution when a target fails.
A Failover automatically moves a request to the next target when the current target returns an error or exceeds its configured timeout.
- Keep multiple inferencers available as fallback paths.
- Set a Timeout for each target to determine when execution moves on.
- Return the first successful response.
Define your Failover
A name and description identify the Failover and make it easier to understand when managing multiple execution paths. The actual fallback sequence is defined through its Targets.
Add your Targets
Targets are tried from top to bottom, each with its own Timeout.
- The Timeout (ms) determines how long a target may run before the request moves to the next one.
- Order determines priority. The first target is tried first, and the others provide fallback paths when it returns an error or exceeds its timeout.
- A target can be any inferencer, including a Load Balancer or even another Failover.
Expand what your Failover can do
You can extend the Failover with functionality for validation, cost management, conversation context, data cleanup, and debugging. Your application keeps calling the same endpoint, while each addition handles its specific responsibility on every request.
- A Validator makes sure every answer arrives in the format you expect, such as clean JSON.
- A Budget caps what the Failover can spend per run, chat, day, week, or month.
- A Memory lets conversations remember earlier context, while a Wiper clears stored data according to your policy.
- A Log Group keeps a record of requests for debugging and review.
Select the functionality you need, and click Apply.
What a Failover changes for your application
A bad minute at one provider stops being a bad minute for you or your users. If one target fails, another can take over.
Keep requests moving when a target fails
When a target returns an error or exceeds its Timeout (ms), the request moves to the next target in the chain. The first successful response is returned to your application.
Give each target its own timeout
Each target has its own Timeout (ms), so you can give a primary target less time to respond while allowing a backup target more time when needed.
Change the fallback order without code changes
Reorder the targets or change their timeouts in the console, and subsequent requests follow the new fallback path without changing your application code.
A Failover can also stand inside any other inferencer. Use it as a target of a Load Balancer, a contender in a Race, or a route in a Router.
Ready to build your AI execution flow?
Configure your first inferencer with help from our team.