Gateway
Wrap execution with additional controls.
A Gateway lets you extend an inferencer with additional functionality that can shape the request, provide context, and check the response.
- Apply input and output Guard Rails.
- Add Memory and Knowledgebases to the execution.
- Validate responses with a Validator, with invalid output corrected through another inferencer.
Define your Gateway
You name the Gateway and choose the inferencer it wraps. The Gateway then adds its configured functionality to that execution.
- The wrapped inferencer can be anything, a Model, a Load Balancer, or a whole Router.
- Your application calls the Gateway exactly the way it called the original inferencer.
Guard Rails for input and output
Guard Rails screen content against safety, privacy, security, and custom rules.
- Safety for hate speech, harassment, racism, sexual content, violence, misconduct, and denied topics.
- Privacy for PII, including email, phone, address, payment cards, credentials, and custom patterns.
- Security for prompt injection.
- Custom rules using words, phrases, prompts, conditions, and classifier classes.
- Applied to User Prompts, AI Output, or both.
Knowledge for every request
Knowledgebases provide content as context to the inferencer, making the same maintained knowledge available across chats and workflows.
- Text Items, URL Items, and uploaded files provide the knowledge.
- Items stay organized under each Knowledgebase as the knowledge grows.
- The knowledge remains available across chats and workflows, whichever model answers.
Expand what your Gateway can do
Additional functionality extends the Gateway with validation, cost management, conversation context, data cleanup, and debugging.
- Validator checks that responses match the format your application expects, such as clean JSON.
- Budget sets spending limits per run, chat, day, week, or month.
- Memory keeps conversation context, while Wiper clears stored data according to your policy.
- Log Group records requests and activity for debugging and review.
What a Gateway changes for your application
A Gateway wraps an existing inferencer and extends its execution with protection, knowledge, memory, validation, cost limits, cleanup, and activity logging. Your application still has one endpoint for the entire flow.
Protect input and output
Guard Rails inspect user prompts, AI output, or both against safety, privacy, security, and custom rules. A Gateway can combine multiple Guard Rails so different protections apply to different parts of the execution.
Give every request the knowledge and context it needs
Knowledgebases provide maintained content to the inferencer as context, while Memory keeps conversation history available across requests. Both remain part of the Gateway regardless of which model or inferencer handles the request.
Keep one endpoint while the flow changes
The Gateway can wrap any inferencer, from a single Model to a Router, Load Balancer, Failover, or Race. Your application calls the Gateway exactly the way it called the original inferencer, while the execution behind that endpoint can change as your requirements evolve.
A Gateway can also become part of a larger flow, serving as a route in a Router, a target of a Load Balancer, a fallback in a Failover, or a contender in a Race.
Ready to build your AI execution flow?
Configure your first inferencer with help from our team.