Designing multi-LLM products without vendor lock-in
The model should be a replaceable capability inside the product—not the architecture of the entire product.
06 Oct 2026 · 6 min
Start with a stable contract
A provider adapter should translate product-level requests into each vendor's format. The rest of the system should deal in capabilities, context and responses rather than provider-specific SDK objects.
Route for a reason
Model routing is useful when it follows an explicit requirement: capability, latency, cost, context size or availability. Routing without evaluation only makes behavior harder to understand.
Observe every boundary
Record provider, latency, usage, retrieval context and failure mode. Interchangeability is only valuable when the team can see how each option behaves under real workloads.
See the engineering decisions applied in a real project.
SLIIT COEAI Chatbot case study