Digital Event Horizon
A global fintech company partnered with Together AI to implement Dedicated Model Inference (DMI) for their coding agent traffic. Through DMI, the fintech gained control over scaling, custom-weight rollouts, and blue-green testing, enabling them to optimize their infrastructure for peak-load, relatively-low-TPS traffic patterns. The platform's performance was enhanced, with sustained performance during working-hours peaks and fast scaling under concentrated load. This partnership serves as a testament to the power of dedicated model inference in addressing complex inference needs.
The fintech company partnered with Together AI to implement Dedicated Model Inference (DMI) for their coding agent traffic. The DMI platform addressed the company's challenges in handling spiky and variable traffic patterns exacerbated by the COVID-19 pandemic and economic shifts. The fintech gained control over scaling, custom-weight rollouts, and blue-green testing, enabling optimization of their infrastructure. The DMI platform provided a metrics API, allowing the team to monitor usage and performance data programmatically. The partnership enabled the fintech to achieve significant improvements in scalability, flexibility, and self-service, while reducing coordination and waiting on capacity. The DMI platform enhanced the fintech's performance, sustaining performance during working-hours peaks and scaling under concentrated loads.
A global fintech company recently partnered with Together AI to implement Dedicated Model Inference (DMI) for their coding agent traffic. The company's coding assistant, which handles financial transactions for millions of users across dozens of markets, relies heavily on inference to process complex data. The fintech's infrastructure was facing challenges in handling the spiky and variable traffic patterns, which were exacerbated by the COVID-19 pandemic and subsequent economic shifts.
The company's initial experience with Together's earlier dedicated offering was successful, but it wasn't built to handle the specific workload and traffic patterns of their coding agent. Together AI's DMI platform was designed to address these challenges and provide a more scalable, flexible, and self-service solution for inference needs.
Through DMI, the fintech's engineers gained control over scaling, custom-weight rollouts, and blue-green testing, allowing them to optimize their infrastructure for peak-load, relatively-low-TPS traffic patterns. The platform provided a metrics API, enabling the team to monitor usage and performance data programmatically and make data-driven decisions.
The fintech's workload runs on GLM-5.2, a mixture-of-experts model built for long-horizon coding and agentic work. The traffic follows a spiky pattern, concentrated in engineering hours, with sharp bursts in concurrency and prompt size as more teams adopt agents into their workflow. To address this, the fintech prioritized concurrency over raw throughput, selecting a 256K context configuration that balanced performance and capacity.
The partnership with Together AI involved a self-service migration of the GLM-5.2 endpoint onto the DMI platform, which enabled the fintech's engineers to stand up their own endpoint and scale capacity directly without filing a request or waiting on the platform group. The customer's platform team also gained control over the entire endpoint lifecycle, including creation, sizing, scaling policy, and configuration changes.
By implementing DMI, the fintech was able to achieve significant improvements in scalability, flexibility, and self-service, while also reducing the need for coordination and waiting on capacity. The platform's performance was also enhanced, with sustained performance during working-hours peaks and fast scaling under concentrated load.
The fintech's experience with DMI serves as a testament to the power of dedicated model inference in addressing complex inference needs. By providing a self-service solution for inference needs, DMI enables organizations to optimize their infrastructure for peak-load, relatively-low-TPS traffic patterns, and achieve significant improvements in scalability, flexibility, and self-service.
Related Information:
https://www.digitaleventhorizon.com/articles/How-a-Global-Fintech-Revolutionized-Inference-with-Dedicated-Model-Inference-deh.shtml
https://www.together.ai/blog/global-fintech-scales-coding-agent-traffic-with-dedicated-model-inference
Published: Fri Sep 18 12:05:53 2026 by llama3.2 3B Q4_K_M
