Digital Event Horizon
Together AI Launches Together Storage Accelerator: A Game-Changing Data Caching Solution for AI Workloads
Together Storage Accelerator (TSA) is a groundbreaking data caching solution designed to alleviate slow data reads in AI applications.TSA provides a distributed caching layer for GPU clusters, leveraging ASTRA technology to accelerate data caching.The solution offers significantly faster warm reads, with a 20x improvement over cross-cloud reads.Cold reads cost nothing extra, and cached data keeps serving even when the upstream is slow or throttled.The solution is available now on Together GPU Clusters at no additional cost.TSA is powered by ASTRA, a distributed caching technology developed by Together AI.
The tech industry is witnessing a significant shift towards AI and machine learning, and companies are racing to provide solutions that can support the growing demand for AI capabilities. In this context, Together AI has introduced a groundbreaking data caching solution called Together Storage Accelerator (TSA), which promises to revolutionize data caching for AI applications. This innovative solution has been designed to alleviate the issue of repeated data reads from object storage and Hugging Face Hub, thereby dramatically accelerating AI data pipelines while cutting data egress bills.
According to the context provided, the problem with current data caching solutions lies in the fact that data lives far from the GPUs, leading to slow, expensive, and rate-limited cross-cloud or cross-region reads. This results in wasted time, money, and failed jobs when the origin throttles the requests. To address this issue, Together Storage Accelerator has been developed, which provides a distributed caching layer for GPU clusters. It leverages the distributed caching technology built by ASTRA, which has been used to feed some of the largest GPU fleets in the world.
The Together Storage Accelerator is designed to run on every GPU node in the cluster and presents drop-in endpoints for the storage already used. The first time any node reads an object, it is cached on the cluster NVMe, resulting in dramatically higher local speed for subsequent reads. According to the context, the result measured on a Together GPU Cluster H100 node is approximately 20 times faster warm reads than pulling from a cross-cloud origin, with a 2 GiB object that took 13-15 seconds to download from Cloudflare R2 being read from the cache in 0.6-0.7 seconds.
Another significant benefit of the Together Storage Accelerator is the fact that cold reads cost nothing extra, and the cached data keeps serving even when the upstream won't. This means that the performance of the system is not affected by the origin being slow or throttled. The solution can be used without making any changes to the application, and the existing boto3 code, transformers calls, and CLI workflows work as-is.
The Together Storage Accelerator is available now on Together GPU Clusters at no additional cost, and it is included as a free feature with every Together GPU Cluster. The solution is powered by ASTRA, a distributed caching technology developed by Together AI. The company plans to share a deeper technical look at ASTRA in an upcoming engineering post.
In conclusion, the Together Storage Accelerator is a game-changing data caching solution that promises to revolutionize data caching for AI applications. With its distributed caching layer and ASTRA technology, it offers significant benefits in terms of performance, reliability, and cost. As the demand for AI capabilities continues to grow, this solution is poised to make a significant impact in the industry.
Related Information:
https://www.digitaleventhorizon.com/articles/Together-Storage-Accelerator-Revolutionizing-Data-Caching-for-AI-Applications-deh.shtml
https://www.together.ai/blog/introducing-together-storage-accelerator
Published: Thu Nov 19 17:00:35 2026 by llama3.2 3B Q4_K_M