Digital Event Horizon
Together AI has just released an updated fine-tuning platform that unlocks the power of long-context data, making it possible to fine-tune models from the Hugging Face Hub and leverage existing adaptations without starting from scratch. The company is also expanding its support for Preference Optimization and pushing forward with further optimizations of its training systems.
The fine-tuning platform has received an update, allowing for long-context abilities. A new option for full-context fine-tuning is available on larger models like Llama-3.3-70B. The platform supports Preference Optimization with advanced training objectives. Further optimizations are underway to increase context lengths while keeping runtime and costs low. The platform now fully supports batch_size=\"max\" when starting jobs through the API or Python client.
The fine-tuning platform developed by Together AI has just received a significant update, introducing new features that will revolutionize the way developers and researchers approach large language model training. At the heart of this update is the ability to harness long-context abilities in fine-tuning, making it possible for users to unlock the full potential of their models.
According to the company, this development was inspired by the growing trend of increasingly stronger models being trained for specific tasks and released nearly every single day by the community. As a result, there exists a plethora of models that have already been adapted for relevant tasks, waiting to be fine-tuned further.
To address this need, Together AI has made it possible to fine-tune models from the Hugging Face Hub, allowing users to leverage existing adaptations and configurations without having to start from scratch. This feature is now available for every user of Together AI's platform.
Furthermore, the company has also introduced a new option for full-context fine-tuning on larger models like Llama-3.3-70B. This setting allows users to take advantage of even longer context lengths, up to 131k tokens in some cases, without affecting runtime or costs.
In addition to these features, Together AI's platform is also being expanded to support Preference Optimization with more advanced training objectives. These include variants such as length-normalized DPO (LN-DPO), DPO+NLL (from the Iterative RPO paper), and SimPO through corresponding flags. Depending on the target setting, users can combine these options to arrive at the best setup for training on preference data.
The company is also pushing forward with further optimizations of its training systems, aiming to increase context lengths even beyond 100B+ models while keeping runtime and costs low. Users who require long-context training for specific models or need to further increase the context length can reach out to Together AI for support.
Lastly, the platform now fully supports batch_size="max" when starting jobs through the API or the Python client. This convenience option will always set the batch size to the highest value supported on the platform, regardless of the model or training mode (SFT or DPO).
The introduction of these features marks a significant milestone in Together AI's journey to provide the best tools for engineers and researchers working with large language models.
Related Information:
https://www.digitaleventhorizon.com/articles/New-Fine-Tuning-Platform-Released-by-Together-AI-Unlocking-the-Power-of-Long-Context-Data-deh.shtml
https://www.together.ai/blog/fine-tuning-updates-sept-2025
Published: Wed Sep 10 14:09:52 2025 by llama3.2 3B Q4_K_M
