IBM has inked a $240 million agreement with Together AI to deliver a new cloud service aimed at running large-capacity, open-source AI inference workloads. The service will be built on a significant cluster of Nvidia HGX B300 systems interconnected with Nvidia Spectrum-X Ethernet networking and hosted within IBM Cloud. Together AI will use that cluster to provide open-source model inference services tailored for enterprise customers, marking another step in the rapid evolution of AI infrastructure offerings.
The deal is notable because it combines the scale and reliability of IBM Cloud with the specialized GPU-based neocloud capabilities of Together AI. Together AI operates GPU infrastructure and AI services for developers and enterprises, with a platform that supports open models such as DeepSeek, Nemotron, MiniMax, Kimi, and GLM. According to the company, those open models give developers and customers greater freedom to customize and fine-tune AI systems at a lower cost than relying on a single proprietary model for their applications.
A hybrid environment built for enterprise AI
IBM says a hybrid environment on IBM Cloud powered by Nvidia GPUs connected with Nvidia Spectrum-X Ethernet networking and Together AI's inference platform gives organizations a reliable foundation to build, deploy, and scale AI systems. This architecture is intended to handle increasingly demanding inference tasks, which often require continuous, real-time execution rather than periodic model training. By offering an open-source model stack, the service aims to address enterprise concerns about vendor lock-in, cost predictability, and customization flexibility.
Together AI's CEO, Vipul Ved Prakash, said the collaboration with IBM will help Together AI continue its expansion into the enterprise space and make open-source AI more accessible to developers and businesses. He noted that working alongside IBM and Nvidia gives the company a strong foundation and that the cluster will allow Together AI to bring production-grade inference to more companies faster. He also described the deal as a significant step in pushing open-source AI to become the obvious choice for enterprises.
Open models and the enterprise shift toward inference
The rapid adoption of open-weight and open-source models has reshaped how enterprises approach AI. Instead of building or licensing a single large proprietary model, many organizations are now choosing from a broad ecosystem of open models that can be run, adapted, and fine-tuned on their own infrastructure or in the cloud. Together AI has emerged as a key enabler of this trend by providing GPU capacity, model serving, and developer tools that simplify the deployment of open models.
DeepSeek, Nemotron, MiniMax, Kimi, and GLM are among the models supported by Together AI. These models are increasingly used for tasks such as code generation, natural language understanding, document processing, and domain-specific applications. The ability to fine-tune these models on proprietary data is often a critical requirement for enterprises that need AI systems aligned with their industry and compliance standards. The new IBM-Together AI service is designed to support these use cases with a cloud-native infrastructure that can scale up when demand spikes and scale down to control costs.
IBM's relationship with Nvidia has also deepened in recent quarters. The two companies have expanded their collaboration to grow the use of GPU-native data analytics, intelligent document processing, and on-premises as well as regulated infrastructure deployments. Those efforts are intended to help enterprise customers move AI from pilot projects to production workloads. The latest deal with Together AI reinforces IBM Cloud's status as an environment where advanced Nvidia accelerators and networking can be combined with flexible AI software platforms.
The rapidly expanding AI IaaS market
The deal is part of a broader wave of AI-optimized infrastructure-as-a-service activity. IBM Cloud now competes with AWS, Google Cloud, Microsoft Azure, CoreWeave, Lambda, and many other providers in one of the fastest-growing segments of the cloud market. AI-optimized IaaS refers to infrastructure that is specifically designed to support AI training, fine-tuning, and inference workloads, often using specialized accelerators and high-speed networking.
Gartner just this week forecast that worldwide spending on AI-optimized IaaS is projected to grow 96% through 2026 and reach $42 billion. The market is expected to sustain high growth and hit $66 billion in 2027. That acceleration is driven by several factors, including the expansion of generative AI, the proliferation of agents and intelligent assistants, and the increasing complexity of AI workloads running in production environments.
One of the most important findings from Gartner's forecast is the growing share of inference workloads within total AI spending. Inference, the process of running a trained model to generate predictions or responses, is expected to overtake training in terms of cloud infrastructure spending in 2026. Gartner projects that global spending on inference will reach $23.3 billion in 2026, surpassing the $19 billion projected for training. By 2027, 59% of AI-optimized IaaS spending is expected to support inference, compared with 55% in 2026.
Agentic AI and the changing compute landscape
Gartner analysts point to the rise of agentic AI as a key factor reshaping cloud investment priorities. Agentic AI refers to AI systems that can perform multistep, autonomous tasks by reasoning, planning, and interacting with tools and other systems. These agents require continuous compute resources to execute each step, making inference the dominant consumption model. As a result, AI-optimized IaaS is becoming a critical enabler for enterprise AI strategies that rely on large numbers of AI agents.
Hardeep Singh, senior principal research analyst at Gartner, said the shift from model development to production-scale deployment is pushing organizations to integrate fine-tuned and domain-specific models into customer-facing and operational systems. Those systems require continuous, real-time execution rather than periodic training. Singh noted that this shift is accelerating cloud consumption patterns and creating sustained demand for AI-optimized infrastructure.
Enterprises are increasingly deploying domain-specific models that are trained or fine-tuned to address particular industries, regulatory environments, and business processes. These models often operate behind APIs, inside customer service systems, or as part of internal knowledge management platforms. The ability to serve these models at scale with low latency is essential for a positive user experience. The partnership between IBM and Together AI is intended to meet that need by providing a resilient, high-performance cluster in the cloud.
What this deal means for cloud infrastructure
For IBM, the deal strengthens its position in the booming AI infrastructure market and demonstrates that its cloud platform can support massive, GPU-intensive deployments. IBM Cloud has long focused on enterprise workloads, security, and hybrid cloud architectures. By partnering with Together AI, IBM can offer customers access to cutting-edge Nvidia hardware and a modern AI inference stack without requiring them to build and manage their own GPU clusters.
For Together AI, the agreement provides immediate capacity and a strategic partnership that can expand its enterprise sales reach. The company previously relied on its own cloud and colocation partnerships to deliver GPU services, and the IBM Cloud relationship gives it additional geographic and enterprise credibility. Having access to a large Nvidia HGX B300 cluster with Spectrum-X networking will also allow Together AI to run some of the most demanding open models with high throughput and reliability.
Nvidia's HGX B300 systems are among the most powerful AI computing platforms currently available. They are designed to handle both training and inference across large AI models, with memory bandwidth and compute density optimized for modern deep learning workloads. Spectrum-X Ethernet networking adds intelligent congestion control and isolation features that are especially valuable in multi-tenant AI cloud environments. This combination of high-performance accelerators and AI-specific networking is becoming a key differentiator for cloud providers that want to attract AI workloads.
Open-source AI has become a major force in the industry because it gives enterprises transparency, control, and flexibility that proprietary models often cannot provide. However, running open-source models in production still requires significant infrastructure and technical expertise. Many enterprises lack the ability to procure GPUs, configure high-speed networking, and manage the lifecycle of AI workloads. Services like the one being built by IBM and Together AI lower those barriers by packaging open-source models with managed AI infrastructure.
The announcement also underscores the changing nature of the AI cloud market. In the early days of generative AI, most spending focused on training increasingly large foundation models. Today, more attention is shifting to inference and the systems that support live AI applications. As organizations move from proof-of-concept to deployment, they are discovering that inference can be far more expensive and operationally demanding than originally anticipated. High availability, cost predictability, and data privacy all become critical concerns when AI is embedded in business workflows.
Gartner's projections for AI-optimized IaaS spending reflect this evolution. Cloud providers that can offer both training and inference capabilities at scale will be best positioned to capture the next wave of enterprise AI spending. The IBM-Together AI collaboration is an example of how cloud providers are using asset-light, partnership-driven models to expand capacity quickly. Rather than building a GPU business from scratch, IBM is leveraging Together AI's software stack while Together AI gains access to IBM Cloud's enterprise channels.
The competitive landscape for AI IaaS is crowded, and differentiation will be crucial. AWS and Google Cloud have their own custom TPUs and AI services. Microsoft Azure has a deep partnership with OpenAI and a strong portfolio of AI tools. CoreWeave and Lambda have become popular among AI startups because they offer dense GPU clusters with minimal enterprise overhead. IBM Cloud's partnership with Nvidia is not new, but the addition of Together AI could give it a distinctive advantage for open-source inference workloads. The service is expected to appeal to enterprises that want to run models like DeepSeek or Nemotron without making the huge capital investment required to build an internal GPU data center.
The broader industry consensus is that AI infrastructure spending will remain robust for the foreseeable future. As inference becomes the dominant workload, workloads will become more continuous and less batch-oriented. That shift has profound implications for energy consumption, data center design, and cloud pricing. Enterprises will demand more efficient ways to serve models, which could lead to more innovation in model quantization, pruning, and hardware acceleration.
IBM and Together AI have not yet disclosed when the new service will become generally available or how the pricing will be structured. Nevertheless, the formal announcement is a strong signal that both companies see open-source inference as a long-term opportunity. By combining a neocloud provider with an established enterprise cloud, the deal could serve as a template for future infrastructure partnerships in the AI economy.
Source: Network World News