Get the latest tech news

Amazon HyperPod Task Governance keeps GPUs running, cutting costs 40%

HyperPod Task Governance can increase GPU utilization, helping enterprises to optimize AI costs and producing potentially significant savings.

At the AWS:reinvent 2024 conference today, the cloud giant announced HyperPod Task Governance, a sophisticated solution targeting one of the most expensive inefficiencies in enterprise AI operations: underutilized GPU resources. Despite heavy investments in GPU infrastructure to power various AI workloads, including training, fine tuning and inference, these expensive computing resources frequently sit idle. “Internally, we had this kind of problem as we were scaling up more than a year ago, and we built a system that takes into account the consumption needs of these accelerators,” Sivasubramanian told VentureBeat.

Get the Android app