New AWS Service Lets Customers Rent Nvidia GPUs For Quick AI Projects 7
An anonymous reader quotes a report from TechCrunch: More and more companies are running large language models, which require access to GPUs. The most popular of those by far are from Nvidia, making them expensive and often in short supply. Renting a long-term instance from a cloud provider when you only need access to these costly resources for a single job, doesn't necessarily make sense. To help solve that problem, AWS launched Amazon Elastic Compute Cloud (EC2) Capacity Blocks for ML today, enabling customers to buy access to these GPUs for a defined amount of time, typically to run some sort of AI-related job such as training a machine learning model or running an experiment with an existing model.
The product gives customers access to NVIDIA H100 Tensor Core GPUs instances in cluster sizes of one to 64 instances with 8 GPUs per instance. They can reserve time for up to 14 days in 1-day increments, up to 8 weeks in advance. When the timeframe is over, the instances will shut down automatically. The new product enables users to sign up for a the number of instances they need for defined block of time, just like reserving a hotel room for a certain number of days (as the company put it). From the customer's perspective, they will know exactly how long the job will run, how many GPUs they'll use and how much it will cost up front, giving them cost certainty. As a users sign up for the service, its displays the total cost for the timeframe and resources. Users can dial that up or down, depending on their resource appetite and budgets before agreeing to buy. The new feature is generally available starting today in the AWS US East (Ohio) region.
The product gives customers access to NVIDIA H100 Tensor Core GPUs instances in cluster sizes of one to 64 instances with 8 GPUs per instance. They can reserve time for up to 14 days in 1-day increments, up to 8 weeks in advance. When the timeframe is over, the instances will shut down automatically. The new product enables users to sign up for a the number of instances they need for defined block of time, just like reserving a hotel room for a certain number of days (as the company put it). From the customer's perspective, they will know exactly how long the job will run, how many GPUs they'll use and how much it will cost up front, giving them cost certainty. As a users sign up for the service, its displays the total cost for the timeframe and resources. Users can dial that up or down, depending on their resource appetite and budgets before agreeing to buy. The new feature is generally available starting today in the AWS US East (Ohio) region.
so at 8K how meny fps can city skylines 2 get at (Score:2)
so at 8K how many fps can city skylines 2 get on this?
batch processing (Score:3)
and thus doth batch processing return to commercial computing!
for anything without a serious real-time element, I would expect these reservations to quickly yield way to queues.
hawk
Re: (Score:2)
Queues actually make more sense.
> When the timeframe is over, the instances will shut down automatically.
Nothing like being 97% through a 2 day job when it shuts down from a hard reservation.
Re: (Score:2)
the new part is just a reservation system. (Score:2)
The whole article seems to be written by someone not familiar with cloud instance pricing. The default usage mode for cloud instances is already just on demand. , not long term lease. The "new" thing is some new scheduling / pricing (?) offering in this slightly tweaked reservation system.
You can already use these machines, "p5.48xlarge" (https://aws.amazon.com/ec2/instance-types/p5/ ), for $95/hour whenever you want, unless they're unavailable for some reason in your zone at that time. just like other