Amazon Elastic Inference Description
Amazon Elastic Inference allows for low-cost GPU-powered acceleration to Amazon EC2 instances and Sagemaker instances, or Amazon ECS tasks. This can reduce the cost of deep learning inference by up 75%. Amazon Elastic Inference supports TensorFlow and Apache MXNet models. Inference is the process by which a trained model makes predictions. Inference can account for as much as 90% of total operational expenses in deep learning applications for two reasons. First, standalone GPU instances are usually used for model training and not inference. Inference jobs typically process one input at a time and use a smaller amount of GPU compute. Training jobs can process hundreds of data samples simultaneously, but inference jobs only process one input in real-time. This makes standalone GPU-based inference expensive. However, standalone CPU instances aren't specialized for matrix operations and are therefore often too slow to perform deep learning inference.
Amazon Elastic Inference Alternatives
AWS Inferentia
AWS Inferentia Accelerators are designed by AWS for high performance and low cost for deep learning (DL), inference applications. The first-generation AWS Inferentia accelerator powers Amazon Elastic Compute Cloud, Amazon EC2 Inf1 instances. These instances deliver up to 2.3x more throughput and up 70% lower cost per input than comparable GPU-based Amazon EC2 instances. Inf1 instances have been adopted by many customers including Snap, Sprinklr and Money Forward. They have seen the performance and cost savings. The first-generation Inferentia features 8 GB of DDR4 memory per accelerator, as well as a large amount on-chip memory. Inferentia2 has 32 GB of HBM2e, which increases the total memory by 4x and memory bandwidth 10x more than Inferentia.
Learn more
Amazon EC2
Amazon Elastic Compute Cloud (Amazon EC2) provides secure, resizable cloud computing capacity. It was designed to make cloud computing at web scale easier for developers. Amazon EC2's web service interface makes it easy to configure and obtain capacity with minimal effort. It gives you complete control over your computing resources and allows you to run on Amazon's proven computing environment.
Learn more
UberCloud
Simr (formerly UberCloud) is revolutionizing the world of simulation operations with our flagship solution, Simulation Operations Automation (SimOps). Designed to streamline and automate complex simulation workflows, Simr enhances productivity, collaboration, and efficiency for engineers and scientists across various industries, including automotive, aerospace, biomedical engineering, defense, and consumer electronics.
Our cloud-based infrastructure provides scalable and cost-effective solutions, eliminating the need for significant upfront investments in hardware. This ensures that our clients have access to the computational power they need, exactly when they need it, leading to reduced costs and improved operational efficiency.
Simr is trusted by some of the world's leading companies, including three of the seven most successful companies globally. One of our notable success stories is BorgWarner, a Tier 1 automotive supplier that leverages Simr to automate its simulation environments, significantly enhancing their efficiency and driving innovation.
Learn more
Amazon EC2 Inf1 Instances
Amazon EC2 Inf1 instances were designed to deliver high-performance, cost-effective machine-learning inference. Amazon EC2 Inf1 instances offer up to 2.3x higher throughput, and up to 70% less cost per inference compared with other Amazon EC2 instance. Inf1 instances are powered by up to 16 AWS inference accelerators, designed by AWS. They also feature Intel Xeon Scalable 2nd generation processors, and up to 100 Gbps of networking bandwidth, to support large-scale ML apps. These instances are perfect for deploying applications like search engines, recommendation system, computer vision and speech recognition, natural-language processing, personalization and fraud detection. Developers can deploy ML models to Inf1 instances by using the AWS Neuron SDK. This SDK integrates with popular ML Frameworks such as TensorFlow PyTorch and Apache MXNet.
Learn more
Integrations
Company Details
Company:
Amazon
Year Founded:
2006
Headquarters:
United States
Website:
aws.amazon.com/machine-learning/elastic-inference/
Media
Recommended Products
Our Free Plans just got better! | Auth0 by Okta
You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your secuirty. Auth0 now, thank yourself later.
Product Details
Platforms
SaaS
Type of Training
Documentation
Customer Support
Online
Amazon Elastic Inference Features and Options
Amazon Elastic Inference User Reviews
Write a Review- Previous
- Next
