TinyLlama Description

The TinyLlama Project aims to pretrain an 1.1B Llama on 3 trillion tokens. We can achieve this in "just" 90 day using 16 A100-40G graphics cards with some optimization. We used the exact same architecture and tokenizers as Llama 2 TinyLlama is compatible with many open-source Llama projects. TinyLlama has only 1.1B of parameters. This compactness allows TinyLlama to be used for a variety of applications that require a small computation and memory footprint.

Pricing

Pricing Starts At:
Free
Pricing Information:
Open source
Free Version:
Yes

Integrations

No Integrations at this time

Reviews

Total
ease
features
design
support

No User Reviews. Be the first to provide a review:

Write a Review

Company Details

Company:
TinyLlama
Website:
github.com/jzhang38/TinyLlama

Media

Get Started
Recommended Products
Manage your IT department more effectively Icon
Manage your IT department more effectively

Streamline your business from end to end with ConnectWise PSA

ConnectWise PSA (formerly Manage) allows you to stop working in separate systems, and helps you build a more profitable business. No more duplicate data entries, inefficient employees, manual invoices, and the inability to accurately track client service issues. Get a behind the scenes look into the award-winning PSA that automates processes for each area of business: sales, help desk, support, finance, and HR.
Learn More

Product Details

Platforms
Windows
Mac
Linux
Type of Training
Documentation

TinyLlama Features and Options

TinyLlama Lists

TinyLlama User Reviews

Write a Review
  • Previous
  • Next