LongLLaMA Description
This repository contains a research preview of LongLLaMA. It is a large language-model capable of handling contexts up to 256k tokens. LongLLaMA was built on the foundation of OpenLLaMA, and fine-tuned with the Focused Transformer method. LongLLaMA code was built on the foundation of Code Llama. We release a smaller base variant of the LongLLaMA (not instruction-tuned) on a permissive licence (Apache 2.0), and inference code that supports longer contexts for hugging face. Our model weights are a drop-in replacement for LLaMA (for short contexts up to 2048 tokens) in existing implementations. We also provide evaluation results, and comparisons with the original OpenLLaMA model.
Pricing
Pricing Starts At:
Free
Free Version:
Yes
Integrations
No Integrations at this time
Company Details
Company:
LongLLaMA
Website:
github.com/CStanKonrad/long_llama
Media
Recommended Products
Take the guesswork out of password management, shadow IT, infrastructure, and secret sharing so you can keep your people safe and your business moving.
Product Details
Platforms
SaaS
On-Premises
Type of Training
Documentation
Customer Support
Online