Average Ratings 0 Ratings
Average Ratings 0 Ratings
Description
This repository showcases an implementation of model parallel autoregressive transformers utilizing GPUs, leveraging the capabilities of the DeepSpeed library. It serves as a record of EleutherAI's framework designed for training extensive language models on GPU architecture. Currently, it builds upon NVIDIA's Megatron Language Model, enhanced with advanced techniques from DeepSpeed alongside innovative optimizations. Our goal is to create a centralized hub for aggregating methodologies related to the training of large-scale autoregressive language models, thereby fostering accelerated research and development in the field of large-scale training. We believe that by providing these resources, we can significantly contribute to the progress of language model research.
Description
Ling 2.6 represents an independently developed and open-source series of large language models created by Ant Group, utilizing a Mixture of Experts (MoE) architecture to enhance inference efficiency, long context modeling, training methodologies, and collaborative reasoning for AI agents. By employing this MoE architecture, Ling effectively directs each token to engage only the most pertinent expert subnetworks, significantly reducing the computational load while preserving the extensive capabilities of the model. This series makes strides in long-sequence modeling, exemplified by Ling-2.6-1T, which accommodates a native context window of up to 1 million tokens and offers a 256K context window through its official API; additionally, Ling-2.6-flash features a native 256K context window, enabling it to handle around 200,000 characters in lengthy inputs. These models are meticulously crafted to ensure dependable retrieval of long-range information without any discernible loss of quality, regardless of whether the data is located at the start, middle, or end of the context. This innovative approach to long-context processing sets a new benchmark for efficiency and reliability in language model performance.
API Access
Has API
API Access
Has API
Integrations
Claude Code
Forefront
Hermes Agent
Kilo Code
OpenAI
OpenClaw
OpenRouter
ZBrain
Integrations
Claude Code
Forefront
Hermes Agent
Kilo Code
OpenAI
OpenClaw
OpenRouter
ZBrain
Pricing Details
Free
Free Trial
Free Version
Pricing Details
$0.0028 per 1M tokens
Free Trial
Free Version
Deployment
Web-Based
On-Premises
iPhone App
iPad App
Android App
Windows
Mac
Linux
Chromebook
Deployment
Web-Based
On-Premises
iPhone App
iPad App
Android App
Windows
Mac
Linux
Chromebook
Customer Support
Business Hours
Live Rep (24/7)
Online Support
Customer Support
Business Hours
Live Rep (24/7)
Online Support
Types of Training
Training Docs
Webinars
Live Training (Online)
In Person
Types of Training
Training Docs
Webinars
Live Training (Online)
In Person
Vendor Details
Company Name
EleutherAI
Founded
2020
Website
github.com/EleutherAI/gpt-neox
Vendor Details
Company Name
Ant Group
Founded
2014
Country
China
Website
developer.ant-ling.com/en/docs/models/ling/