Average Ratings 0 Ratings

Total
ease
features
design
support

No User Reviews. Be the first to provide a review:

Write a Review

Average Ratings 0 Ratings

Total
ease
features
design
support

No User Reviews. Be the first to provide a review:

Write a Review

Description

Baichuan-13B is an advanced large-scale language model developed by Baichuan Intelligent, featuring 13 billion parameters and available for open-source and commercial use, building upon its predecessor Baichuan-7B. This model has set new records for performance among similarly sized models on esteemed Chinese and English evaluation metrics. The release includes two distinct pre-training variations: Baichuan-13B-Base and Baichuan-13B-Chat. By significantly increasing the parameter count to 13 billion, Baichuan-13B enhances its capabilities, training on 1.4 trillion tokens from a high-quality dataset, which surpasses LLaMA-13B's training data by 40%. It currently holds the distinction of being the model with the most extensive training data in the 13B category, providing robust support for both Chinese and English languages, utilizing ALiBi positional encoding, and accommodating a context window of 4096 tokens for improved comprehension and generation. This makes it a powerful tool for a variety of applications in natural language processing.

Description

This repository showcases an implementation of model parallel autoregressive transformers utilizing GPUs, leveraging the capabilities of the DeepSpeed library. It serves as a record of EleutherAI's framework designed for training extensive language models on GPU architecture. Currently, it builds upon NVIDIA's Megatron Language Model, enhanced with advanced techniques from DeepSpeed alongside innovative optimizations. Our goal is to create a centralized hub for aggregating methodologies related to the training of large-scale autoregressive language models, thereby fostering accelerated research and development in the field of large-scale training. We believe that by providing these resources, we can significantly contribute to the progress of language model research.

API Access

Has API

API Access

Has API

Screenshots View All

Screenshots View All

Integrations

APIPark
C
C#
C++
CSS
Clojure
Elixir
F#
Forefront
HTML
Java
Julia
Kotlin
Python
R
Ruby
Rust
Scala
Visual Basic
ZBrain

Integrations

APIPark
C
C#
C++
CSS
Clojure
Elixir
F#
Forefront
HTML
Java
Julia
Kotlin
Python
R
Ruby
Rust
Scala
Visual Basic
ZBrain

Pricing Details

Free
Free Trial
Free Version

Pricing Details

Free
Free Trial
Free Version

Deployment

Web-Based
On-Premises
iPhone App
iPad App
Android App
Windows
Mac
Linux
Chromebook

Deployment

Web-Based
On-Premises
iPhone App
iPad App
Android App
Windows
Mac
Linux
Chromebook

Customer Support

Business Hours
Live Rep (24/7)
Online Support

Customer Support

Business Hours
Live Rep (24/7)
Online Support

Types of Training

Training Docs
Webinars
Live Training (Online)
In Person

Types of Training

Training Docs
Webinars
Live Training (Online)
In Person

Vendor Details

Company Name

Baichuan Intelligent Technology

Founded

1998

Country

China

Website

github.com/baichuan-inc/Baichuan-13B

Vendor Details

Company Name

EleutherAI

Founded

2020

Website

github.com/EleutherAI/gpt-neox

Product Features

Product Features

Alternatives

Mistral 7B Reviews

Mistral 7B

Mistral AI

Alternatives

OPT Reviews

OPT

Meta
ChatGLM Reviews

ChatGLM

Zhipu AI
GPT-J Reviews

GPT-J

EleutherAI
Llama 2 Reviews

Llama 2

Meta
DeepSpeed Reviews

DeepSpeed

Microsoft
Qwen-7B Reviews

Qwen-7B

Alibaba
Pythia Reviews

Pythia

EleutherAI