ERNIE 3.0 Titan Description

Pre-trained models of language have achieved state-of the-art results for various Natural Language Processing (NLP). GPT-3 has demonstrated that scaling up language models pre-trained can further exploit their immense potential. Recently, a framework named ERNIE 3.0 for pre-training large knowledge enhanced models was proposed. This framework trained a model that had 10 billion parameters. ERNIE 3.0 performed better than the current state-of-the art models on a variety of NLP tasks. In order to explore the performance of scaling up ERNIE 3.0, we train a hundred-billion-parameter model called ERNIE 3.0 Titan with up to 260 billion parameters on the PaddlePaddle platform. We also design a self supervised adversarial and a controllable model language loss to make ERNIE Titan generate credible texts.

Integrations

Reviews

Total
ease
features
design
support

No User Reviews. Be the first to provide a review:

Write a Review

Company Details

Company:
Baidu
Year Founded:
2000
Headquarters:
China
Website:
research.baidu.com/Public/uploads/61c4362c79ee8.pdf

Media

ERNIE 3.0 Titan Screenshot 1
Recommended Products
Secure your business by securing your people. Icon
Secure your business by securing your people.

Over 100,000 businesses trust 1Password

Take the guesswork out of password management, shadow IT, infrastructure, and secret sharing so you can keep your people safe and your business moving.

Product Details

Platforms
SaaS
Windows
Mac
Linux
On-Premises
Type of Training
Documentation

ERNIE 3.0 Titan Features and Options