ERNIE 3.0 Titan Description

Pre-trained models of language have achieved state-of the-art results for various Natural Language Processing (NLP). GPT-3 has demonstrated that scaling up language models pre-trained can further exploit their immense potential. Recently, a framework named ERNIE 3.0 for pre-training large knowledge enhanced models was proposed. This framework trained a model that had 10 billion parameters. ERNIE 3.0 performed better than the current state-of-the art models on a variety of NLP tasks. In order to explore the performance of scaling up ERNIE 3.0, we train a hundred-billion-parameter model called ERNIE 3.0 Titan with up to 260 billion parameters on the PaddlePaddle platform. We also design a self supervised adversarial and a controllable model language loss to make ERNIE Titan generate credible texts.

Integrations

Reviews

Total
ease
features
design
support

No User Reviews. Be the first to provide a review:

Write a Review

Company Details

Company:
Baidu
Year Founded:
2000
Headquarters:
China
Website:
research.baidu.com/Public/uploads/61c4362c79ee8.pdf

Media

ERNIE 3.0 Titan Screenshot 1
Recommended Products
Top Rated Business VoIP Provider for 2024 for as low as $20/mo*! Icon
Top Rated Business VoIP Provider for 2024 for as low as $20/mo*!

Message, video, and phone on any device. Trusted by over 400,000 businesses.

- Includes 100+ Premium Features
- Unlimited Calling, Faxing, SMS, Conferencing.

Product Details

Platforms
SaaS
Windows
Mac
Linux
On-Premises
Type of Training
Documentation

ERNIE 3.0 Titan Features and Options