Alternatives to PaLM 2

Megatron-Turing Natural Language Generation Model (MT-NLG) is the largest and most powerful monolithic English language model. It has 530 billion parameters. This 105-layer transformer-based MTNLG improves on the previous state-of-the art models in zero, one, and few shot settings. It is unmatched in its accuracy across a wide range of natural language tasks, including Completion prediction and Reading comprehension. NVIDIA has announced an Early Access Program for its managed API service in MT-NLG Mode. This program will allow customers to experiment with, employ and apply a large language models on downstream language tasks.

Gemini is Google’s advanced AI chatbot that engages in natural language conversation to boost creativity and productivity. Gemini is accessible via web and mobile apps. It integrates seamlessly with Google services such as Docs, Drive and Gmail. Users can draft content, summarize data, and manage tasks. Its multimodal capabilities enable it to process and produce diverse data types such as text images and audio. This provides comprehensive assistance in different contexts. Gemini is a constantly learning model that adapts to the user's interactions and offers personalized and context-aware answers to meet a variety of user needs.

Gemini models are a family of light-open, state-of-the art models that was created using the same research and technology as Gemini models. These models include comprehensive security measures, and help to ensure responsible and reliable AI through selected data sets. Gemma models have exceptional comparative results, even surpassing some larger open models, in their 2B and 7B sizes. Keras 3.0 offers seamless compatibility with JAX TensorFlow PyTorch and JAX. Gemma 2 has been redesigned to deliver unmatched performance and efficiency. It is optimized for inference on a variety of hardware. The Gemma models are available in a variety of models that can be customized to meet your specific needs. The Gemma models consist of large text-to text lightweight language models that have a decoder and are trained on a large set of text, code, or mathematical content.

Llama 3.3, the latest in the Llama language model series, was developed to push the limits of AI-powered communication and understanding. Llama 3.3, with its enhanced contextual reasoning, improved generation of language, and advanced fine tuning capabilities, is designed to deliver highly accurate responses across diverse applications. This version has a larger dataset for training, refined algorithms to improve nuanced understanding, and reduced biases as compared to previous versions. Llama 3.3 excels at tasks such as multilingual communication, technical explanations, creative writing and natural language understanding. It is an indispensable tool for researchers, developers and businesses. Its modular architecture enables customization in specialized domains and ensures performance at scale.

XLNet, a new unsupervised language representation method, is based on a novel generalized Permutation Language Modeling Objective. XLNet uses Transformer-XL as its backbone model. This model is excellent for language tasks that require long context. Overall, XLNet achieves state of the art (SOTA) results in various downstream language tasks, including question answering, natural languages inference, sentiment analysis and document ranking.

Gemini Flash, a large language model from Google, is specifically designed for low-latency, high-speed language processing tasks. Gemini Flash, part of Google DeepMind’s Gemini series is designed to handle large-scale applications and provide real-time answers. It's ideal for interactive AI experiences such as virtual assistants, live chat, and customer support. Gemini Flash is built on sophisticated neural structures that ensure contextual relevance, coherence, and precision. Google has built in rigorous ethical frameworks as well as responsible AI practices to Gemini Flash. It also equipped it with guardrails that manage and mitigate biased outcomes, ensuring alignment with Google's standards of safe and inclusive AI. Google's Gemini Flash empowers businesses and developers with intelligent, responsive language tools that can keep up with fast-paced environments.

Gemma is the family of lightweight open models that are built using the same research and technology as the Gemini models. Gemma was developed by Google DeepMind, along with other teams within Google. The name is derived from the Latin gemma meaning "precious stones". We're also releasing new tools to encourage developer innovation, encourage collaboration, and guide responsible use of Gemma model. Gemma models are based on the same infrastructure and technical components as Gemini, Google's largest and most powerful AI model. Gemma 2B, 7B and other open models can achieve the best performance possible for their size. Gemma models can run directly on a desktop or laptop computer for developers. Gemma is able to surpass much larger models in key benchmarks, while adhering our rigorous standards of safe and responsible outputs.

Through scientific rigor and human insight, healthcare breakthroughs can change the world, bringing hope to humanity. We believe that AI can help in this area, through collaboration between researchers, healthcare organisations, and the wider ecosystem. Today, we are sharing exciting progress in these initiatives with the announcement that Google's large language model (LLM) for medical applications, called Med PaLM 2, will be available to a limited number of customers. In the coming weeks, it will be available to a small group of Google Cloud users for limited testing. We will explore use cases, share feedback, and investigate safe, responsible and meaningful ways to utilize this technology. Med-PaLM 2, which harnesses Google's LLMs aligned with the medical domain, is able to answer medical questions more accurately and safely. Med-PaLM 2 is the first LLM that has performed at an "expert" level on the MedQA dataset consisting of US Medical Licensing Examination-style questions.

Aya is an open-source, state-of-the art, massively multilingual large language research model (LLM), which covers 101 different languages. This is more than twice the number of languages that are covered by open-source models. Aya helps researchers unlock LLMs' powerful potential for dozens of cultures and languages that are largely ignored by the most advanced models available today. We open-source both the Aya Model, as well as the most comprehensive multilingual instruction dataset with 513 million words covering 114 different languages. This data collection contains rare annotations by native and fluent speakers from around the world. This ensures that AI technology is able to effectively serve a global audience who have had limited access up until now.

Ai2 OLMoE, an open-source mixture-of experts language model, can run completely on the device. This allows you to test our model in a private and secure environment. Our app is designed to help researchers explore ways to improve on-device intelligence and to allow developers to quickly prototype AI experiences. All without cloud connectivity. OLMoE is the highly efficient mix-of-experts model of the Ai2 OLMo models. Discover what real-world tasks are possible with state-of-the art local models. Learn how to improve AI models for small systems. You can test your own models using our open-source codebase. Integrate OLMoE with other iOS applications. The Ai2 OLMoE application provides privacy and security because it operates entirely on the device. Share the output of your conversation with friends and colleagues. The OLMoE application code and model are both open source.

DeepSeek-V3 is an advanced AI model built to excel in natural language comprehension, sophisticated reasoning, and decision-making across a wide range of applications. Harnessing innovative neural architectures and vast datasets, it offers exceptional capabilities for addressing complex challenges in fields like research, development, business analytics, and automation. Designed for both scalability and efficiency, DeepSeek-V3 empowers developers and organizations to drive innovation and unlock new possibilities with state-of-the-art AI solutions.

Code Llama, a large-language model (LLM), can generate code using text prompts. Code Llama, the most advanced publicly available LLM for code tasks, has the potential to improve workflows for developers and reduce the barrier for those learning to code. Code Llama can be used to improve productivity and educate programmers to create more robust, well documented software. Code Llama, a state-of the-art LLM, is capable of generating both code, and natural languages about code, based on both code and natural-language prompts. Code Llama can be used for free in research and commercial purposes. Code Llama is a new model that is built on Llama 2. It is available in 3 models: Code Llama is the foundational model of code; Codel Llama is a Python-specific language. Code Llama-Instruct is a finely tuned natural language instruction interpreter.

CodeGemma consists of powerful lightweight models that are capable of performing a variety coding tasks, including fill-in the middle code completion, code creation, natural language understanding and mathematical reasoning. CodeGemma offers 3 variants: a 7B model that is pre-trained to perform code completion, code generation, and natural language-to code chat. A 7B model that is instruction-tuned for instruction following and natural language-to code chat. You can complete lines, functions, or even entire blocks of code whether you are working locally or with Google Cloud resources. CodeGemma models are trained on 500 billion tokens primarily of English language data taken from web documents, mathematics and code. They generate code that is not only syntactically accurate but also semantically meaningful. This reduces errors and debugging times.

DeepSeek-V2, developed by DeepSeek-AI, is a cutting-edge Mixture-of-Experts (MoE) language model designed for cost-effective training and high-speed inference. Boasting a massive 236 billion parameters—though only 21 billion are active per token—it efficiently handles a context length of up to 128K tokens. The model leverages advanced architectural innovations such as Multi-head Latent Attention (MLA) to optimize inference by compressing the Key-Value (KV) cache and DeepSeekMoE to enable economical training via sparse computation. Compared to its predecessor, DeepSeek 67B, it slashes training costs by 42.5%, shrinks the KV cache by 93.3%, and boosts generation throughput by 5.76 times. Trained on a vast 8.1 trillion token dataset, DeepSeek-V2 excels in natural language understanding, programming, and complex reasoning, positioning itself as a premier choice in the open-source AI landscape.

Amazon Nova is the new generation of foundation models (FMs), which are state-of-the art (SOTA), and offer industry-leading price-performance. They are available exclusively through Amazon Bedrock. Amazon Nova Micro and Amazon Nova Lite are understanding models which accept text, images, or videos as inputs and produce text output. They offer a wide range of capabilities, accuracy, speed and cost operation points. Amazon Nova Micro, a text-only model, delivers the lowest latency at a very low price. Amazon Nova Lite, a multimodal model with a low cost, is lightning-fast at processing text, image, and video inputs. Amazon Nova Pro is an extremely capable multimodal model that offers the best combination of accuracy and speed for a variety of tasks. Amazon Nova Pro is a powerful model that can handle almost any task. Its speed and cost efficiency are industry-leading.

Llama (Large Language Model meta AI) is a state of the art foundational large language model that was created to aid researchers in this subfield. Llama allows researchers to use smaller, more efficient models to study these models. This further democratizes access to this rapidly-changing field. Because it takes far less computing power and resources than large language models, such as Llama, to test new approaches, validate other's work, and explore new uses, training smaller foundation models like Llama can be a desirable option. Foundation models are trained on large amounts of unlabeled data. This makes them perfect for fine-tuning for many tasks. We make Llama available in several sizes (7B-13B, 33B and 65B parameters), and also share a Llama card that explains how the model was built in line with our Responsible AI practices.

Chinchilla has a large language. Chinchilla has the same compute budget of Gopher, but 70B more parameters and 4x as much data. Chinchilla consistently and significantly outperforms Gopher 280B, GPT-3 175B, Jurassic-1 178B, and Megatron-Turing (530B) in a wide range of downstream evaluation tasks. Chinchilla also uses less compute to perform fine-tuning, inference and other tasks. This makes it easier for downstream users to use. Chinchilla reaches a high-level average accuracy of 67.5% for the MMLU benchmark. This is a greater than 7% improvement compared to Gopher.

The Gemini 1.5 Pro AI Model is a state of the art language model that delivers highly accurate, context aware, and human like responses across a wide range of applications. It excels at natural language understanding, generation and reasoning tasks. The model has been fine-tuned to support tasks such as content creation, code-generation, data analysis, or complex problem-solving. Its advanced algorithms allow it to adapt seamlessly to different domains, conversational styles and languages. The Gemini 1.5 Pro, with its focus on scalability, is designed for both small-scale and enterprise-level implementations. It is a powerful tool to enhance productivity and innovation.

Pre-trained models of language have achieved state-of the-art results for various Natural Language Processing (NLP). GPT-3 has demonstrated that scaling up language models pre-trained can further exploit their immense potential. Recently, a framework named ERNIE 3.0 for pre-training large knowledge enhanced models was proposed. This framework trained a model that had 10 billion parameters. ERNIE 3.0 performed better than the current state-of-the art models on a variety of NLP tasks. In order to explore the performance of scaling up ERNIE 3.0, we train a hundred-billion-parameter model called ERNIE 3.0 Titan with up to 260 billion parameters on the PaddlePaddle platform. We also design a self supervised adversarial and a controllable model language loss to make ERNIE Titan generate credible texts.

Phi-4 is the latest small language model (SLM), with 14B parameters. It excels in complex reasoning, including math, as well as conventional language processing. Phi-4, the latest member of the Phi family of SLMs, demonstrates what is possible as we continue exploring the boundaries of SLMs. Phi-4 will be available in Hugging Face and Azure AI Foundry, under a Microsoft Research License Agreement. Phi-4 is superior to comparable and larger models in math-related reasoning thanks to improvements throughout the process, including the use high-quality synthetic data, curation of organic data of high quality, and innovations post-training. Phi-4 continues pushing the boundaries of size vs. quality.

InstructGPT is an open source framework that trains language models to generate natural language instruction from visual input. It uses a generative, pre-trained transformer model (GPT) and the state of the art object detector Mask R-CNN to detect objects in images. Natural language sentences are then generated that describe the image. InstructGPT has been designed to be useful in all domains including robotics, gaming, and education. It can help robots navigate complex tasks using natural language instructions or it can help students learn by giving descriptive explanations of events or processes.

Phi-2 is a 2.7-billion-parameter language-model that shows outstanding reasoning and language-understanding capabilities. It represents the state-of-the art performance among language-base models with less than thirteen billion parameters. Phi-2 can match or even outperform models 25x larger on complex benchmarks, thanks to innovations in model scaling. Phi-2's compact size makes it an ideal playground for researchers. It can be used for exploring mechanistic interpretationability, safety improvements or fine-tuning experiments on a variety tasks. We have included Phi-2 in the Azure AI Studio catalog to encourage research and development of language models.

GPT-J, a cutting edge language model developed by EleutherAI, is a leading-edge language model. GPT-J's performance is comparable to OpenAI's GPT-3 model on a variety of zero-shot tasks. GPT-J, in particular, has shown that it can surpass GPT-3 at tasks relating to code generation. The latest version of this language model is GPT-J-6B and is built on a linguistic data set called The Pile. This dataset is publically available and contains 825 gibibytes worth of language data organized into 22 subsets. GPT-J has some similarities with ChatGPT. However, GPTJ is not intended to be a chatbot. Its primary function is to predict texts. Databricks made a major development in March 2023 when they introduced Dolly, an Apache-licensed model that follows instructions.

ChatGPT is an OpenAI language model. It can generate human-like responses to a variety prompts, and has been trained on a wide range of internet texts. ChatGPT can be used to perform natural language processing tasks such as conversation, question answering, and text generation. ChatGPT is a pretrained language model that uses deep-learning algorithms to generate text. It was trained using large amounts of text data. This allows it to respond to a wide variety of prompts with human-like ease. It has a transformer architecture that has been proven to be efficient in many NLP tasks. ChatGPT can generate text in addition to answering questions, text classification and language translation. This allows developers to create powerful NLP applications that can do specific tasks more accurately. ChatGPT can also process code and generate it.

We introduce the Granite family of decoder only code models for code generation tasks (e.g. fixing bugs, explaining codes, documenting codes), trained with code in 116 programming language. The Granite Code family has been evaluated on a variety of tasks and demonstrates that the models are consistently at the top of their game among open source code LLMs. Granite Code models have a number of key advantages. Granite Code models are able to perform at a competitive level or even at the cutting edge of technology in a variety of code-related tasks including code generation, explanations, fixing, translation, editing, and more. Demonstrating the ability to solve a variety of coding tasks. IBM's Corporate Legal team guides all models for trustworthy enterprise use. All models are trained using license-permissible datasets collected according to IBM's AI Ethics Principles.

Qwen2-VL, the latest version in the Qwen model family of vision language models, is based on Qwen2. Qwen2-VL is a newer version of Qwen-VL that has: SoTA understanding of images with different resolutions & ratios: Qwen2-VL reaches state-of-the art performance on visual understanding benchmarks including MathVista DocVQA RealWorldQA MTVQA etc. Understanding videos over 20 min: Qwen2-VL is able to understand videos longer than 20 minutes, allowing for high-quality video-based questions, dialogs, content creation, and more. Agent that can control your mobiles, robotics, etc. Qwen2-VL, with its complex reasoning and decision-making abilities, can be integrated into devices such as mobile phones, robots and other devices for automatic operation using visual environment and text instruction. Multilingual Support - To serve users worldwide, Qwen2-VL supports texts in other languages within images, besides English or Chinese.

Cohere is an AI company that provides advanced language models designed to help businesses and developers create intelligent text-based applications. Their models support tasks like text generation, summarization, and semantic search, with options such as the Command family for high-performance applications and Aya Expanse for multilingual capabilities across 23 languages. Cohere emphasizes flexibility and security, offering deployment on cloud platforms, private environments, and on-premises systems. The company partners with major enterprises like Oracle and Salesforce to enhance automation and customer interactions through generative AI. Additionally, its research division, Cohere For AI, contributes to machine learning innovation by fostering global collaboration and open-source advancements.

OpenGPT is a German initiative that focuses on developing large AI languages models tailored to European requirements, with an emphasis on versatility, trustworthiness and multilingual capabilities. It also emphasizes open-source accessibility. The project brings together partners to cover the whole generative AI value-chain, from scalable GPU-based infrastructure to data for training large language model to model design, practical applications, and prototypes and proofs-of concept. OpenGPT-X aims at advancing cutting-edge research, with a focus on business applications. This will accelerate the adoption of generative AI within the German economy. The project also stresses responsible AI development to ensure that the models are reliable and aligned with European values and laws. The project provides resources, such as the LLM Workbook and a three part reference guide with examples and resources to help users better understand the key features and characteristics of large AI language model.

Gemini Advanced is an AI model that delivers unmatched performance in natural language generation, understanding, and problem solving across diverse domains. It features a revolutionary neural structure that delivers exceptional accuracy, nuanced context comprehension, and deep reason capabilities. Gemini Advanced can handle complex and multifaceted tasks. From creating detailed technical content to writing code, to providing strategic insights and conducting in-depth analysis of data, Gemini Advanced is designed to handle them all. Its adaptability, scalability and flexibility make it an ideal solution for both enterprise-level and individual applications. Gemini Advanced is a new standard in AI-powered solutions for intelligence, innovation and reliability. Google One also includes 2 TB of storage and access to Gemini, Docs and more. Gemini Advanced offers access to Gemini Deep Research. You can perform real-time and in-depth research on virtually any subject.

Gemini 2.0, an advanced AI model developed by Google is designed to offer groundbreaking capabilities for natural language understanding, reasoning and multimodal interaction. Gemini 2.0 builds on the success of Gemini's predecessor by integrating large language processing and enhanced problem-solving, decision-making, and interpretation abilities. This allows it to interpret and produce human-like responses more accurately and nuanced. Gemini 2.0, unlike traditional AI models, is trained to handle a variety of data types at once, including text, code, images, etc. This makes it a versatile tool that can be used in research, education, business and creative industries. Its core improvements are better contextual understanding, reduced biased, and a more effective architecture that ensures quicker, more reliable results. Gemini 2.0 is positioned to be a major step in the evolution AI, pushing the limits of human-computer interactions.

Information overload is a major barrier to scientific progress. The explosion of scientific literature and data makes it harder to find useful insights among a vast amount of information. Search engines are used to access scientific knowledge today, but they cannot organize it. Galactica is an extensive language model which can store, combine, and reason about scientific information. We train using a large corpus of scientific papers, reference material and knowledge bases, among other sources. We outperform other models in a variety of scientific tasks. Galactica performs better than the latest GPT-3 on technical knowledge probes like LaTeX Equations by 68.2% to 49.0%. Galactica is also good at reasoning. It outperforms Chinchilla in mathematical MMLU with a score between 41.3% and 35.7%. And PaLM 540B in MATH, with a score between 20.4% and 8.8%.

LLaVA is a multimodal model that combines a Vicuna language model with a vision encoder to facilitate comprehensive visual-language understanding. LLaVA's chat capabilities are impressive, emulating multimodal functionality of models such as GPT-4. LLaVA 1.5 has achieved the best performance in 11 benchmarks using publicly available data. It completed training on a single 8A100 node in about one day, beating methods that rely upon billion-scale datasets. The development of LLaVA involved the creation of a multimodal instruction-following dataset, generated using language-only GPT-4. This dataset comprises 158,000 unique language-image instruction-following samples, including conversations, detailed descriptions, and complex reasoning tasks. This data has been crucial in training LLaVA for a wide range of visual and linguistic tasks.

Qwen2 is a large language model developed by Qwen Team, Alibaba Cloud. Qwen2 is an extensive series of large language model developed by the Qwen Team at Alibaba Cloud. It includes both base models and instruction-tuned versions, with parameters ranging from 0.5 to 72 billion. It also features dense models and a Mixture of Experts model. The Qwen2 Series is designed to surpass previous open-weight models including its predecessor Qwen1.5 and to compete with proprietary model across a wide spectrum of benchmarks, such as language understanding, generation and multilingual capabilities.

Falcon 2 11B is a cutting-edge open-source AI model, designed for multilingual and multimodal tasks, and the only one featuring vision-to-language capabilities. It outperforms Meta’s Llama 3 8B and rivals Google’s Gemma 7B, as verified by the Hugging Face Leaderboard. The next step in its evolution includes integrating a 'Mixture of Experts' framework to further elevate its performance and expand its capabilities.

Databricks has created an open, general purpose LLM called DBRX. DBRX is the new benchmark for open LLMs. It also provides open communities and enterprises that are building their own LLMs capabilities that were previously only available through closed model APIs. According to our measurements, DBRX surpasses GPT 3.5 and is competitive with Gemini 1.0 Pro. It is a code model that is more capable than specialized models such as CodeLLaMA 70B, and it also has the strength of a general-purpose LLM. This state-of the-art quality is accompanied by marked improvements in both training and inference performances. DBRX is the most efficient open model thanks to its finely-grained architecture of mixtures of experts (MoE). Inference is 2x faster than LLaMA2-70B and DBRX has about 40% less parameters in total and active count compared to Grok-1.

DeepSeek-R1 is a cutting-edge open-source reasoning model crafted by DeepSeek, designed to compete with leading models like OpenAI's o1. Available through web platforms, applications, and APIs, it excels in tackling complex challenges such as mathematics and programming. With outstanding performance on benchmarks like the AIME and MATH, DeepSeek-R1 leverages a mixture of experts (MoE) architecture, utilizing 671 billion total parameters while activating 37 billion parameters per token for exceptional efficiency and accuracy. This model exemplifies DeepSeek’s dedication to driving advancements in artificial general intelligence (AGI) through innovative and open source solutions.

Pixtral Large is Mistral AI’s latest open-weight multimodal model, featuring a powerful 124-billion-parameter architecture. It combines a 123-billion-parameter multimodal decoder with a 1-billion-parameter vision encoder, allowing it to excel at interpreting documents, charts, and natural images while maintaining top-tier text comprehension. With a 128,000-token context window, it can process up to 30 high-resolution images simultaneously. The model has achieved cutting-edge results on benchmarks like MathVista, DocVQA, and VQAv2, outperforming competitors such as GPT-4o and Gemini-1.5 Pro. Available under the Mistral Research License for non-commercial use and the Mistral Commercial License for enterprise applications, Pixtral Large is designed for advanced AI-powered understanding.

VideoPoet, a simple modeling technique, can convert any large language model or autoregressive model into a high quality video generator. It is composed of a few components. The autoregressive model learns from video, image, text, and audio modalities in order to predict the next audio or video token in the sequence. The LLM training framework introduces a mixture of multimodal generative objectives, including text to video, text to image, image-to video, video frame continuation and inpainting/outpainting, styled video, and video-to audio. Moreover, these tasks can be combined to provide additional zero-shot capabilities. This simple recipe shows how language models can edit and synthesize videos with a high level of temporal consistency.

Inception Labs is redefining language model performance with its diffusion-based large language models (dLLMs), delivering unparalleled speed, efficiency, and precision. Unlike traditional models that generate text token-by-token, Inception’s dLLMs refine an initial noisy output into structured, high-quality responses. This innovation results in faster processing, reduced computational costs, and superior multimodal capabilities, making it ideal for complex reasoning, AI agents, and structured text generation. With its first commercial-scale model, Mercury, Inception is pushing AI to the next frontier, offering organizations and developers an advanced tool for next-gen AI applications.

You can use advanced language models and coding to solve a variety of problems. To build cutting-edge applications, leverage large-scale, generative AI models that have deep understandings of code and language to allow for new reasoning and comprehension. These coding and language models can be applied to a variety use cases, including writing assistance, code generation, reasoning over data, and code generation. Access enterprise-grade Azure security and detect and mitigate harmful use. Access generative models that have been pretrained with trillions upon trillions of words. You can use them to create new scenarios, including code, reasoning, inferencing and comprehension. A simple REST API allows you to customize generative models with labeled information for your particular scenario. To improve the accuracy of your outputs, fine-tune the hyperparameters of your model. You can use the API's few-shot learning capability for more relevant results and to provide examples.

Yi-Large, a proprietary large language engine developed by 01.AI with a 32k context size and input and output costs of $2 per million tokens. It is distinguished by its advanced capabilities in common-sense reasoning and multilingual support. It performs on par with leading models such as GPT-4 and Claude3 when it comes to various benchmarks. Yi-Large was designed to perform tasks that require complex inference, language understanding, and prediction. It is suitable for applications such as knowledge search, data classifying, and creating chatbots. Its architecture is built on a decoder only transformer with enhancements like pre-normalization, Group Query attention, and has been trained using a large, high-quality, multilingual dataset. The model's versatility, cost-efficiency and global deployment potential make it a strong competitor in the AI market.

DataGemma is a pioneering project by Google that aims to improve the accuracy and reliability large language models (LLMs), when dealing with numerical and statistical data. DataGemma, launched as a collection of open models, leverages Google's Data Commons - a vast repository for public statistical data - to ground its responses in actual facts. This initiative uses two innovative approaches, Retrieval Interleaved Generation and Retrieval Augmented Generation. RIG integrates real-time checks of data during the generation process, ensuring factual accuracy. RAG retrieves pertinent information before generating answers, reducing the likelihood that AI hallucinations will occur. DataGemma's goal is to provide users with factual and trustworthy answers. This marks a significant step in reducing the amount of misinformation that AI-generated content contains.

The expansion of large language model has led to significant advancements in natural language processing, understanding and generation. This study introduces a new system that uses Ascend 910 AI processing units and the MindSpore framework in order to train a language with over one trillion parameters, 1.085T specifically, called PanGu-Sigma. This model, which builds on the foundation laid down by PanGu-alpha transforms the traditional dense Transformer model into a sparse model using a concept called Random Routed Experts. The model was trained efficiently on a dataset consisting of 329 billion tokens, using a technique known as Expert Computation and Storage Separation. This led to a 6.3 fold increase in training performance via heterogeneous computer. The experiments show that PanGu-Sigma is a new standard for zero-shot learning in various downstream Chinese NLP tasks.

Mistral NeMo, our new best small model. A state-of the-art 12B with 128k context and released under Apache 2.0 license. Mistral NeMo, a 12B-model built in collaboration with NVIDIA, is available. Mistral NeMo has a large context of up to 128k Tokens. Its reasoning, world-knowledge, and coding precision are among the best in its size category. Mistral NeMo, which relies on a standard architecture, is easy to use. It can be used as a replacement for any system that uses Mistral 7B. We have released Apache 2.0 licensed pre-trained checkpoints and instruction-tuned base checkpoints to encourage adoption by researchers and enterprises. Mistral NeMo has been trained with quantization awareness to enable FP8 inferences without performance loss. The model was designed for global applications that are multilingual. It is trained in function calling, and has a large contextual window. It is better than Mistral 7B at following instructions, reasoning and handling multi-turn conversation.

OLMo 2 is an open language model family developed by the Allen Institute for AI. It provides researchers and developers with open-source code and reproducible training recipes. These models can be trained with up to 5 trillion tokens, and they are competitive against other open-weight models such as Llama 3.0 on English academic benchmarks. OLMo 2 focuses on training stability by implementing techniques that prevent loss spikes in long training runs. It also uses staged training interventions to address capability deficits during late pretraining. The models incorporate the latest post-training methods from AI2's Tulu 3 resulting in OLMo 2-Instruct. The Open Language Modeling Evaluation System, or OLMES, was created to guide improvements throughout the development stages. It consists of 20 evaluation benchmarks assessing key capabilities.

Falcon 3 is the latest open-source large language model (LLM) from the Technology Innovation Institute (TII), designed to bring powerful AI capabilities to a wider audience. Built for efficiency, it can run smoothly on lightweight devices, including laptops, without compromising speed or performance. The Falcon 3 ecosystem features four scalable models, each optimized for different applications, and supports multiple languages while maintaining resource efficiency. Excelling in tasks such as reasoning, language comprehension, instruction following, coding, and mathematics, Falcon 3 sets a new benchmark in AI accessibility. With its balance of high performance and low computational requirements, it aims to make advanced AI more available to users across industries.

Falcon Mamba 7B is the first open-source State Space Language Model (SSLM), introducing a revolutionary advancement in Falcon's architecture. Independently ranked as the top-performing open-source SSLM by Hugging Face, it redefines efficiency in AI language models. With low memory requirements and the ability to generate long text sequences without additional computational costs, Falcon Mamba 7B outperforms traditional transformer models like Meta’s Llama 3.1 8B and Mistral’s 7B. This cutting-edge model highlights Abu Dhabi’s leadership in AI research and innovation, pushing the boundaries of what’s possible in open-source machine learning.

Qwen LLM is a family of large-language models (LLMs), developed by Damo Academy, an Alibaba Cloud subsidiary. These models are trained using a large dataset of text and codes, allowing them the ability to understand and generate text that is human-like, translate languages, create different types of creative content and answer your question in an informative manner. Here are some of the key features of Qwen LLMs. Variety of sizes: Qwen's series includes sizes ranging from 1.8 billion parameters to 72 billion, offering options that meet different needs and performance levels. Open source: Certain versions of Qwen have open-source code, which is available to anyone for use and modification. Qwen is multilingual and can translate multiple languages including English, Chinese and Japanese. Qwen models are capable of a wide range of tasks, including text summarization and code generation, as well as generation and translation.

Mistral AI has introduced two state of the art models for on-device computing, and edge use cases. These models are called "les Ministraux", Ministral 3B, and Ministral 8B. These models are a new frontier for knowledge, commonsense, function-calling and efficiency within the sub-10B category. They can be used for a variety of applications, from orchestrating workflows to creating task workers. Both models support contexts up to 128k (currently 32k for vLLM) and Ministral 8B has a sliding-window attention pattern that allows for faster and more memory-efficient inference. These models were designed to provide a low-latency and compute-efficient solution for scenarios like on-device translators, internet-less intelligent assistants, local analytics and autonomous robotics. Les Ministraux, when used in conjunction with larger languages models such as Mistral Large or other agentic workflows, can also be efficient intermediaries in function-calling.

Mistral Large is a state-of-the-art language model developed by Mistral AI, designed for advanced text generation, multilingual reasoning, and complex problem-solving. Supporting multiple languages, including English, French, Spanish, German, and Italian, it provides deep linguistic understanding and cultural awareness. With an extensive 32,000-token context window, the model can process and retain information from long documents with exceptional accuracy. Its strong instruction-following capabilities and native function-calling support make it an ideal choice for AI-driven applications and system integrations. Available via Mistral’s platform, Azure AI Studio, and Azure Machine Learning, it can also be self-hosted for privacy-sensitive use cases. Benchmark results position Mistral Large as one of the top-performing models accessible through an API, second only to GPT-4.