Alternatives to LTM-1

Baichuan-13B, a large-scale language model with 13 billion parameters that is open source and available commercially by Baichuan Intelligent, was developed following Baichuan -7B. It has the best results for a language model of the same size in authoritative Chinese and English benchmarks. This release includes two versions of pretraining (Baichuan-13B Base) and alignment (Baichuan-13B Chat). Baichuan-13B has more data and a larger size. It expands the number parameters to 13 billion based on Baichuan -7B, and trains 1.4 trillion coins on high-quality corpus. This is 40% more than LLaMA-13B. It is open source and currently the model with the most training data in 13B size. Support Chinese and English bi-lingual, use ALiBi code, context window is 4096.

LTM-2 mini is a 100M token model: LTM-2 mini. 100M tokens is 10,000,000 lines of code, or 750 novels. LTM-2 mini's sequence-dimension algorithms is approximately 1000x cheaper for each token decoded than the attention mechanism of Llama 3.0 405B1 when a 100M tokens context window is used. LTM only requires a fraction of one H100 HBM per user to store the same context.

This repository contains a research preview of LongLLaMA. It is a large language-model capable of handling contexts up to 256k tokens. LongLLaMA was built on the foundation of OpenLLaMA, and fine-tuned with the Focused Transformer method. LongLLaMA code was built on the foundation of Code Llama. We release a smaller base variant of the LongLLaMA (not instruction-tuned) on a permissive licence (Apache 2.0), and inference code that supports longer contexts for hugging face. Our model weights are a drop-in replacement for LLaMA (for short contexts up to 2048 tokens) in existing implementations. We also provide evaluation results, and comparisons with the original OpenLLaMA model.

Claude Pro is a large language model that can handle complex tasks with a friendly and accessible demeanor. It is trained on high-quality, extensive data and excels at understanding contexts, interpreting subtleties, and producing well structured, coherent responses to a variety of topics. Claude Pro is able to create detailed reports, write creative content, summarize long documents, and assist with coding tasks by leveraging its robust reasoning capabilities and refined knowledge base. Its adaptive algorithms constantly improve its ability learn from feedback. This ensures that its output is accurate, reliable and helpful. Whether Claude Pro is serving professionals looking for expert support or individuals seeking quick, informative answers - it delivers a versatile, productive conversational experience.

GPT-5 is OpenAI's Generative Pretrained Transformer. It is a large-language model (LLM), which is still in development. LLMs have been trained to work with massive amounts of text and can generate realistic and coherent texts, translate languages, create different types of creative content and answer your question in a way that is informative. It's still not available to the public. OpenAI has not announced a release schedule, but some believe it could launch in 2024. It's expected that GPT-5 will be even more powerful. GPT-4 has already proven to be impressive. It is capable of writing creative content, translating languages and generating text of human-quality. GPT-5 will be expected to improve these abilities, with improved reasoning, factual accuracy and ability to follow directions.

Gemini models are a family of light-open, state-of-the art models that was created using the same research and technology as Gemini models. These models include comprehensive security measures, and help to ensure responsible and reliable AI through selected data sets. Gemma models have exceptional comparative results, even surpassing some larger open models, in their 2B and 7B sizes. Keras 3.0 offers seamless compatibility with JAX TensorFlow PyTorch and JAX. Gemma 2 has been redesigned to deliver unmatched performance and efficiency. It is optimized for inference on a variety of hardware. The Gemma models are available in a variety of models that can be customized to meet your specific needs. The Gemma models consist of large text-to text lightweight language models that have a decoder and are trained on a large set of text, code, or mathematical content.

CodeQwen, developed by the Qwen Team, Alibaba Cloud, is the code version. It is a transformer based decoder only language model that has been pre-trained with a large number of codes. A series of benchmarks shows that the code generation is strong and that it performs well. Supporting long context generation and understanding with a context length of 64K tokens. CodeQwen is a 92-language coding language that provides excellent performance for text-to SQL, bug fixes, and more. CodeQwen chat is as simple as writing a few lines of code using transformers. We build the tokenizer and model using pre-trained methods and use the generate method for chatting. The chat template is provided by the tokenizer. Following our previous practice, we apply the ChatML Template for chat models. The model will complete the code snippets in accordance with the prompts without any additional formatting.

DataGemma is a pioneering project by Google that aims to improve the accuracy and reliability large language models (LLMs), when dealing with numerical and statistical data. DataGemma, launched as a collection of open models, leverages Google's Data Commons - a vast repository for public statistical data - to ground its responses in actual facts. This initiative uses two innovative approaches, Retrieval Interleaved Generation and Retrieval Augmented Generation. RIG integrates real-time checks of data during the generation process, ensuring factual accuracy. RAG retrieves pertinent information before generating answers, reducing the likelihood that AI hallucinations will occur. DataGemma's goal is to provide users with factual and trustworthy answers. This marks a significant step in reducing the amount of misinformation that AI-generated content contains.

Small language models (SLMs), a powerful family of small language models, with low cost and low-latency performance. Maximize AI capabilities and lower resource usage, while ensuring cost-effective generative AI implementations across your applications. Accelerate response time in real-time interaction, autonomous systems, low latency apps, and other critical scenarios. Phi-3 can be run in the cloud, on the edge or on the device. This allows for greater flexibility in deployment and operation. Phi-3 models have been developed according to Microsoft AI principles, including accountability, transparency and fairness, reliability, safety and security, privacy, and inclusivity. Operate efficiently in offline environments, where data privacy or connectivity are limited. Expanded context window allows for more accurate, contextually relevant and coherent outputs. Deploy at edge to deliver faster response.

Mistral Large 2 comes with a 128k window that supports dozens of different languages, including French, German and Spanish. It also supports Arabic, Hindi, Russian and Chinese. It also supports 80+ programming languages, including Python, Java and C++. Mistral Large 2 was designed with single-node applications in mind. Its size of 123 million parameters allows it to run fast on a single computer. Mistral Large 2 is released under the Mistral Research License which allows modification and usage for research and noncommercial purposes.

Mistral NeMo, our new best small model. A state-of the-art 12B with 128k context and released under Apache 2.0 license. Mistral NeMo, a 12B-model built in collaboration with NVIDIA, is available. Mistral NeMo has a large context of up to 128k Tokens. Its reasoning, world-knowledge, and coding precision are among the best in its size category. Mistral NeMo, which relies on a standard architecture, is easy to use. It can be used as a replacement for any system that uses Mistral 7B. We have released Apache 2.0 licensed pre-trained checkpoints and instruction-tuned base checkpoints to encourage adoption by researchers and enterprises. Mistral NeMo has been trained with quantization awareness to enable FP8 inferences without performance loss. The model was designed for global applications that are multilingual. It is trained in function calling, and has a large contextual window. It is better than Mistral 7B at following instructions, reasoning and handling multi-turn conversation.

A small model with superior textual Intelligence and multimodal reasoning. GPT-4o Mini's low cost and low latency enable a wide range of tasks, including applications that chain or paralelize multiple model calls (e.g. calling multiple APIs), send a large amount of context to the models (e.g. full code base or history of conversations), or interact with clients through real-time, fast text responses (e.g. customer support chatbots). GPT-4o Mini supports text and vision today in the API. In the future, it will support text, image and video inputs and outputs. The model supports up to 16K outputs tokens per request and has knowledge until October 2023. It has a context of 128K tokens. The improved tokenizer shared by GPT-4o makes it easier to handle non-English text.

Claude is an artificial intelligence language model that can generate text with human-like processing. Anthropic is an AI safety company and research firm that focuses on building reliable, interpretable and steerable AI systems. While large, general systems can provide significant benefits, they can also be unpredictable, unreliable and opaque. Our goal is to make progress in these areas. We are currently focusing on research to achieve these goals. However, we see many opportunities for our work in the future to create value both commercially and for the public good.

StarCoderBase and StarCoder are Large Language Models (Code LLMs), trained on permissively-licensed data from GitHub. This includes data from 80+ programming language, Git commits and issues, Jupyter Notebooks, and Git commits. We trained a 15B-parameter model for 1 trillion tokens, similar to LLaMA. We refined the StarCoderBase for 35B Python tokens. The result is a new model we call StarCoder. StarCoderBase is a model that outperforms other open Code LLMs in popular programming benchmarks. It also matches or exceeds closed models like code-cushman001 from OpenAI, the original Codex model which powered early versions GitHub Copilot. StarCoder models are able to process more input with a context length over 8,000 tokens than any other open LLM. This allows for a variety of interesting applications. By prompting the StarCoder model with a series dialogues, we allowed them to act like a technical assistant.

John Snow Labs Medical LLM is a domain-specific large langauge model (LLM) that revolutionizes the way healthcare organizations harness artificial intelligence. This innovative platform was designed specifically for the healthcare sector, combining cutting edge natural language processing capabilities with a profound understanding of medical terminology and clinical workflows. The result is an innovative tool that allows healthcare providers, researchers and administrators to unlock new insight, improve patient outcomes and drive operational efficiency. The Healthcare LLM's comprehensive training is at the core of its functionality. This includes a vast amount of healthcare data such as clinical notes, research papers and regulatory documents. This specialized training allows for the model to accurately generate and interpret medical text. It is an invaluable tool for tasks such clinical documentation, automated coding and medical research.

Qwen LLM is a family of large-language models (LLMs), developed by Damo Academy, an Alibaba Cloud subsidiary. These models are trained using a large dataset of text and codes, allowing them the ability to understand and generate text that is human-like, translate languages, create different types of creative content and answer your question in an informative manner. Here are some of the key features of Qwen LLMs. Variety of sizes: Qwen's series includes sizes ranging from 1.8 billion parameters to 72 billion, offering options that meet different needs and performance levels. Open source: Certain versions of Qwen have open-source code, which is available to anyone for use and modification. Qwen is multilingual and can translate multiple languages including English, Chinese and Japanese. Qwen models are capable of a wide range of tasks, including text summarization and code generation, as well as generation and translation.

With just a few lines, you can integrate natural language understanding and generation into the product. The Cohere API allows you to access models that can read billions upon billions of pages and learn the meaning, sentiment, intent, and intent of every word we use. You can use the Cohere API for human-like text. Simply fill in a prompt or complete blanks. You can create code, write copy, summarize text, and much more. Calculate the likelihood of text, and retrieve representations from your model. You can filter text using the likelihood API based on selected criteria or categories. You can create your own downstream models for a variety of domain-specific natural languages tasks by using representations. The Cohere API is able to compute the similarity of pieces of text and make categorical predictions based on the likelihood of different text options. The model can see ideas through multiple lenses so it can identify abstract similarities between concepts as distinct from DNA and computers.

Command R+, Cohere's latest large language model, is optimized for conversational interactions and tasks with a long context. It is designed to be extremely performant and enable companies to move from proof-of-concept into production. We recommend Command R+ when working with workflows that rely on complex RAG functionality or multi-step tool usage (agents). Command R is better suited for retrieval augmented creation (RAG) tasks and single-step tool usage, or applications where cost is a key consideration.

PygmalionAI, a community of open-source projects based upon EleutherAI’s GPT-J 6B models and Meta’s LLaMA model, was founded in 2009. Pygmalion AI is designed for roleplaying and chatting. The 7B variant of the Pygmalion AI is currently actively supported. It is based on Meta AI’s LLaMA AI model. Pygmalion's chat capabilities are superior to larger language models that require much more resources. Our curated datasets of high-quality data on roleplaying ensure that your bot is the best RP partner. The model weights as well as the code used to train the model are both open-source. You can modify/re-distribute them for any purpose you like. Pygmalion and other language models run on GPUs because they require fast memory and massive processing to produce coherent text at a reasonable speed.

BLOOM (autoregressive large language model) is trained to continue text using a prompt on large amounts of text data. It uses industrial-scale computational resources. It can produce coherent text in 46 languages and 13 programming language, which is almost impossible to distinguish from text written by humans. BLOOM can be trained to perform text tasks that it hasn’t been explicitly trained for by casting them as text generation jobs.

Codestral Mamba is a Mamba2 model that specializes in code generation. It is available under the Apache 2.0 license. Codestral Mamba represents another step in our efforts to study and provide architectures. We hope that it will open up new perspectives in architecture research. Mamba models have the advantage of linear inference of time and the theoretical ability of modeling sequences of unlimited length. Users can interact with the model in a more extensive way with rapid responses, regardless of the input length. This efficiency is particularly relevant for code productivity use-cases. We trained this model with advanced reasoning and code capabilities, enabling the model to perform at par with SOTA Transformer-based models.

Ai2 OpenScholar, a collaboration between the University of Washington's Allen Institute for AI and the University of Washington, is designed to help scientists navigate and synthesize the vast expanse of the scientific literature. OpenScholar uses a retrieval-augmented model of language to answer user queries. It does this by identifying relevant papers and then generating answers based on those sources. This ensures that information is accurate and linked directly to existing research. OpenScholar-8B set new standards for factuality and accuracy of citations on the ScholarQABench benchmark. OpenScholar-8B, for example, maintains a solid grounding in real retrieved articles in the biomedical domain. This is in contrast to models like GPT-4 which tend to hallucinate references. Twenty scientists from computer science, biomedicine and physics evaluated OpenScholar's answers against expert-written responses to evaluate its real-world application.

Qwen2.5, an advanced multimodal AI system, is designed to provide highly accurate responses that are context-aware across a variety of applications. It builds on its predecessors' capabilities, integrating cutting edge natural language understanding, enhanced reasoning, creativity and multimodal processing. Qwen2.5 is able to analyze and generate text as well as interpret images and interact with complex data in real-time. It is highly adaptable and excels at personalized assistance, data analytics, creative content creation, and academic research. This makes it a versatile tool that can be used by professionals and everyday users. Its user-centric approach emphasizes transparency, efficiency and alignment with ethical AI.

A model parallel autoregressive transformator implementation on GPUs based on the DeepSpeed Library. This repository contains EleutherAI’s library for training large language models on GPUs. Our current framework is based upon NVIDIA's Megatron Language Model, and has been enhanced with techniques from DeepSpeed, as well as some novel improvements. This repo is intended to be a central and accessible place for techniques to train large-scale autoregressive models and to accelerate research into large scale training.

StableLM: Stability AI language models StableLM builds upon our experience with open-sourcing previous language models in collaboration with EleutherAI. This nonprofit research hub. These models include GPTJ, GPTNeoX and the Pythia Suite, which were all trained on The Pile dataset. Cerebras GPT and Dolly-2 are two recent open-source models that continue to build upon these efforts. StableLM was trained on a new dataset that is three times bigger than The Pile and contains 1.5 trillion tokens. We will provide more details about the dataset at a later date. StableLM's richness allows it to perform well in conversational and coding challenges, despite the small size of its dataset (3-7 billion parameters, compared to GPT-3's 175 billion). The development of Stable LM 3B broadens the range of applications that are viable on the edge or on home PCs. This means that individuals and companies can now develop cutting-edge technologies with strong conversational capabilities – like creative writing assistance – while keeping costs low and performance high.

Gemini Flash, a large language model from Google, is specifically designed for low-latency, high-speed language processing tasks. Gemini Flash, part of Google DeepMind’s Gemini series is designed to handle large-scale applications and provide real-time answers. It's ideal for interactive AI experiences such as virtual assistants, live chat, and customer support. Gemini Flash is built on sophisticated neural structures that ensure contextual relevance, coherence, and precision. Google has built in rigorous ethical frameworks as well as responsible AI practices to Gemini Flash. It also equipped it with guardrails that manage and mitigate biased outcomes, ensuring alignment with Google's standards of safe and inclusive AI. Google's Gemini Flash empowers businesses and developers with intelligent, responsive language tools that can keep up with fast-paced environments.

Mixtral 8x22B is our latest open model. It sets new standards for performance and efficiency in the AI community. It is a sparse Mixture-of-Experts model (SMoE), which uses only 39B active variables out of 141B. This offers unparalleled cost efficiency in relation to its size. It is fluently bilingual in English, French Italian, German and Spanish. It has strong math and coding skills. It is natively able to call functions; this, along with the constrained-output mode implemented on La Plateforme, enables application development at scale and modernization of tech stacks. Its 64K context window allows for precise information retrieval from large documents. We build models with unmatched cost-efficiency for their respective sizes. This allows us to deliver the best performance-tocost ratio among models provided by the Community. Mixtral 8x22B continues our open model family. Its sparse patterns of activation make it faster than any 70B model.

The next generation of the large language model. This release includes modelweights and starting code to pretrained and fine tuned Llama languages models, ranging from 7B-70B parameters. Llama 1 models have a context length of 2 trillion tokens. Llama 2 models have a context length double that of Llama 1. The fine-tuned Llama 2 models have been trained using over 1,000,000 human annotations. Llama 2, a new open-source language model, outperforms many other open-source language models in external benchmarks. These include tests of reasoning, coding and proficiency, as well as knowledge tests. Llama 2 has been pre-trained using publicly available online data sources. Llama-2 chat, a fine-tuned version of the model, is based on publicly available instruction datasets, and more than 1 million human annotations. We have a wide range of supporters in the world who are committed to our open approach for today's AI. These companies have provided early feedback and have expressed excitement to build with Llama 2

Through scientific rigor and human insight, healthcare breakthroughs can change the world, bringing hope to humanity. We believe that AI can help in this area, through collaboration between researchers, healthcare organisations, and the wider ecosystem. Today, we are sharing exciting progress in these initiatives with the announcement that Google's large language model (LLM) for medical applications, called Med PaLM 2, will be available to a limited number of customers. In the coming weeks, it will be available to a small group of Google Cloud users for limited testing. We will explore use cases, share feedback, and investigate safe, responsible and meaningful ways to utilize this technology. Med-PaLM 2, which harnesses Google's LLMs aligned with the medical domain, is able to answer medical questions more accurately and safely. Med-PaLM 2 is the first LLM that has performed at an "expert" level on the MedQA dataset consisting of US Medical Licensing Examination-style questions.

Code Llama, a large-language model (LLM), can generate code using text prompts. Code Llama, the most advanced publicly available LLM for code tasks, has the potential to improve workflows for developers and reduce the barrier for those learning to code. Code Llama can be used to improve productivity and educate programmers to create more robust, well documented software. Code Llama, a state-of the-art LLM, is capable of generating both code, and natural languages about code, based on both code and natural-language prompts. Code Llama can be used for free in research and commercial purposes. Code Llama is a new model that is built on Llama 2. It is available in 3 models: Code Llama is the foundational model of code; Codel Llama is a Python-specific language. Code Llama-Instruct is a finely tuned natural language instruction interpreter.

OpenGPT is a German initiative that focuses on developing large AI languages models tailored to European requirements, with an emphasis on versatility, trustworthiness and multilingual capabilities. It also emphasizes open-source accessibility. The project brings together partners to cover the whole generative AI value-chain, from scalable GPU-based infrastructure to data for training large language model to model design, practical applications, and prototypes and proofs-of concept. OpenGPT-X aims at advancing cutting-edge research, with a focus on business applications. This will accelerate the adoption of generative AI within the German economy. The project also stresses responsible AI development to ensure that the models are reliable and aligned with European values and laws. The project provides resources, such as the LLM Workbook and a three part reference guide with examples and resources to help users better understand the key features and characteristics of large AI language model.

You can use advanced language models and coding to solve a variety of problems. To build cutting-edge applications, leverage large-scale, generative AI models that have deep understandings of code and language to allow for new reasoning and comprehension. These coding and language models can be applied to a variety use cases, including writing assistance, code generation, reasoning over data, and code generation. Access enterprise-grade Azure security and detect and mitigate harmful use. Access generative models that have been pretrained with trillions upon trillions of words. You can use them to create new scenarios, including code, reasoning, inferencing and comprehension. A simple REST API allows you to customize generative models with labeled information for your particular scenario. To improve the accuracy of your outputs, fine-tune the hyperparameters of your model. You can use the API's few-shot learning capability for more relevant results and to provide examples.

GPT-J, a cutting edge language model developed by EleutherAI, is a leading-edge language model. GPT-J's performance is comparable to OpenAI's GPT-3 model on a variety of zero-shot tasks. GPT-J, in particular, has shown that it can surpass GPT-3 at tasks relating to code generation. The latest version of this language model is GPT-J-6B and is built on a linguistic data set called The Pile. This dataset is publically available and contains 825 gibibytes worth of language data organized into 22 subsets. GPT-J has some similarities with ChatGPT. However, GPTJ is not intended to be a chatbot. Its primary function is to predict texts. Databricks made a major development in March 2023 when they introduced Dolly, an Apache-licensed model that follows instructions.

LLaMA (Large Language Model meta AI) is a state of the art foundational large language model that was created to aid researchers in this subfield. LLaMA allows researchers to use smaller, more efficient models to study these models. This furtherdemocratizes access to this rapidly-changing field. Because it takes far less computing power and resources than large language models, such as LLaMA, to test new approaches, validate other's work, and explore new uses, training smaller foundation models like LLaMA can be a desirable option. Foundation models are trained on large amounts of unlabeled data. This makes them perfect for fine-tuning for many tasks. We make LLaMA available in several sizes (7B-13B, 33B and 65B parameters), and also share a LLaMA card that explains how the model was built in line with our Responsible AI practices.

Qwen-7B, also known as Qwen-7B, is the 7B-parameter variant of the large language models series Qwen. Tongyi Qianwen, proposed by Alibaba Cloud. Qwen-7B, a Transformer-based language model, is pretrained using a large volume data, such as web texts, books, code, etc. Qwen-7B is also used to train Qwen-7B Chat, an AI assistant that uses large models and alignment techniques. The Qwen-7B features include: Pre-trained with high quality data. We have pretrained Qwen-7B using a large-scale, high-quality dataset that we constructed ourselves. The dataset contains over 2.2 trillion tokens. The dataset contains plain texts and codes and covers a wide range domains including general domain data as well as professional domain data. Strong performance. We outperform our competitors in a series benchmark datasets that evaluate natural language understanding, mathematics and coding. And more.

Gemini Nano is a tiny version of the Gemini family. It is the latest generation of Google DeepMind multimodal language models. Nano is a super-powered AI that fits snugly into your smartphone. Nano is the smallest (along with its siblings Ultra and Pro), but it packs a powerful punch. It is specifically designed to run on mobile devices, such as your phone, and brings powerful AI capabilities to your fingertips even when you are offline. Imagine it as your ultimate assistant on your device, whispering intelligent suggestions and automating tasks effortlessly. Want to summarize that long recorded lecture quickly? Nano has you covered. Want to create the perfect response to a tricky text message? Nano will give you options that will make your friends think you're an expert wordsmith.

The ability of large language models to learn in zero- and few shots, despite being trained for hundreds of thousands or even millions of days, has been remarkable. These models are expensive to replicate, due to their high computational cost. The few models that are available via APIs do not allow access to the full weights of the model, making it difficult to study. Open Pre-trained Transformers is a suite decoder-only pre-trained transforms with parameters ranging from 175B to 125M. We aim to share this fully and responsibly with interested researchers. We show that OPT-175B has a carbon footprint of 1/7th that of GPT-3. We will also release our logbook, which details the infrastructure challenges we encountered, as well as code for experimenting on all of the released model.

Alpa aims automate large-scale distributed training. Alpa was originally developed by people at UC Berkeley's Sky Lab. Alpa's advanced techniques were described in a paper published by OSDI'2022. Google is adding new members to the Alpa community. A language model is a probabilistic distribution of probability over a sequence of words. It uses all the words it has seen to predict the next word. It is useful in a variety AI applications, including the auto-completion of your email or chatbot service. You can find more information on the language model Wikipedia page. GPT-3 is a large language model with 175 billion parameters that uses deep learning to produce text that looks human-like. GPT-3 was described by many researchers and news articles as "one the most important and interesting AI systems ever created." GPT-3 is being used as a backbone for the latest NLP research.

LLaVA is a multimodal model that combines a Vicuna language model with a vision encoder to facilitate comprehensive visual-language understanding. LLaVA's chat capabilities are impressive, emulating multimodal functionality of models such as GPT-4. LLaVA 1.5 has achieved the best performance in 11 benchmarks using publicly available data. It completed training on a single 8A100 node in about one day, beating methods that rely upon billion-scale datasets. The development of LLaVA involved the creation of a multimodal instruction-following dataset, generated using language-only GPT-4. This dataset comprises 158,000 unique language-image instruction-following samples, including conversations, detailed descriptions, and complex reasoning tasks. This data has been crucial in training LLaVA for a wide range of visual and linguistic tasks.

OLMo 2 is an open language model family developed by the Allen Institute for AI. It provides researchers and developers with open-source code and reproducible training recipes. These models can be trained with up to 5 trillion tokens, and they are competitive against other open-weight models such as Llama 3.0 on English academic benchmarks. OLMo 2 focuses on training stability by implementing techniques that prevent loss spikes in long training runs. It also uses staged training interventions to address capability deficits during late pretraining. The models incorporate the latest post-training methods from AI2's Tulu 3 resulting in OLMo 2-Instruct. The Open Language Modeling Evaluation System, or OLMES, was created to guide improvements throughout the development stages. It consists of 20 evaluation benchmarks assessing key capabilities.

Phi-2 is a 2.7-billion-parameter language-model that shows outstanding reasoning and language-understanding capabilities. It represents the state-of-the art performance among language-base models with less than thirteen billion parameters. Phi-2 can match or even outperform models 25x larger on complex benchmarks, thanks to innovations in model scaling. Phi-2's compact size makes it an ideal playground for researchers. It can be used for exploring mechanistic interpretationability, safety improvements or fine-tuning experiments on a variety tasks. We have included Phi-2 in the Azure AI Studio catalog to encourage research and development of language models.

PaLM 2 is Google's next-generation large language model, which builds on Google’s research and development in machine learning. It excels in advanced reasoning tasks including code and mathematics, classification and question-answering, translation and multilingual competency, and natural-language generation better than previous state-of the-art LLMs including PaLM. It is able to accomplish these tasks due to the way it has been built - combining compute-optimal scale, an improved dataset mix, and model architecture improvement. PaLM 2 is based on Google's approach for building and deploying AI responsibly. It was rigorously evaluated for its potential biases and harms, as well as its capabilities and downstream applications in research and product applications. It is being used to power generative AI tools and features at Google like Bard, the PaLM API, and other state-ofthe-art models like Sec-PaLM and Med-PaLM 2.

RoBERTa is based on BERT's language-masking strategy. The system learns to predict hidden sections of text in unannotated language examples. RoBERTa was implemented in PyTorch and modifies key hyperparameters of BERT. This includes removing BERT’s next-sentence-pretraining objective and training with larger mini-batches. This allows RoBERTa improve on the masked-language modeling objective, which is comparable to BERT. It also leads to improved downstream task performance. We are also exploring the possibility of training RoBERTa with a lot more data than BERT and for a longer time. We used both existing unannotated NLP data sets as well as CC-News which was a new set of public news articles.

Gemma is the family of lightweight open models that are built using the same research and technology as the Gemini models. Gemma was developed by Google DeepMind, along with other teams within Google. The name is derived from the Latin gemma meaning "precious stones". We're also releasing new tools to encourage developer innovation, encourage collaboration, and guide responsible use of Gemma model. Gemma models are based on the same infrastructure and technical components as Gemini, Google's largest and most powerful AI model. Gemma 2B, 7B and other open models can achieve the best performance possible for their size. Gemma models can run directly on a desktop or laptop computer for developers. Gemma is able to surpass much larger models in key benchmarks, while adhering our rigorous standards of safe and responsible outputs.

Claude 3.5 Sonnet is a new benchmark for the industry in terms of graduate-level reasoning (GPQA), undergrad-level knowledge (MMLU), as well as coding proficiency (HumanEval). It is exceptional in writing high-quality, relatable content that is written with a natural and relatable tone. It also shows marked improvements in understanding nuance, humor and complex instructions. Claude 3.5 Sonnet is twice as fast as Claude 3 Opus. Claude 3.5 Sonnet is ideal for complex tasks, such as providing context-sensitive support to customers and orchestrating workflows. Claude 3.5 Sonnet can be downloaded for free from Claude.ai and Claude iOS, and subscribers to the Claude Pro and Team plans will have access to it at rates that are significantly higher. It is also accessible via the Anthropic AI, Amazon Bedrock and Google Cloud Vertex AI. The model costs $3 for every million input tokens. It costs $15 for every million output tokens. There is a 200K token window.

NVIDIA NeMoLLM is a service that allows you to quickly customize and use large language models that have been trained on multiple frameworks. Developers can use NeMo LLM to deploy enterprise AI applications on both public and private clouds. They can also experiment with Megatron 530B, one of the most powerful language models, via the cloud API or the LLM service. You can choose from a variety of NVIDIA models or community-developed models to best suit your AI applications. You can get better answers in minutes to hours by using prompt learning techniques and providing context for specific use cases. Use the NeMo LLM Service and the cloud API to harness the power of NVIDIA megatron 530B, the largest language model, or NVIDIA Megatron 535B. Use models for drug discovery in the NVIDIA BioNeMo framework and the cloud API.

Our new flagship model, the Small Language Model (SLM), has all the power and performance that you would expect from a leading LLM. Excels at generalized tasks, instruction-following, and human preferences. The best 70B model available. SuperNova is a generalized task-based AI that can be used for any generalized task. It's similar to Open AI's GPT4o and Claude Sonnet 3.5. SuperNova is trained with the most advanced optimization & learning techniques to generate highly accurate responses. It is the most flexible, cost-effective, and secure language model available. Customers can save up to 95% in total deployment costs when compared with traditional closed-source models. SuperNova can be used to integrate AI in apps and products, as well as for general chat and a variety of other uses. Update your models regularly with the latest open source tech to ensure you're not locked into a single solution. Protect your data using industry-leading privacy features.

GPT-4 (Generative Pretrained Transformer 4) a large-scale, unsupervised language model that is yet to be released. GPT-4, which is the successor of GPT-3, is part of the GPT -n series of natural-language processing models. It was trained using a dataset of 45TB text to produce text generation and understanding abilities that are human-like. GPT-4 is not dependent on additional training data, unlike other NLP models. It can generate text and answer questions using its own context. GPT-4 has been demonstrated to be capable of performing a wide range of tasks without any task-specific training data, such as translation, summarization and sentiment analysis.

Qwen2 is a large language model developed by Qwen Team, Alibaba Cloud. Qwen2 is an extensive series of large language model developed by the Qwen Team at Alibaba Cloud. It includes both base models and instruction-tuned versions, with parameters ranging from 0.5 to 72 billion. It also features dense models and a Mixture of Experts model. The Qwen2 Series is designed to surpass previous open-weight models including its predecessor Qwen1.5 and to compete with proprietary model across a wide spectrum of benchmarks, such as language understanding, generation and multilingual capabilities.

Pixtral 12B, a multimodal AI model pioneered by Mistral AI and designed to process and understand both text and images data seamlessly, is a groundbreaking AI model. This model represents a significant advance in the integration of data types. It allows for more intuitive interaction and enhanced content creation abilities. Pixtral 12B, which is based on Mistral's NeMo 12B Text Model, incorporates an additional Vision Adapter that adds 400 million parameters. This allows it to handle visual inputs of up to 1024x1024 pixels. This model is capable of a wide range of applications from image analysis to answering visual content questions. Its versatility is demonstrated in real-world scenarios. Pixtral 12B is a powerful tool for developers, as it not only has a large context of 128k tokens, but also uses innovative techniques such as GeLU activation and RoPE 2D for its vision components.