Compare Leanstral 1.5 vs. Olmo 3

Leanstral 1.5 is a model licensed under Apache-2.0, designed for effective proof engineering in Lean 4, aimed at enhancing the capabilities and accessibility of formal verification. It boasts a total of 119 billion parameters, with 6 billion of them being active, marking a significant improvement in performance for tasks such as theorem proving, agent-based proof engineering, and the verification of practical code. The development of Leanstral 1.5 involved a comprehensive three-stage training process, which included mid-training, supervised fine-tuning, and reinforcement learning utilizing CISPO. In a multiturn environment, the model is tasked with receiving a theorem statement, submitting a proof, and refining its approach based on feedback from the Lean compiler until the proof is either successfully compiled or the available resources are depleted. In the code agent setting, Leanstral functions similarly to a developer navigating a raw filesystem, allowing it to edit files, execute bash commands, and interact with the Lean language server to monitor goals, errors, and type information in real time. This innovative approach not only streamlines the proof engineering process but also significantly enhances the user experience in formal verification tasks.

Olmo 3 represents a comprehensive family of open models featuring variations with 7 billion and 32 billion parameters, offering exceptional capabilities in base performance, reasoning, instruction, and reinforcement learning, while also providing transparency throughout the model development process, which includes access to raw training datasets, intermediate checkpoints, training scripts, extended context support (with a window of 65,536 tokens), and provenance tools. The foundation of these models is built upon the Dolma 3 dataset, which comprises approximately 9 trillion tokens and utilizes a careful blend of web content, scientific papers, programming code, and lengthy documents; this thorough pre-training, mid-training, and long-context approach culminates in base models that undergo post-training enhancements through supervised fine-tuning, preference optimization, and reinforcement learning with accountable rewards, resulting in the creation of the Think and Instruct variants. Notably, the 32 billion Think model has been recognized as the most powerful fully open reasoning model to date, demonstrating performance that closely rivals that of proprietary counterparts in areas such as mathematics, programming, and intricate reasoning tasks, thereby marking a significant advancement in open model development. This innovation underscores the potential for open-source models to compete with traditional, closed systems in various complex applications.