Moxin 7B: A Fully Open-Source 7B Language Model with Unprecedented Transparency

We’re thrilled to unveil Moxin 7B, a new milestone in open large language model (LLM) development — designed to push the boundaries of performance and openness.

In an era where many "open" LLMs lack true transparency (e.g., missing training code, data, or restrictive licenses), Moxin 7B sets a new gold standard by committing to full disclosure and reproducibility. Developed under the Model Openness Framework (MOF), Moxin 7B achieves the top classification level of Open Science, thanks to:

What we’ve open-sourced:

• Pre-training code, data, and Moxin Base model.

• Post-training code, data, and Moxin Instruct model.

• RL code with GRPO, data and Moxin Reasoning model.

Performance Highlights:

• Zero-shot / Few-shot: Outperforms Mistral, Qwen, and LLaMA on tasks like HellaSwag, ARC, MMLU, and PIQA

• Reasoning: Moxin-Reasoning-7B achieves superior performance on MATH-500, AMC, and OlympiadBench — proving reinforcement learning can work for small 7B models

• Training cost: ~$160K for full pretraining — efficient and reproducible at scale

Post-training Frameworks:

• SFT and DPO with Tülu 3

• CoT-enhanced reasoning with GRPO via DeepScaleR

Get the models and code:

• Base model: Moxin-LLM-7B

• Instruction model: Moxin-Instruct-7B

• Reasoning model: Moxin-Reasoning-7B

• Code & docs: github.com/moxin-org/Moxin-LLM

• Arxiv paper: https://arxiv.org/abs/2412.06845

We believe this is a step toward a more transparent, reproducible, and innovation-friendly AI ecosystem — especially for researchers, developers, and startups looking to build upon a robust, open foundation. Let’s build open AI the right way.