Falcon 180B: The largest openly available language model

Falcon 180B sets a new state-of-the-art for open models. It is the largest openly available language model, with 180 billion parameters, and was trained on a massive 3.5 trillion tokens. It tops the leaderboard for (pre-trained) open-access models and rivals proprietary models like PaLM-2. While difficult to rank definitively yet, it is considered on par with PaLM-2 Large, making Falcon 180B one of the most capable LLMs publicly known.

Read more

State of GPT

Learn about the training pipeline of GPT assistants like ChatGPT, from tokenization to pretraining, supervised finetuning, and Reinforcement Learning from Human Feedback (RLHF). Dive deeper into practical techniques and mental models for the effective use of these models, including prompting strategies, finetuning, the rapidly growing ecosystem of tools, and their future extensions.

Read more

OpenAI Whisper: A Robust and Versatile Speech Recognition System

Whisper is an automatic speech recognition (ASR) system trained on a massive 680,000-hour multilingual and multitask dataset collected from the web. This extensive and diverse dataset enhances Whisper's robustness to accents, background noise, and technical language. Additionally, it facilitates transcription in multiple languages and translation into English. Open-sourcing models and inference code aims to provide a foundation for developing practical applications and conducting further research on robust speech processing.

Read more