falcon 40b
tiiuae/falcon-40b
Open Source · chat · open-weights
Open
Alert me on changes
Context
2K
Max output
—
Weights
Open
API $/1M
—
Modalities
text
Released
24 May 2023
License: apache-2.0 · tiiuae/falcon-40b
AI summary
● machine-written
Falcon-40B: 40B open-source LLM trained on 1T refined web tokens
Falcon-40B is a 40-billion parameter causal decoder-only model developed by TII, trained on over 1 trillion tokens of RefinedWeb data enhanced with curated corpora. Released under Apache 2.0 license for both commercial and research use, it features rotary positional embeddings, multiquery attention, and FlashAttention for efficient inference. The model supports text generation, chatbot applications, content creation, and code completion tasks across English and 10 additional languages.
What's new
- Trained on 1,000B tokens of RefinedWeb enhanced with curated corpora
- Implements rotary positional embeddings and multiquery attention mechanisms
- Uses FlashAttention for reduced computational complexity and faster inference
- Parallel attention/MLP decoder architecture with two-layer normalization
- Available under Apache 2.0 license, free for commercial and research use
Best for
Enterprise chatbot and conversational AI systemsContent generation and multi-language text productionCode completion and developer toolingText generation and semantic search applications