Скачать с ютуб The Era of 1-bit LLMs | BitNet | Microsoft | Paper Explained в хорошем качестве

The Era of 1-bit LLMs | BitNet | Microsoft | Paper Explained

BitNet model introduced in 2023 initiated the era of 1-bit LLMs. This model marks a significant shift away from traditional, high-precision, computationally expensive LLMs to low-precision and cost-effective LLMs. BitNet b1.58 is a new 1-bit LLM variant based on BitNet and this model restricts each weight to just three possible values: -1, 0, or 1. The BitNet b1.58 model can match the performance (perplexity and end-task outcomes) of full-precision (FP16 or BF16) LLMs while using the same model size and training data. Moreover, BitNet b1.58 dramatically reduces latency, memory usage, and energy consumption compared to traditional LLMs.