DeepSeek推出NSA,用于超快速的长上下文训练和推理

Blockbeats
18 Feb

BlockBeats 消息,2 月 18 日,DeepSeek 宣布推出 NSA,这是一种与硬件一致且本机可训练的稀疏注意力机制,用于超快速的长上下文训练和推理。

通过针对现代硬件的优化设计,NSA 加快了推理速度,同时降低了预训练成本,而不会影响性能。在一般基准测试、长上下文任务和基于指令的推理上,它的表现与完全注意力模型相当甚至更好。(金十)

Disclaimer: Investing carries risk. This is not financial advice. The above content should not be regarded as an offer, recommendation, or solicitation on acquiring or disposing of any financial products, any associated discussions, comments, or posts by author or other users should not be considered as such either. It is solely for general information purpose only, which does not consider your own investment objectives, financial situations or needs. TTM assumes no responsibility or warranty for the accuracy and completeness of the information, investors should do their own research and may seek professional advice before investing.

Most Discussed

  1. 1
     
     
     
     
  2. 2
     
     
     
     
  3. 3
     
     
     
     
  4. 4
     
     
     
     
  5. 5
     
     
     
     
  6. 6
     
     
     
     
  7. 7
     
     
     
     
  8. 8
     
     
     
     
  9. 9
     
     
     
     
  10. 10