3FS (Fire-Flyer 文件系统)
一种专为 AI 训练和推理工作负载设计的高性能分布式文件系统，提供分离式架构、强一致性和熟悉的文件接口。
先进存储技术
3FS 的关键特性
3FS 结合现代存储技术和创新设计原则，为 AI 工作负载提供卓越性能。
- Disaggregated Architecture
Combines the throughput of thousands of SSDs with the network bandwidth of hundreds of storage nodes, allowing applications to access storage resources in a location-agnostic manner.
- Strong Consistency
Implements Chain Replication with Apportioned Queries (CRAQ) to provide strong consistency guarantees, making application code simple and easy to reason about.
- Familiar File Interface
Developed a stateless metadata service backed by a transactional key-value store (like FoundationDB), providing a familiar file interface without requiring learning new storage APIs.
- High Performance
Achieves peak throughput of approximately 6.6 TiB/s in read stress tests on a 180-node cluster, with efficient garbage collection operations.
- RDMA Networking
Leverages high-performance RDMA networking for efficient data transfer between storage and compute nodes, minimizing latency and maximizing throughput.
- Efficient KV Cache
Provides a cost-effective alternative to DRAM caching with high throughput and larger capacity, ideal for AI inference workloads.
系统设计
3FS 架构
3FS 采用模块化设计，包含存储、元数据管理、客户端接口和复制等独立组件。
存储层
存储层负责数据块的物理存储和检索。它利用现代 SSD 和 RDMA 网络提供高吞吐量、低延迟的数据访问。存储节点以分离式方式组织，允许存储和计算资源独立扩展。
元数据服务
元数据服务管理文件系统结构和元数据。它实现为由事务性键值存储（如 FoundationDB）支持的无状态服务。这种设计提供强一致性保证，同时允许元数据服务水平扩展以处理大型文件系统。
性能基准
3FS 性能
3FS 在各种工作负载下提供卓越性能，从高吞吐量读取到复杂数据处理任务。
Peak Read Throughput
6.6 TiB/s
Achieved in read stress tests on a 180-node cluster
GraySort Benchmark
3.66 TiB/min
Sorted 110.5 TiB of data in 30 minutes 14 seconds
KV Cache Performance
40 GiB/s
Peak read throughput for KV cache operations
Cluster Size
180 nodes
Scale tested with hundreds of storage nodes
应用场景
3FS 使用场景
3FS 针对 AI 工作流程的不同阶段进行了优化，从数据准备到推理。
- Data Preparation
Organize outputs from data processing pipelines into hierarchical directory structures, efficiently managing large numbers of intermediate outputs.
- Data Loaders
Eliminate the need for prefetching or shuffling datasets by supporting random access to training samples across compute nodes.
- Checkpoint Saving
Support high-throughput parallel checkpoint saving for large-scale training, ensuring model progress is safely preserved.
- Inference KV Cache
Provide a cost-effective alternative to DRAM caching with high throughput and larger capacity for AI inference workloads.
常见问题
找不到您要找的答案？查看我们的 GitHub 仓库或联系我们的团队。
- What is 3FS?
- 3FS (Fire-Flyer File System) is a high-performance distributed file system designed specifically for AI training and inference workloads. It offers a disaggregated architecture, strong consistency guarantees, and familiar file interfaces.
- How does 3FS differ from other distributed file systems?
- 3FS is specifically optimized for AI workloads, with a focus on high throughput, strong consistency, and efficient handling of both small and large files. Its disaggregated architecture allows for independent scaling of storage and compute resources, and it leverages modern technologies like RDMA networking and transactional key-value stores for metadata management.
- What programming languages is 3FS implemented in?
- 3FS is primarily implemented in C++ (87.0%), with additional components in Rust (4.3%) and Python (2.1%). This combination provides both high performance and safety for critical system components.
- What kind of performance can I expect from 3FS?
- 3FS has demonstrated peak read throughput of approximately 6.6 TiB/s in stress tests on a 180-node cluster. In the GraySort benchmark, it sorted 110.5 TiB of data in 30 minutes 14 seconds, achieving an average throughput of 3.66 TiB/minute. For KV cache operations, it can reach peak read throughput of 40 GiB/s.
- What are the main components of 3FS?
- 3FS consists of several key components: a storage layer for handling data blocks, a metadata service for managing file system structure, client interfaces (including FUSE), and a replication system implementing the CRAQ protocol for strong consistency.
- Is 3FS open source?
- Yes, 3FS is available as an open-source project on GitHub at https://github.com/deepseek-ai/3FS. It is developed by DeepSeek AI to support high-performance AI infrastructure needs.