Metadata describes the attributes of user data, such as data location distribution, health status, and composition pattern. In distributed storage systems, metadata services are often challenging to design. If improperly designed, they can easily become a system bottleneck or lead to correctness issues.
The ABS metadata design follows the fundamental principle of enhancing availability and metadata processing performance while maintaining a reasonable degree of security and consistency. It has the following features:
Large-granularity data unit: ABS uses larger data blocks as the basic data unit to reduce metadata volume and corresponding memory consumption, ensuring all metadata can be stored in memory, thereby improving access efficiency.
Centralized metadata management: By centrally managing metadata and collecting the status of data in the cluster, the system can precisely control data allocation, recovery, migration and other operations within the cluster.
Replication mechanism: The metadata is stored as a complete replica across all master nodes. This ensures that even if some nodes fail, the metadata remains intact, guaranteeing high reliability.
Metadata caching: By caching metadata on the Access, interactions between components and Meta are reduced, thereby lowering the Meta load and improving I/O processing efficiency.
