An Integrated Distributed Storage Design Offering Data Retrievability and Recoverability Using Soft Decision Decoding of Block Codes

Abstract

Active distributed storages need to assure both consistency and dynamic data support, in addition to availability, confidentiality and resiliency. Further, since storage durability suffers in untrusted and unreliable environments, it becomes crucial to (a) select the most reliable set of servers to assure data retrievability and (b) dynamically identify errant servers and restore the data to ensure data recoverability. We address the issues of concurrency, consistency, dynamic data support, data share repair, and trust management in providing persistent storage and access. The paper focuses primarily on erasure coded distributed storages (storages employing erasure coding for data dispersal). Integration of Quorum based approach using Notification propagation, with a reliability model based on server trust-reputation forms the comprehensive design proposed. Treating servers and their data shares equally at data reconstructions during data retrievals is rather inadequate in untrusted environments. The design provides a suitable platform for use of Soft Decision Decoding to overcome this inadequacy. The design has been validated by the simulation, study, and analysis carried out for Reed Solomon coded storage with varying levels of resiliency and concurrency. The proposed design can be suitably adapted in typical distributed information storages catering to global networked audience in public, untrusted, and unreliable operating environments.