DDN SFA & EXAScaler Data Recovery
DDN's SFA storage arrays and EXAScaler, DDN's distribution of the Lustre parallel file system, together form a widely deployed HPC storage stack. Recovery cases typically involve either the declustered RAID protection within an SFA array or the Lustre-level structures — metadata targets, management targets and object storage targets — layered on top of it.
Platform Lineage and Naming
DDN's SFA (Storage Fusion Architecture) product line spans multiple generations of block storage arrays, from earlier SFA models through SFA7990, SFA400NVX and the ES200NVX series, each generation increasing drive density and adding NVMe-based configurations while retaining SFAOS as the underlying array operating environment and its declustered RAID protection scheme.
EXAScaler is DDN's commercially supported distribution of the open-source Lustre parallel file system, layered on top of SFA (or other qualified) block storage to provide the metadata, management and object storage target structure that Lustre clients see as a single POSIX namespace; it is widely used in HPC, AI training and research computing environments where SFA arrays are commonly the underlying block layer.
- SFAOS — the operating environment running on DDN SFA controllers
- EXAScaler — DDN's supported Lustre distribution and appliance line
- MDT — Metadata Target, MGT — Management Target, OST — Object Storage Target (Lustre components)
- ldiskfs and ZFS — the two OSD (object storage device) backend options for Lustre targets
Generations and Models We Evaluate
| Generation / family | Models |
|---|---|
| SFA array generations | SFA7990, SFA400NVX, ES200NVX and related SFA/ES series |
| EXAScaler | EXAScaler appliances and software builds layered on SFA or qualified block storage |
| Lustre backend options | ldiskfs (ext4-derived) and ZFS-based OSD backends for MDT/OST |
Architecture and Data Layout
SFA arrays present block storage protected by declustered RAID, which spreads data, parity and spare capacity evenly across all drives in a pool rather than confining a rebuild to a fixed set of drives in a traditional RAID group; this generally reduces rebuild time and the performance impact of a drive failure compared with conventional RAID.
EXAScaler's Lustre file system is built from several distinct target types: one or more Metadata Targets (MDT) storing file system namespace and metadata, a Management Target (MGT) holding cluster configuration, and many Object Storage Targets (OST) holding the actual striped file data, all typically backed by SFA-provided LUNs.
Each Lustre target is formatted using one of two OSD backends — ldiskfs, an ext4-derived on-disk format, or ZFS, using OpenZFS pools and datasets — and the choice affects both normal operational characteristics and how a damaged target must be approached during recovery, since the two backends have materially different on-disk structures.
Protocols and formats: Lustre native client protocol, LNet (Lustre networking), NFS/SMB (via Lustre gateway nodes, where deployed)
- Declustered RAID at the SFA layer distributes both data/parity and spare capacity across all drives in a pool, changing how a multi-drive failure is analysed compared with a fixed RAID group.
- Lustre striping spreads individual file data across multiple OSTs for performance, meaning a single file's complete recovery can depend on the availability of several OSTs rather than one.
- MDT damage affects the file system's namespace and directory structure, while OST damage affects file content within the stripes stored on the affected targets — the two failure modes require different reconstruction approaches.
Failure Scenarios
Logical failures
- MDT metadata corruption affecting namespace and directory resolution across the file system
- MGT loss or corruption disrupting cluster configuration and target registration
- ldiskfs or ZFS pool corruption on one or more Lustre targets
- Failed Lustre software upgrade leaving targets unable to mount or clients unable to connect
- Accidental file or directory deletion within the Lustre namespace
Hardware failures
- Multiple drive failures within an SFA declustered RAID pool exceeding its protection level
- SFA controller failure affecting LUNs backing one or more Lustre targets
- Concentrated OST failures affecting files striped across the affected targets
- LNet network fabric failures disrupting client-to-target or target-to-target communication
What Not To Do Before an Evaluation
- Do not run rebuilds, reconstructions or re-initialisations against an array that has already lost more drives than its protection level allows.
- Do not recreate pools, aggregates, disk groups, storage pools or clusters — these operations write new metadata over the structures a recovery needs.
- Do not swap drives between slots, and do not reorder shelves. Record the original slot and shelf positions before removing anything.
- Do not run file-system repair tools against production volumes before the underlying storage layer has been evaluated.
- Do not restore a backup or replication set over the affected volumes until the recovery scope has been assessed.
- Do not eradicate deleted volumes or empty recycle/destroyed states on platforms that hold deleted data for a retention window.
Our Evaluation and Recovery Process
- Intake and platform identification — array model, generation, firmware, protection layout and the sequence of events that led to the failure.
- Read-only evaluation of the media and array structures, including assessment of drive health and the extent of any physical damage.
- Forensic imaging of all contributing media, with cleanroom work where drives require it. Originals are preserved unaltered.
- Reconstruction of the storage layer — pools, aggregates, parity groups, chunklets, extent groups or objects — from the images.
- Extraction of the layers above: file systems, virtual machines, databases, mailboxes and shares.
- Verification against a file list and customer-nominated critical data, followed by secure return on encrypted media.
Frequently Asked Questions
Does an OST failure affect the whole Lustre file system?
It mainly affects files that had stripes on the failed OST(s); other files striped entirely on unaffected OSTs typically remain accessible, though this depends on the striping configuration used.
Is recovery different for ldiskfs versus ZFS-backed targets?
Yes. The two OSD backends have materially different on-disk structures, so the approach to reconstructing a damaged MDT, MGT or OST depends on which backend was configured for that target.
Does declustered RAID make SFA arrays harder to recover than traditional RAID?
Not inherently — declustered RAID spreads data and spare capacity differently across drives, which changes the analysis approach but does not by itself make a recovery evaluation more difficult.