Oracle ZFS Storage Appliance Data Recovery
The Oracle ZFS Storage Appliance family — descended from the Sun Storage 7000 Unified Storage line — presents NFS, SMB and block shares from ZFS storage pools managed through the Fishworks browser-based interface. Recovery cases typically follow a controller failure, a pool import problem, or disk shelves that have been separated from their original head and reassembled in the wrong order.
Platform Lineage and Naming
Sun Microsystems launched the Sun Storage 7000 Unified Storage series — the 7110, 7210, 7310 and 7410 — built on commodity x86 hardware, Solaris and ZFS, managed through the Fishworks appliance kit software stack. After Oracle's acquisition of Sun, the line continued as the Oracle ZFS Storage Appliance.
Subsequent generations were badged ZS3-2 and ZS3-4, followed by ZS4-4, ZS5-2 and ZS5-4, then ZS7-2 and the current ZS9-2. Each generation retained the same underlying ZFS pool concepts while increasing controller performance, cache sizing and supported shelf counts.
Across generations the appliance has remained a ZFS-based unified storage platform rather than a proprietary block controller, which means the same ZFS pool, dataset and snapshot concepts found in open ZFS deployments apply directly to recovery work on these systems.
- Sun Storage 7000 Unified Storage series (7110, 7210, 7310, 7410)
- Fishworks — the internal project name for the appliance kit software
- Oracle ZFS Storage ZS3-2, ZS3-4, ZS4-4, ZS5-2, ZS5-4, ZS7-2, ZS9-2
- Sun ZFS Storage Appliance — the branding used immediately after acquisition
- BUI (Browser User Interface) — the Fishworks management console
Generations and Models We Evaluate
| Generation / family | Models |
|---|---|
| Sun 7000 series | 7110, 7210, 7310, 7410 |
| Early Oracle generations | ZS3-2, ZS3-4, ZS4-4 |
| Mid generations | ZS5-2, ZS5-4 |
| Current generations | ZS7-2, ZS9-2 |
| Disk shelves | Sun Disk Shelf (J4400 family), DE2-24, DE3-24 SAS expansion shelves |
Architecture and Data Layout
Storage is organised into ZFS pools built from one or more virtual devices (vdevs) — mirrors, RAID-Z1, RAID-Z2 or RAID-Z3 groups — with the pool metadata, checksums and copy-on-write transaction structure of ZFS underpinning every share and LUN presented to clients.
A separate intent log device (ZIL, often called a log or slog device) accelerates synchronous writes, and an L2ARC read cache extends the in-memory ARC onto flash to absorb read-heavy workloads. Neither device holds the primary copy of data, but a missing or corrupted log device can complicate replay of the most recent transactions during import.
Data is organised into shares and projects — ZFS filesystems and LUNs grouped for quota, replication and snapshot policy — with snapshots and clones providing point-in-time and writable copies without duplicating unchanged blocks.
Higher-end configurations use clustered head pairs (active/active or active/passive controller nodes) attached to shared DE2 or DE3 SAS disk shelves, with pool ownership information recorded on the disks themselves so that either head can import the pool during failover.
Protocols and formats: NFS, SMB/CIFS, iSCSI, Fibre Channel, HTTP/WebDAV, ZFS send/receive replication
- Pool layout follows standard ZFS vdev concepts — mirrored pairs, RAID-Z1, RAID-Z2 or RAID-Z3 — and the exported pool configuration (which disks belong to which vdev, in which shelf slot) must be established before any reconstruction is attempted.
- Because ZFS records pool, vdev and disk identity in on-disk labels rather than relying purely on physical slot order, disks that have been removed and reinserted out of their original order can often still be identified correctly — provided none have been overwritten or reinitialised.
- Shelf identity matters in clustered configurations because pool ownership and SAS reservations are tracked per shelf; shelves relocated to a different pair of heads, or split apart from their sibling shelf in a multi-shelf pool, need to be reassembled in a way that preserves the original vdev membership.
Failure Scenarios
Logical failures
- Pool that fails to import after an unclean shutdown or firmware update
- Deleted shares, projects, snapshots or clones
- Failed Fishworks software upgrade or configuration corruption
- Corrupted pool metadata following a multi-disk event within tolerance
- Accidental pool destroy or re-provisioning of shares
- Replication or clustering failover leaving an inconsistent pool state
Hardware failures
- Multiple drive failures within a vdev beyond RAID-Z tolerance
- Failed head node with an unclear or split-brain failover to its cluster partner
- DE2/DE3 shelf controller, expander or SAS cable failure
- Failed or removed ZIL/log device coinciding with an unclean shutdown
- Disks separated from their original shelf and head pairing during relocation
Encryption and credentials
Where ZFS encryption or appliance-level encryption is enabled, key material is tied to the appliance configuration rather than the disks alone, so the controller configuration should be preserved alongside the shelves whenever encryption is in use.
What Not To Do Before an Evaluation
- Do not run rebuilds, reconstructions or re-initialisations against an array that has already lost more drives than its protection level allows.
- Do not recreate pools, aggregates, disk groups, storage pools or clusters — these operations write new metadata over the structures a recovery needs.
- Do not swap drives between slots, and do not reorder shelves. Record the original slot and shelf positions before removing anything.
- Do not run file-system repair tools against production volumes before the underlying storage layer has been evaluated.
- Do not restore a backup or replication set over the affected volumes until the recovery scope has been assessed.
- Do not eradicate deleted volumes or empty recycle/destroyed states on platforms that hold deleted data for a retention window.
Our Evaluation and Recovery Process
- Intake and platform identification — array model, generation, firmware, protection layout and the sequence of events that led to the failure.
- Read-only evaluation of the media and array structures, including assessment of drive health and the extent of any physical damage.
- Forensic imaging of all contributing media, with cleanroom work where drives require it. Originals are preserved unaltered.
- Reconstruction of the storage layer — pools, aggregates, parity groups, chunklets, extent groups or objects — from the images.
- Extraction of the layers above: file systems, virtual machines, databases, mailboxes and shares.
- Verification against a file list and customer-nominated critical data, followed by secure return on encrypted media.
Frequently Asked Questions
The pool will not import after a head failure. Is the data still there?
Often yes — ZFS pool metadata is self-describing, so a failed import does not necessarily mean the data is gone. An evaluation examines the pool labels and transaction history before any write-based repair is attempted.
Do the disks need to stay in their original shelf slots?
It helps, but ZFS records vdev and pool membership on each disk's own labels, so disks reordered within the same pool can usually still be correctly identified. Mixing disks between different pools or systems is the higher-risk scenario.
Can a clustered head pair complicate recovery?
Yes. Shared DE2/DE3 shelves and SAS reservations mean both heads' state matters — evaluation typically requires access to both controllers and all shelves that made up the affected pool.