Cloudian HyperStore Data Recovery
Cloudian HyperStore is an S3-compatible object storage platform that uses Apache Cassandra as its distributed metadata store alongside its own storage layer for object data. Recovery cases typically involve the health of the Cassandra ring holding bucket and object metadata, as well as the replication or erasure coding storage policy applied to affected buckets.
Platform Lineage and Naming
Cloudian built HyperStore as an S3-API-compatible object storage platform, distinguishing itself early on by using Apache Cassandra, a widely used open-source distributed database, as the backing store for object and bucket metadata rather than a proprietary metadata design.
HyperStore has been offered as both a software product deployable on qualified or commodity hardware and as pre-configured appliances, with storage policies allowing administrators to choose between full replication and erasure coding on a per-bucket or per-group basis to balance durability against capacity efficiency.
- HyperStore File System (HSFS) — Cloudian's on-disk object data layout
- Cloudian HyperStore — the full product name
Generations and Models We Evaluate
| Generation / family | Models |
|---|---|
| Deployment | Software on qualified/commodity servers, Cloudian appliances |
| Metadata store | Apache Cassandra ring, cluster-wide |
| Storage policies | Replication (multiple full copies), erasure coding (configurable schemes) |
Architecture and Data Layout
HyperStore nodes form a Cassandra ring that stores metadata describing buckets, objects, versions and their physical locations, separate from the HyperStore file system layer that holds the actual object data on disk; Cassandra's own replication settings for metadata are configured independently from the storage policy applied to object data itself.
Object data is written according to a storage policy selected per bucket or storage policy group — either straightforward replication across nodes or erasure coding that splits objects into data and parity fragments — allowing different durability and efficiency trade-offs within the same cluster.
Because metadata (in Cassandra) and object data (in the HyperStore file system layer) are managed by separate subsystems, a healthy Cassandra ring with damaged object data — or vice versa — are both possible and require different reconstruction approaches.
Protocols and formats: S3 API, Object Lock (WORM) support, IAM-style access control
- Replication-based storage policies keep multiple full object copies across nodes, while erasure-coding policies split objects into fragments distributed for space efficiency; the choice affects how many node or drive failures a given bucket's data can tolerate.
- Cassandra metadata replication factor is configured separately from object data storage policy, so metadata and data availability should be assessed independently during a recovery evaluation.
Failure Scenarios
Logical failures
- Cassandra ring inconsistency or corruption affecting bucket/object metadata lookups
- Accidental bucket or object deletion, including where object lock/versioning was not enabled
- Storage policy misconfiguration leaving data under-protected for the actual failure scenario
- Failed cluster upgrade leaving Cassandra schema or HyperStore file system layer out of sync
Hardware failures
- Multiple node or drive failures exceeding the configured replication or erasure coding policy
- Cassandra ring node failures affecting metadata quorum
- Network partition disrupting communication between HyperStore nodes during rebuild
What Not To Do Before an Evaluation
- Do not run rebuilds, reconstructions or re-initialisations against an array that has already lost more drives than its protection level allows.
- Do not recreate pools, aggregates, disk groups, storage pools or clusters — these operations write new metadata over the structures a recovery needs.
- Do not swap drives between slots, and do not reorder shelves. Record the original slot and shelf positions before removing anything.
- Do not run file-system repair tools against production volumes before the underlying storage layer has been evaluated.
- Do not restore a backup or replication set over the affected volumes until the recovery scope has been assessed.
- Do not eradicate deleted volumes or empty recycle/destroyed states on platforms that hold deleted data for a retention window.
Our Evaluation and Recovery Process
- Intake and platform identification — array model, generation, firmware, protection layout and the sequence of events that led to the failure.
- Read-only evaluation of the media and array structures, including assessment of drive health and the extent of any physical damage.
- Forensic imaging of all contributing media, with cleanroom work where drives require it. Originals are preserved unaltered.
- Reconstruction of the storage layer — pools, aggregates, parity groups, chunklets, extent groups or objects — from the images.
- Extraction of the layers above: file systems, virtual machines, databases, mailboxes and shares.
- Verification against a file list and customer-nominated critical data, followed by secure return on encrypted media.
Frequently Asked Questions
If Cassandra is corrupted, is the object data itself lost?
Not necessarily. Object data in the HyperStore file system layer can remain intact even if Cassandra metadata is damaged, though reconstructing which objects the data belongs to depends on how much metadata can be recovered.
Does object lock protect against accidental deletion?
Object lock and versioning, where enabled, protect specific objects from deletion or overwrite for a configured retention period, but only for data that had those settings applied before the incident.
How does the storage policy affect how many failures we can tolerate?
Replication policies tolerate failures up to the number of extra full copies kept, while erasure coding policies tolerate failures based on the configured data-to-parity fragment ratio; the specific policy applied to affected buckets determines this.