How does erasure coding work

If we want to have better storage efficiency, we use a higher ratio of data segments to data protection segments. What makes erasure coding so powerful is that it gives us a ready method for scaling our data protection levels as hard drives continue to grow. The equations are independent each equation cannot be created from the other equations in the solution set. For example, these two equations are not independent:.

The second equation can be created by multiplying each side of the first equation by two. Every value of X and Y that solves the first equation will also solve the second equation. Because they are independent, there cannot be more than one set of values for X and Y that solves both equations. Depending on which solution you go with, the erasure coding may occur on a per file or per object basis or may occur as part of the RAID implementation. A given workload or data type may be better suited to one of these methods than it is to another method.

But, that topic is big enough that it probably deserves to be in its own blog post. To sum everything up no pun intended , erasure coding and parity are both the names of mathematical algorithms which are designed to recover data that is erased or lost.

As hard drives get bigger, the rebuild times have grown from hours to days, and bigger hard drives are on the horizon. Parity-based solutions are relatively fast, but scaling has proven difficult. Erasure coding gives us a ready model to scale beyond double-drive failures. If you would like to learn more about enterprise storage solutions, SAN storage, object storage, etc.

As I said above, I do math for fun. So, in the hopes that somebody wants to see the math, here you go…. Integers are a lot easier to type, and the principles are largely the same. Computers would probably use multi-dimensional matrix-based math, and while matrices can make computation faster, they make explaining what is going on much more difficult. The math would be different depending on which pieces of data were lost.

The equations we use to calculate the data protection values do not need to be the ones I supplied here. They can be any set of mathematically independent equations. This post is powered by Mix Digital Marketing. What is Erasure Coding June 12, In Dasher Solutions , Events , News. By Kelsey Outram. The utilization of erasure coding prevents the loss of data due to system failure or disaster.

In simple words Erasure coding EC is one of the methods of data protection through which the data is broken into sectors. Then they are expanded and encoded with redundant data pieces and stored across different storage media. Erasure coding adds the redundancy to the system that tolerates failures.

Erasure coding takes original data and encodes it in such a way that, when you require the data it only needs the subset of the pieces to recreate the original information. The encoding process will create a series of equations.

When you want to recreate the object you require any two of those three equations, to be able to be decoded. So when you solve the equations together, you will be able to get the values for x and y. We have 3 equations but we can get the original information from two of them. The data protection scheme that breaks data into fragments, encodes them and stores them across multiple locations is Erasure coding. Erasure coding and RAID are sometimes mixed up but they are very much different from each other.

RAID allows data to be stored at different locations and it protects against drive failures. In erasure coding, the data is broken in parts, then expanded and encoded.

After that the data segments are kept in multiple locations. You keep data intact but copy and store it in multiple devices and locations. In erasure coding, you break the data apart, expand and encode it, and store it in various places. Erasure coding also has several benefits over RAID. One is that depending on how it is configured, erasure coding can rebuild failed disks much faster than RAID.

Plotting storage strategies requires organizations to consider several factors. They should consider protecting against data loss and providing disaster recovery. Storage strategies can come in different forms, including data replication, RAID usage, and erasure coding. Like any technology, though, erasure coding needs to adapt to industry changes. What is Erasure Coding?

A short definition of Erasure Coding.