One of the practical applications of information theory is the derivaiton of error correcting codes. There are strategies to deal with errors introduced by noise.

As a first step, we can develop error detecting codes which will warn us that an error has occured. The simplest of these techniques is the parity-check scheme, in where a single 0 or 1 is added to end of the data block to indicate where number of bits in the block are even or odd. The resulting block-plus-parity-check therefore will always have an even number of ones. If the data is contaminated at only one place during transmission, then the received data will have an odd number of 1’s. This tells the receiver that the data has been affected by noise.

More advanced error detecting schemes will be able to detect more errors, at the cost of appending more “parity-check” bits at end of the data. The cost to pay for error correcting is that data we have to transmit or store mode data. For example, the real data capacity of a CDROM is around 1Gb, but because an error correcting code is used the capacity available to you is 700Mb. We lose 30% of the capacity, but we get error correction, so that the CD will still play even after a few scratches.

• $\mathcal{C}=[n,k,d]$: A code
• $n$: codeword size
• $k$: message size
• $d$: distance
• $R$: rate