Sparse coding is the development of a distributed binary representation of arbitrary entities. It needs two requirements:
- sparseness means that there are many more 0s than 1s in the codewords
- given a notion of similarity or dissimilarity on the set of entities, for example a distance or metric, the code should be similarity preserving, i.e., similar entities should set similar codewords (with respect to the Hamming distance).
(Excerpt taken from www.informatik.uni-ulm.de/ni/forschung/forschungsthemen/sparsecoding.html)
Note that in sparse
coding a
pattern or entity is usually represented with few bits, over the length of the codewords. On the contrary, in so-called
dense coding, the bits involved are on the average half of the codeword.