noun
entropy
A measure of uncertainty in the possible outcomes of an information source.
The source's entropy measured uncertainty in its output.
Etymology
Entropy was coined around a Greek word for transformation. Information theory borrowed the term for uncertainty, using a mathematical form closely related to the physical measure.
More examples
- A predictable message had lower entropy than a random one.
- The coding scheme approached the source's entropy limit.
- They calculated entropy from the outcome probabilities.
Forms
entropy
Similar words
- uncertainty
- Uncertainty is general; information entropy quantifies it from outcome probabilities.
- redundancy
- Redundancy repeats predictable information; higher predictability generally lowers entropy per symbol.