On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus - Jan A. Bergstra; John V. Tucker (Entropy)
Classic formulae for entropy and cross-entropy contain operations <semantics><mstyle scriptlevel="0" displaystyle="true">x0</mstyle></semantics> and <semantics>log2x</semantics> that are not defined on all inputs. This can lead to calculations with problematic subexpressions such as <semantics>0log20</semantics> and uncertainties in large scale calculations; partiality also introduces complications in logical analysis. Instead of adding conventions or splitting formulae into cases, we create a new algebra of real numbers with two symbols <semantics>±∞</semantics> for signed infinite values and a symbol named ⊥ for the undefined. In this resulting arithmetic, entropy, cross-entropy, Kullback–Leibler divergence, and Shannon divergence can be expressed without concerning any further conventions. The algebra may form a basis for probability theory more generally.