On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus

On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus

Classic formulae for entropy and cross-entropy contain operations <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mstyle scriptlevel="0" displaystyle="true"><mfrac><mi>x</mi&g...

Full description

Saved in:
Bibliographic Details
Main Authors: Jan A. Bergstra, John V. Tucker
Format: Article
Language:English
Published: MDPI AG 2025-01-01
Series:Entropy
Subjects:
partial formulae
fracterm calculus
transreals
entropic transreals
peripheral numbers
entropy
Online Access:https://www.mdpi.com/1099-4300/27/1/31
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Classic formulae for entropy and cross-entropy contain operations <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mstyle scriptlevel="0" displaystyle="true"><mfrac><mi>x</mi><mn>0</mn></mfrac></mstyle></semantics></math></inline-formula> and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msub><mo form="prefix">log</mo><mn>2</mn></msub><mi>x</mi></mrow></semantics></math></inline-formula> that are not defined on all inputs. This can lead to calculations with problematic subexpressions such as <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0</mn><msub><mo form="prefix">log</mo><mn>2</mn></msub><mn>0</mn></mrow></semantics></math></inline-formula> and uncertainties in large scale calculations; partiality also introduces complications in logical analysis. Instead of adding conventions or splitting formulae into cases, we create a new algebra of real numbers with two symbols <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mo>±</mo><mo>∞</mo></mrow></semantics></math></inline-formula> for signed infinite values and a symbol named ⊥ for the undefined. In this resulting arithmetic, entropy, cross-entropy, Kullback–Leibler divergence, and Shannon divergence can be expressed without concerning any further conventions. The algebra may form a basis for probability theory more generally.
ISSN:1099-4300

Similar Items