On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus
Classic formulae for entropy and cross-entropy contain operations <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mstyle scriptlevel="0" displaystyle="true"><mfrac><mi>x</mi&g...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/27/1/31 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832588551873626112 |
|---|---|
| author | Jan A. Bergstra John V. Tucker |
| author_facet | Jan A. Bergstra John V. Tucker |
| author_sort | Jan A. Bergstra |
| collection | DOAJ |
| description | Classic formulae for entropy and cross-entropy contain operations <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mstyle scriptlevel="0" displaystyle="true"><mfrac><mi>x</mi><mn>0</mn></mfrac></mstyle></semantics></math></inline-formula> and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msub><mo form="prefix">log</mo><mn>2</mn></msub><mi>x</mi></mrow></semantics></math></inline-formula> that are not defined on all inputs. This can lead to calculations with problematic subexpressions such as <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0</mn><msub><mo form="prefix">log</mo><mn>2</mn></msub><mn>0</mn></mrow></semantics></math></inline-formula> and uncertainties in large scale calculations; partiality also introduces complications in logical analysis. Instead of adding conventions or splitting formulae into cases, we create a new algebra of real numbers with two symbols <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mo>±</mo><mo>∞</mo></mrow></semantics></math></inline-formula> for signed infinite values and a symbol named ⊥ for the undefined. In this resulting arithmetic, entropy, cross-entropy, Kullback–Leibler divergence, and Shannon divergence can be expressed without concerning any further conventions. The algebra may form a basis for probability theory more generally. |
| format | Article |
| id | doaj-art-58bdde8331d64de0b05419d1e3eab0b9 |
| institution | Kabale University |
| issn | 1099-4300 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj-art-58bdde8331d64de0b05419d1e3eab0b92025-01-24T13:31:44ZengMDPI AGEntropy1099-43002025-01-012713110.3390/e27010031On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm CalculusJan A. Bergstra0John V. Tucker1Informatics Institute, University of Amsterdam, Science Park 900, 1098 XH Amsterdam, The NetherlandsDepartment of Computer Science, Bay Campus, Fabian Way, Swansea University, Swansea SA1 8EN, UKClassic formulae for entropy and cross-entropy contain operations <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mstyle scriptlevel="0" displaystyle="true"><mfrac><mi>x</mi><mn>0</mn></mfrac></mstyle></semantics></math></inline-formula> and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msub><mo form="prefix">log</mo><mn>2</mn></msub><mi>x</mi></mrow></semantics></math></inline-formula> that are not defined on all inputs. This can lead to calculations with problematic subexpressions such as <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0</mn><msub><mo form="prefix">log</mo><mn>2</mn></msub><mn>0</mn></mrow></semantics></math></inline-formula> and uncertainties in large scale calculations; partiality also introduces complications in logical analysis. Instead of adding conventions or splitting formulae into cases, we create a new algebra of real numbers with two symbols <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mo>±</mo><mo>∞</mo></mrow></semantics></math></inline-formula> for signed infinite values and a symbol named ⊥ for the undefined. In this resulting arithmetic, entropy, cross-entropy, Kullback–Leibler divergence, and Shannon divergence can be expressed without concerning any further conventions. The algebra may form a basis for probability theory more generally.https://www.mdpi.com/1099-4300/27/1/31partial formulaefracterm calculustransrealsentropic transrealsperipheral numbersentropy |
| spellingShingle | Jan A. Bergstra John V. Tucker On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus Entropy partial formulae fracterm calculus transreals entropic transreals peripheral numbers entropy |
| title | On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus |
| title_full | On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus |
| title_fullStr | On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus |
| title_full_unstemmed | On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus |
| title_short | On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus |
| title_sort | on defining expressions for entropy and cross entropy the entropic transreals and their fracterm calculus |
| topic | partial formulae fracterm calculus transreals entropic transreals peripheral numbers entropy |
| url | https://www.mdpi.com/1099-4300/27/1/31 |
| work_keys_str_mv | AT janabergstra ondefiningexpressionsforentropyandcrossentropytheentropictransrealsandtheirfractermcalculus AT johnvtucker ondefiningexpressionsforentropyandcrossentropytheentropictransrealsandtheirfractermcalculus |