Introduction baxkus extended

Author: ZigaSajovic

latex/Introduction.tex

@@ -5,11 +5,11 @@ \section{Introduction}
 properties \cite{backus}. This issue has partially been addressed by algebraic data types employed by functional programming, where a mapping has been shown between grammars and semirings \cite{7Trees}.
 Yet due to the lack of inverses (hence the semiring structure) they remain limited in the algebraic manipulations they are allowed to employ \cite{complexCat}.
 
-As computer programs are the dominant tool for modern problem solving, the need for examining the analytic properties of programs led to the development of various tools for dealing with derivatives of computer programs (automatic differentiation). Yet the developed techniques are only efficient ways of calculating derivatives, and do not construct any meaningful algebraic structure over differentiable programs. As such, there is still a need for a framework that properly captures the analytic properties of differentiable programs and provides higher-order constructs that can reason about them. 
+As computer programs are the dominant tool for modern problem solving, the need for examining the analytic properties of programs led to the development of various tools for dealing with derivatives of computer programs (automatic differentiation). Yet the developed techniques are only efficient ways of calculating derivatives, and do not construct any meaningful algebraic structure over differentiable programs. As such, there is still a need for a framework that properly captures the analytic properties of differentiable programs and provides higher-order constructs that can reason about them. Such a framework can be provided by \emph{Operational Calculus}, because \emph{unlike von Neumann languages, the language of ordinary algebra is suitable both for stating its laws and for transforming an equation into its solution, all within the language} \cite{backus}.
 
 The ideas of functional programming and automatic differentiation have been combined to some extent successfully in the field of Deep Learning for example. It has shown itself to be more than a collection of machine learning algorithms and the name \emph{Differentiable Programming} emerged as a new programming paradigm. But because the field is still in its youth, most of the advances come as a result of empirical investigations. Yet, as it is founded on rigorous mathematical objects, it offers an opportunity to be formalized as an algebraic language.
 
 We have been inspired by the development of differentiable programming to formalize a theoretical model, that encompasses the ideas underlying differentiable programming and provides a more general setting for investigations of differentiable programs. The presented theoretical model enables analytic investigations of differentiable programs through algebraic tools, that are closer to the field of programming; i.e. the presented operators can take the same role as higher order functions in functional programming. We introduce a \emph{Virtual Tensor Machine} as a language that extends functional definition of programs with a \emph{Tensor Series Algebra} of the memory. Such a tensor description of the memory can also serve as a formalization of recent advancements in high performance computing hardware, ex. tensor processing units by Google and tensor cores by Nvidia. This algebraic structure inherent to our model allows us to establish an \emph{Operational Calculus} of higher-order constructs that can facilitate reasoning about differentiable programs. Furthermore, the presented model is self-sufficient, as the Operational Calculus presented herein is implemented strictly within the language itself.
 
-Mathematical analysis and calculus found its way into programming, where different fields employ analytic properties of programs. What seems to be lacking in these attempts is a mechanism that would abstract away the gory details of calculus and provide the ability to employ it on a higher level in computer programming. The proposed theoretical model and the constructed operational calculus aim to fill this gap. To demonstrate its algebraic power over analytic properties of differentiable programs, we analyse iterators, considering fractional iterations and their (higher-order) rates of change in relation to the number of iterations; i.e. (higher-order) \emph{ iterating velocities }. We than solve the special case of \emph{ReduceSum} and its (higher-order) iterating velocities, and provide their explicit solutions within the language.
+Mathematical analysis and calculus found their way into programming, where different fields employ analytic properties of programs. What seems to be lacking in these attempts is a mechanism that would abstract away the gory details of calculus and provide the ability to employ it on a higher level in computer programming. The proposed theoretical model and the constructed operational calculus aim to fill this gap. To demonstrate its algebraic power over analytic properties of differentiable programs, we analyse iterators, considering fractional iterations and their (higher-order) rates of change in relation to the number of iterations; i.e. (higher-order) \emph{ iterating velocities }. We than solve the special case of \emph{ReduceSum} and its (higher-order) iterating velocities, and provide their explicit solutions within the language.