# Improvements for symbolic expressions simplification in TMS Analytics & Physics developing library

Bookmarks:

#### Wednesday, December 20, 2017

One of the unique features of the Analytics library is symbolic derivatives calculation. The derivative can be calculated just with several lines of Delphi code, for an example:

```var
f, df: string;
t: TTranslator;
begin
t:= TTranslator.Create;
f:= 'x^2*e^(x/3)';
df:= t.Derivative(f, 'x');
//...
end;```

The Analytics library guarantees that the result derivative expression is syntactically and mathematically correct and satisfies the derivative rules. From the first version, the derivative algorithm automatically simplifies the result derivative expression. Nevertheless, the library is not a Computer Algebra system, so it cannot simplify all the result expressions to the optimal representation.

New version 2.3 introduces new simplification algorithms for providing more readable and short symbolic derivatives. New simplification algorithms include: constants reducing in sum and product expressions; reducing nested sum or product expressions to one; reducing negation pairs in product expressions and others.

There are some examples of functions and their derivative expressions, evaluated with the old version of the library and with the 2.3 version:

```f(x)       : x^2*e^(x/3)
df/dx (old): (2*x)*e^(x/3)+(e^(x/3)*(1/3))*x^2
df/dx (2.3): 2*x*e^(x/3)+e^(x/3)/3*x^2

f(x)       : 2*x^3+A/4*x^2-x/3
df/dx (old): (3*x^2)*2+((2*x)*A)/4-1/3
df/dx (2.3): 6*x^2+1/2*x*A-1/3

f(x)       : (A/2)*arctan(x^2)
df/dx (old): ((1/(1+(x^2)^2))*(2*x))*(A/2)
df/dx (2.3): x*A/(1+x^4)

f(x)       : P{n-1 m+2}(x)
df/dx (old): ((((n-1)+1)-(m+2))*P{(n-1)+1 m+2}(x) -(((n-1)+1)*x)*P{n-1 m+2}(x))/(x^2-1)
df/dx (2.3): ((n-m-2)*P{n m+2}(x)-n*x*P{n-1 m+2}(x))/(x^2-1)

```

As can be seen from the examples, new version gives shorter and more readable symbolic expressions for different function cases. The simplified representation of the formulae also reduces the calculation time. The new library version is already available here

Bruno Fierens

Bookmarks:

# TMS Analytics for nonlinear multiparametric approximation of surface data

Bookmarks:

#### Thursday, October 05, 2017

The new version 2.2 of TMS Analytics and Physics pack contains the nonlinear least squares approximation tools. The nonlinear approximation tasks are widely used in statistics to fit experimental data with some random distribution function.

Let us consider the example of surface data fitting with the 2D Gaussian function. The Gaussian distribution of 2 variables x and y (https://en.wikipedia.org/wiki/Gaussian_function) has the following math expression:

Except the amplitude A, the distribution nonlinearly depends on four parameters: center coordinates x0, y0 and deviations sx, sy. The task of fitting data with the Gaussian function is to find optimal values of the four parameters, those minimize the error of the distribution and the experimental data zi(xi,yi), i=1..N.

The approximation task can be solved by the numerical tool with the following code:

```var
variables, coefficients: TArray;
f: string;
basis: TNonlinearBasis;
cValues: TArray;
appr: TNonlinearApproximator;
opt: TSolverOptions;
eValues: TArray;
eValue: TFloat;
begin
variables:= TArray.Create('x', 'y'); // 1
coefficients:= TArray.Create('x0', 'Sx', 'y0', 'Sy'); // 2
f:= 'e^-((x-x0)^2/(2*Sx^2)+(y-y0)^2/(2*Sy^2))'; // 3
basis:= TNonlinearScalarBasis.Create(variables, coefficients, nil, f); // 4

appr:= TGaussNewtonLeastSquares.Create; // 5
appr.C0:= TArray.Create(0.1, 0.1, 0.1, 0.1); // 6
opt:= TSolverOptions.Create(true);
opt.MaxIterationCount:= 100;
opt.Precision:= 0.2;
appr.Options:= opt; // 7

cValues:= appr.Approximate(basis, xyData, zData); // 8

// use basis with optimal coefficients
end;
```

Line 1: Create array of variable names.
Line 2: Create array of four distribution coefficient names.
Line 3: Set up the approximation function expression.
Line 4: Create the nonlinear basis instance with specified data.
Line 5: Create the appropriate nonlinear approximator instance.
Line 6: Set up initial guess of coefficient values for nonlinear solution.
Line 7: Set up appropriate nonlinear solution options.
Line 8: Solve the problem (finding optimal values).

When optimal values of the Gauss distribution parameters found, they can be used with the basis instance to calculate the distribution function in any specified point (x,y) or for other analysis methods, like derivative calculations and so on.

On the pictures below there are results of the approximation made with some generated noisy data. The result presented as 1D section of the 2D function for two different x values: x=0.0 and x=0.6.

In this sample, we used the TMS FNC Chart to display the values and the approximated function. The advantage of using the TMS FNC Chart is that it can be used both for VCL and FMX applications and we can use the same code to create a VCL Windows application or a FireMonkey cross platform application.

The TMS Analytics and Physics pack version 2.2 includes also many other useful numerical tools: linear least squares approximation, 1D and 2D integration, ordinary differential equation solution, function analysis. Due to the addition of array and matrix support in version 2.2, many numerical calculations can be implemented in vector form with small and compact formulae. The version can be downloaded from the product page.

Bruno Fierens

Bookmarks:

# Array and Matrix operations with Analytics 2.2

Bookmarks:

#### Tuesday, September 26, 2017

New version 2.2 of TMS Analytics and Physics pack includes total support of operations with N-element arrays and MxN matrices. The operations realized in the Linear Algebra extension package. Array and matrix operations allow advanced calculations with small, compact formulae.

Let us consider the examples of using the array/matrix operations for solving some common math/statistics problems. First of all we need to add special variables to the translator instance:
```var
av: TArray<TFloat>;
mv: TArray<TArray<TFloat>>;
begin
translator:= TTranslator.Create;

av:=TArray<TFloat>.Create(0.2, -0.1, -0.25, 0.4);

av:=TArray<TFloat>.Create(-0.2, 0.2, -0.1, 0.25, 0.35, 0.15, -0.04, 0.01, -0.12, 0.1);

SetLength(mv, 4, 3);
mv[0,0]:= 0.5; mv[0,1]:=-0.4; mv[0,2]:= 0.33;
mv[1,0]:=-0.3; mv[1,1]:= 0.1; mv[1,2]:= 0.28;
mv[2,0]:=-0.4; mv[2,1]:=-0.2; mv[2,2]:=-0.75;
mv[3,0]:=0.44; mv[3,1]:=0.25; mv[3,2]:= 1.00;
end;
```
The code above adds to the translator instance the ‘µ’ variable with value 0.0, the array variable ‘A’ with 4 components, the array variable ‘B’ with 10 components and the 4x3 matrix ‘M’.

The first problem to solve is calculation of statistics square deviation from the mean value for the ‘B’ array. Math formula for the deviation of discrete random variable is (https://en.wikipedia.org/wiki/Standard_deviation):

where µ is the mean value

N is the number of elements.

This formula can be evaluated with using array operations by the following code:
```var
m, d: TValue;
begin
m:= translator.Calculate('∑B/#B');
translator.Variables['µ'].Value:= m;
d:= translator.Calculate('∑((B-µ)^2)/#B');
// print the ‘d’ value
end;```
First we calculate the mean value. The number operator ‘#’ in the formula returns the array element count. Then we assign the calculated value to the ‘µ’ variable and, finally, calculate the deviation value. Notice that there are the parentheses for the square operation in the formula, because the summation operation has higher priority. Here is the output for the evaluation:

µ = ∑B/#B = 0.06
D = ∑((B-µ)^2)/#B = 0.02876

Next problem is solving an over-determined linear equation system with the least squares approach (https://en.wikipedia.org/wiki/Linear_least_squares_(mathematics)). Let the system is set up by the ‘M’ matrix and the ‘A’ vector. Then the solution ‘X’ is defined by the formula:

where MT – is the transposed matrix, 1 means the inversed matrix.

This formula can be evaluated with the Analytics library as the following ‘M'*A/(M'*M)’. Here the apostrophe ‘'’ operator is for transposition operation and the division is equivalent to the multiplication by the inversed matrix. Evaluating the formula we get the following output:

X = M'*A/(M'*M) = TArray<TFloat>[3]=
(0.459953357586789 0.163947437790482 0.113927221302585 )

Here the ‘X’ array is the solution vector that minimizes the square error for the over-determined system.

As can be seen from the examples above, the realized array/matrix operations allow easily solve complicated problems with small, compact formulae. The complicated code for the array/matrix operations implemented inside the Analytics library and totally integrated with analytical evaluations. For an example, linear algebra operations allow solving so called ‘matrix’ ordinary differential equations (https://en.wikipedia.org/wiki/Matrix_differential_equation).

The TMS Analytics and Physics pack version 2.2 can be downloaded here. Total source code of the example application can be downloaded here .

Nancy Lescouhier

Bookmarks:

# Exploring parametric ODE with TMS Analytics and Physics numerical tool

Bookmarks:

#### Thursday, July 13, 2017

Ordinary differential equations (ODE) used in many applications for physical, economical and other problem solution. The equations commonly describe the time evolution of some system from known initial state – initial value problem (https://en.wikipedia.org/wiki/Initial_value_problem).

The TMS Analytics and Physics pack contains the ODE solution tool of solving initial value problems for systems of ODEs. This tool is totally integrated with the symbolic capabilities of the library. This allows solving parametric problems and exploring how the parameter value influences the system behavior.

Let us consider the following simple parametric ODE:

with initial condition y(0)=1 on the time interval t=[0, 10]. The problem is parametric because there are parameters ‘A’ and ‘b’ in the equation. Our goal is exploring how the solution (function y(t)) depends on the value of ‘b’ parameter.

First we create the instance of analytical ODE object, here is the code:

```var
A, b: TVariable;
prms: TArray<TVariable>;
fv, equations: TArray<string>;
ode: TAnalyticalODE;
solver: TODESolverClass;
y0: TArray<TFloat>;
t: TArray<TFloat>;
y: TArray<TArray<TFloat>>;
i: Integer;
begin
A:= TRealVariable.Create('A', 1.0);    // 1
b:= TRealVariable.Create('b', 0.0);    // 2
prms:= TArray<TVariable>.Create(A, b); // 3
fv:= TArray<string>.Create('y');       // 4
equations:= TArray<string>.Create('sin(t)+A*t/y^b'); // 5
ode:= TScalarODE.Create('t', fv, equations, prms);   // 6
...
```
Line 1: Create the variable for ‘A’ parameter.
Line 2: Create the variable for ‘b’ parameter.
Line 3: Create array of parameters.
Line 4: Create array of function names (one name required for the problem).
Line 5: Create array of equations (one equation).
Line 6: Create the instance of analytical ODE system with specified equation and parameters.

After the instance of the analytical ODE system created we must select appropriate solver and specify initial condition for the problem. We will use the Runge-Kutta direct solver of the 4-th order. The code continued:

```solver:= TRungeKutta4Solver;          // 7
y0:= TArray.Create(1.0);      // 8```
And finally we must solve the initial value problem for various values of ‘b’ parameter, say b=1..8:

```for i:=1 to 8 do
begin
ode.Parameters['b'].Value:= TValue.From(i); // 9
y:= solver.Solve(ode, y0, 10.0, 1000, t);           // 10

// Code for using the result data ‘y’ and ‘t’.
end; ```
Line 9: Change value of ‘b’ parameter using the array property ‘Parameters’ of the analytical ODE class. Line 10: Solving the specified initial value problem for current value of ‘b’ parameter, on the time interval t=[0, 10], with 1000 discretization steps.

The solution result for changing value of ‘b’ parameter presented on the picture below.

TMSFNCChart cross-platform / cross-framework chart displaying the result

As can be seen from the code above, there is no need creating new analytical ODE instance for exploring the parametric problem. As the analytical ODE is parametric, it is solved for the current parameter values and they can be easily changed via parametric interface.

Other advantages of using the numerical ODE tool with the symbolic calculus are:
- Minimal code writing, because there is no need to write special classes for the function method references (delegates) as in other numerical libraries.
- Convenient data representation for the developer and user as math expressions, not as programming code.
- Easily using the user input data (string data) for manipulations in programming code.
- Easily transfer input data via network as strings, easily storing and serializing the data.

The TMS Analytics and Physics pack includes the library with 5 ready-to-use numerical tools totally integrated with analytical features, such as symbolic derivatives. The trial version of TMS Analytics and Physics pack can be downloaded here. Total source code of the example application of ODE tool can be downloaded here.

Bruno Fierens

Bookmarks:

# Function approximation with TMS Analytics and Physics numerical tools

Bookmarks:

#### Monday, June 19, 2017

The problem of data approximation (curve fitting, surface fitting) arises from many areas: economics, physics, informatics and many others. There are many libraries, realizing numerical tools for using the approximation in software applications. The TMS Analytics and Physics pack contains the approximation numerical tool which has one unique feature – it is totally integrated with symbolic calculus. Using symbolic expressions with the numerical approximation gives the developer many advantages.

Let us consider a simple case of using the approximation tool for curve fitting (one-dimensional approximation). For this case we have a set of points Pi=(xi,yi), i=1..N. From the programming point of view we just have two arrays of real values, say X and Y. The approximation problem is to build a function y=f(x) which is close to the set of points. The most popular approach of the approximation is the linear least squares (https://en.wikipedia.org/wiki/Linear_least_squares_(mathematics)). With TMS approximation tool the problem can be solved in five lines of code:

```var
approximator: TLinearApproximator;
basis: TLinearBasis;
variables: TArray;
functions: TArray;
cValues: TArray;
begin
variables:= TArray.Create('x'); // 1
functions:= TArray.Create('e^x', 'sin(x)', 'e^-x', 'x^2', '3^x',   'sinh(2*x)'); // 2
basis:= TLinearScalarBasis.Create(variables, 'C', nil, functions); // 3
approximator:= TLinearLeastSquares.Create; // 4
cValues:= approximator.Approximate(basis, X, Y); // 5

// Using the basis with optimal coefficient values.
end;
```
Line 1: Create the array of variable names, as this is 1D approximation, there is one variable name ‘x’ in the array.
Line 2: Create the array of approximation functions (basis functions). The functions only depend on the ‘x’ variable. The type and the number of the basis functions can be arbitrary.
Line 3: Create the basis instance with the specified arguments. The ‘C’ argument specifies the name of approximation coefficients. The ‘nil’ argument tells that there is no parameters in the functions.
Line 4: Create the appropriate approximator instance. For the linear least squares approximation it is ‘TLinearLeastSquares’;
Line 5: Calculate the approximation coefficients for the specified basis and arrays of points ‘X’ and ‘Y’.

After the optimal coefficients found, we can evaluate the basis in any point and make other analytical manipulations. As example, here is the plotted approximation function with the original data points on the picture.

The main feature of the TMS numerical tools is they are totally integrated with the symbolic calculus. It brings the following advantages to the developer:
- Minimal code writing, because there is no need to write special classes for the function method references (delegates) as in other numerical libraries.
- Convenient data representation for the developer and user as math expressions, not as programming code.
- Easily using the user input data (string data) for manipulations in programming code.
- Almost unlimited choice of math expressions for manipulations, including special functions.
- Easily transfer input data and the output results via network as strings, easily storing and serializing the data.

The result of the approximation can be easily used with other numerical tools. For example, we can analyze the approximation function for the roots and special points – extremums. The code for the analysis is the following:
```var
f: string;
sf1D: TSymbolicFunction1D;
opt: TSolverOptions;
points: TArray;
begin
f:= basis.Evaluate(cValues); // 1
sf1D:= TSymbolicFunction1D.Create('x', f); // 2
opt:= TSolverOptions.Create(true);  // 3
points:= TFunctionAnalyser.Analyse(sf1D, TNewton, opt, -2, 3, 10); // 4

// Using the points array.
end;```

Line 1: Make the function expression with optimal coefficient values.
Line 2: Create the symbolic function instance for the analysis.
Line 3: Create default options for the nonlinear solver (which used for the function analysis).
Line 4: Finding all function's special points on the interval [-2..3]. The Newton’s method used for finding roots and extremums of the function which specified with the second argument.

The result of the analysis – function’s roots and extremums, presented on the picture below.

The TMS Analytics and Physics pack includes the library with 5 ready-to-use numerical tools totally integrated with analytical features, such as symbolic derivatives. The version can be downloaded from the product page. There is the source code example for all numerical tools with the new version download.

Bruno Fierens

Bookmarks: