TMS Analytics for nonlinear multiparametric approximation of surface data

Bookmarks: 

Thursday, October 05, 2017

The new version 2.2 of TMS Analytics and Physics pack contains the nonlinear least squares approximation tools. The nonlinear approximation tasks are widely used in statistics to fit experimental data with some random distribution function.


Let us consider the example of surface data fitting with the 2D Gaussian function. The Gaussian distribution of 2 variables x and y (https://en.wikipedia.org/wiki/Gaussian_function) has the following math expression:

Except the amplitude A, the distribution nonlinearly depends on four parameters: center coordinates x0, y0 and deviations sx, sy. The task of fitting data with the Gaussian function is to find optimal values of the four parameters, those minimize the error of the distribution and the experimental data zi(xi,yi), i=1..N.

The approximation task can be solved by the numerical tool with the following code:

var
  variables, coefficients: TArray;
  f: string;
  basis: TNonlinearBasis;
  cValues: TArray;
  appr: TNonlinearApproximator;
  opt: TSolverOptions;
  eValues: TArray;
  eValue: TFloat;
begin
  variables:= TArray.Create('x', 'y'); // 1
  coefficients:= TArray.Create('x0', 'Sx', 'y0', 'Sy'); // 2 
  f:= 'e^-((x-x0)^2/(2*Sx^2)+(y-y0)^2/(2*Sy^2))'; // 3
  basis:= TNonlinearScalarBasis.Create(variables, coefficients, nil, f); // 4 

  appr:= TGaussNewtonLeastSquares.Create; // 5
  appr.C0:= TArray.Create(0.1, 0.1, 0.1, 0.1); // 6
  opt:= TSolverOptions.Create(true);
  opt.MaxIterationCount:= 100;
  opt.Precision:= 0.2;
  appr.Options:= opt; // 7

  cValues:= appr.Approximate(basis, xyData, zData); // 8

  // use basis with optimal coefficients
end;

Line 1: Create array of variable names.
Line 2: Create array of four distribution coefficient names.
Line 3: Set up the approximation function expression.
Line 4: Create the nonlinear basis instance with specified data.
Line 5: Create the appropriate nonlinear approximator instance.
Line 6: Set up initial guess of coefficient values for nonlinear solution.
Line 7: Set up appropriate nonlinear solution options.
Line 8: Solve the problem (finding optimal values).

When optimal values of the Gauss distribution parameters found, they can be used with the basis instance to calculate the distribution function in any specified point (x,y) or for other analysis methods, like derivative calculations and so on.

On the pictures below there are results of the approximation made with some generated noisy data. The result presented as 1D section of the 2D function for two different x values: x=0.0 and x=0.6.





In this sample, we used the TMS FNC Chart to display the values and the approximated function. The advantage of using the TMS FNC Chart is that it can be used both for VCL and FMX applications and we can use the same code to create a VCL Windows application or a FireMonkey cross platform application.
The source code of the example application can be downloaded here.

The TMS Analytics and Physics pack version 2.2 includes also many other useful numerical tools: linear least squares approximation, 1D and 2D integration, ordinary differential equation solution, function analysis. Due to the addition of array and matrix support in version 2.2, many numerical calculations can be implemented in vector form with small and compact formulae. The version can be downloaded from the product page.

Bruno Fierens


Bookmarks: 

This blog post has not received any comments yet. Add a comment.



Array and Matrix operations with Analytics 2.2

Bookmarks: 

Tuesday, September 26, 2017

New version 2.2 of TMS Analytics and Physics pack includes total support of operations with N-element arrays and MxN matrices. The operations realized in the Linear Algebra extension package. Array and matrix operations allow advanced calculations with small, compact formulae.

Let us consider the examples of using the array/matrix operations for solving some common math/statistics problems. First of all we need to add special variables to the translator instance:
var
  av: TArray<TFloat>;
  mv: TArray<TArray<TFloat>>;
begin
  translator:= TTranslator.Create;

  translator.Add('µ', 0.0);

  av:=TArray<TFloat>.Create(0.2, -0.1, -0.25, 0.4);
  translator.Add('A', av);

  av:=TArray<TFloat>.Create(-0.2, 0.2, -0.1, 0.25, 0.35, 0.15, -0.04, 0.01, -0.12, 0.1);
  translator.Add('B', av);

  SetLength(mv, 4, 3);
  mv[0,0]:= 0.5; mv[0,1]:=-0.4; mv[0,2]:= 0.33;
  mv[1,0]:=-0.3; mv[1,1]:= 0.1; mv[1,2]:= 0.28;
  mv[2,0]:=-0.4; mv[2,1]:=-0.2; mv[2,2]:=-0.75;
  mv[3,0]:=0.44; mv[3,1]:=0.25; mv[3,2]:= 1.00;
  translator.Add('M', mv);
end;
The code above adds to the translator instance the ‘µ’ variable with value 0.0, the array variable ‘A’ with 4 components, the array variable ‘B’ with 10 components and the 4x3 matrix ‘M’.

The first problem to solve is calculation of statistics square deviation from the mean value for the ‘B’ array. Math formula for the deviation of discrete random variable is (https://en.wikipedia.org/wiki/Standard_deviation):

where µ is the mean value

N is the number of elements.

This formula can be evaluated with using array operations by the following code:
var
  m, d: TValue;
begin
  m:= translator.Calculate('∑B/#B');
  translator.Variables['µ'].Value:= m;
  d:= translator.Calculate('∑((B-µ)^2)/#B');
  // print the ‘d’ value
end;
First we calculate the mean value. The number operator ‘#’ in the formula returns the array element count. Then we assign the calculated value to the ‘µ’ variable and, finally, calculate the deviation value. Notice that there are the parentheses for the square operation in the formula, because the summation operation has higher priority. Here is the output for the evaluation:

µ = ∑B/#B = 0.06
D = ∑((B-µ)^2)/#B = 0.02876


Next problem is solving an over-determined linear equation system with the least squares approach (https://en.wikipedia.org/wiki/Linear_least_squares_(mathematics)). Let the system is set up by the ‘M’ matrix and the ‘A’ vector. Then the solution ‘X’ is defined by the formula:

where MT – is the transposed matrix, 1 means the inversed matrix.

This formula can be evaluated with the Analytics library as the following ‘M'*A/(M'*M)’. Here the apostrophe ‘'’ operator is for transposition operation and the division is equivalent to the multiplication by the inversed matrix. Evaluating the formula we get the following output:

X = M'*A/(M'*M) = TArray<TFloat>[3]=
(0.459953357586789 0.163947437790482 0.113927221302585 )

Here the ‘X’ array is the solution vector that minimizes the square error for the over-determined system.

As can be seen from the examples above, the realized array/matrix operations allow easily solve complicated problems with small, compact formulae. The complicated code for the array/matrix operations implemented inside the Analytics library and totally integrated with analytical evaluations. For an example, linear algebra operations allow solving so called ‘matrix’ ordinary differential equations (https://en.wikipedia.org/wiki/Matrix_differential_equation).

The TMS Analytics and Physics pack version 2.2 can be downloaded here. Total source code of the example application can be downloaded here .

Nancy Lescouhier


Bookmarks: 

This blog post has received 1 comment. Add a comment.



Exploring parametric ODE with TMS Analytics and Physics numerical tool

Bookmarks: 

Thursday, July 13, 2017

Ordinary differential equations (ODE) used in many applications for physical, economical and other problem solution. The equations commonly describe the time evolution of some system from known initial state – initial value problem (https://en.wikipedia.org/wiki/Initial_value_problem).

The TMS Analytics and Physics pack contains the ODE solution tool of solving initial value problems for systems of ODEs. This tool is totally integrated with the symbolic capabilities of the library. This allows solving parametric problems and exploring how the parameter value influences the system behavior.

Let us consider the following simple parametric ODE:



with initial condition y(0)=1 on the time interval t=[0, 10]. The problem is parametric because there are parameters ‘A’ and ‘b’ in the equation. Our goal is exploring how the solution (function y(t)) depends on the value of ‘b’ parameter.

First we create the instance of analytical ODE object, here is the code:

var
  A, b: TVariable;
  prms: TArray<TVariable>;
  fv, equations: TArray<string>;
  ode: TAnalyticalODE;
  solver: TODESolverClass;
  y0: TArray<TFloat>;
  t: TArray<TFloat>;
  y: TArray<TArray<TFloat>>;
  i: Integer;
begin
  A:= TRealVariable.Create('A', 1.0);    // 1
  b:= TRealVariable.Create('b', 0.0);    // 2
  prms:= TArray<TVariable>.Create(A, b); // 3
  fv:= TArray<string>.Create('y');       // 4
  equations:= TArray<string>.Create('sin(t)+A*t/y^b'); // 5
  ode:= TScalarODE.Create('t', fv, equations, prms);   // 6
  ...
 
Line 1: Create the variable for ‘A’ parameter.
Line 2: Create the variable for ‘b’ parameter.
Line 3: Create array of parameters.
Line 4: Create array of function names (one name required for the problem).
Line 5: Create array of equations (one equation).
Line 6: Create the instance of analytical ODE system with specified equation and parameters.

After the instance of the analytical ODE system created we must select appropriate solver and specify initial condition for the problem. We will use the Runge-Kutta direct solver of the 4-th order. The code continued:

solver:= TRungeKutta4Solver;          // 7
y0:= TArray.Create(1.0);      // 8
And finally we must solve the initial value problem for various values of ‘b’ parameter, say b=1..8:

for i:=1 to 8 do
begin
  ode.Parameters['b'].Value:= TValue.From(i); // 9
  y:= solver.Solve(ode, y0, 10.0, 1000, t);           // 10
  
  // Code for using the result data ‘y’ and ‘t’.
end; 
Line 9: Change value of ‘b’ parameter using the array property ‘Parameters’ of the analytical ODE class. Line 10: Solving the specified initial value problem for current value of ‘b’ parameter, on the time interval t=[0, 10], with 1000 discretization steps.

The solution result for changing value of ‘b’ parameter presented on the picture below.


TMSFNCChart cross-platform / cross-framework chart displaying the result

As can be seen from the code above, there is no need creating new analytical ODE instance for exploring the parametric problem. As the analytical ODE is parametric, it is solved for the current parameter values and they can be easily changed via parametric interface.

Other advantages of using the numerical ODE tool with the symbolic calculus are:
- Minimal code writing, because there is no need to write special classes for the function method references (delegates) as in other numerical libraries.
- Convenient data representation for the developer and user as math expressions, not as programming code.
- Easily using the user input data (string data) for manipulations in programming code.
- Easily transfer input data via network as strings, easily storing and serializing the data.

The TMS Analytics and Physics pack includes the library with 5 ready-to-use numerical tools totally integrated with analytical features, such as symbolic derivatives. The trial version of TMS Analytics and Physics pack can be downloaded here. Total source code of the example application of ODE tool can be downloaded here.

Bruno Fierens


Bookmarks: 

This blog post has not received any comments yet. Add a comment.



Function approximation with TMS Analytics and Physics numerical tools

Bookmarks: 

Monday, June 19, 2017

The problem of data approximation (curve fitting, surface fitting) arises from many areas: economics, physics, informatics and many others. There are many libraries, realizing numerical tools for using the approximation in software applications. The TMS Analytics and Physics pack contains the approximation numerical tool which has one unique feature – it is totally integrated with symbolic calculus. Using symbolic expressions with the numerical approximation gives the developer many advantages.

Let us consider a simple case of using the approximation tool for curve fitting (one-dimensional approximation). For this case we have a set of points Pi=(xi,yi), i=1..N. From the programming point of view we just have two arrays of real values, say X and Y. The approximation problem is to build a function y=f(x) which is close to the set of points. The most popular approach of the approximation is the linear least squares (https://en.wikipedia.org/wiki/Linear_least_squares_(mathematics)). With TMS approximation tool the problem can be solved in five lines of code:

var
  approximator: TLinearApproximator;
  basis: TLinearBasis;
  variables: TArray;
  functions: TArray;
  cValues: TArray;
begin
  variables:= TArray.Create('x'); // 1
  functions:= TArray.Create('e^x', 'sin(x)', 'e^-x', 'x^2', '3^x',   'sinh(2*x)'); // 2
  basis:= TLinearScalarBasis.Create(variables, 'C', nil, functions); // 3
  approximator:= TLinearLeastSquares.Create; // 4
  cValues:= approximator.Approximate(basis, X, Y); // 5
  
  // Using the basis with optimal coefficient values.
end;
Line 1: Create the array of variable names, as this is 1D approximation, there is one variable name ‘x’ in the array.
Line 2: Create the array of approximation functions (basis functions). The functions only depend on the ‘x’ variable. The type and the number of the basis functions can be arbitrary.
Line 3: Create the basis instance with the specified arguments. The ‘C’ argument specifies the name of approximation coefficients. The ‘nil’ argument tells that there is no parameters in the functions.
Line 4: Create the appropriate approximator instance. For the linear least squares approximation it is ‘TLinearLeastSquares’;
Line 5: Calculate the approximation coefficients for the specified basis and arrays of points ‘X’ and ‘Y’.

After the optimal coefficients found, we can evaluate the basis in any point and make other analytical manipulations. As example, here is the plotted approximation function with the original data points on the picture.

The main feature of the TMS numerical tools is they are totally integrated with the symbolic calculus. It brings the following advantages to the developer:
- Minimal code writing, because there is no need to write special classes for the function method references (delegates) as in other numerical libraries.
- Convenient data representation for the developer and user as math expressions, not as programming code.
- Easily using the user input data (string data) for manipulations in programming code.
- Almost unlimited choice of math expressions for manipulations, including special functions.
- Easily transfer input data and the output results via network as strings, easily storing and serializing the data.

The result of the approximation can be easily used with other numerical tools. For example, we can analyze the approximation function for the roots and special points – extremums. The code for the analysis is the following:
var
  f: string;
  sf1D: TSymbolicFunction1D;
  opt: TSolverOptions;
  points: TArray;
begin
  f:= basis.Evaluate(cValues); // 1 
  sf1D:= TSymbolicFunction1D.Create('x', f); // 2 
  opt:= TSolverOptions.Create(true);  // 3
  points:= TFunctionAnalyser.Analyse(sf1D, TNewton, opt, -2, 3, 10); // 4 
  
  // Using the points array.
end;

Line 1: Make the function expression with optimal coefficient values.
Line 2: Create the symbolic function instance for the analysis.
Line 3: Create default options for the nonlinear solver (which used for the function analysis).
Line 4: Finding all function's special points on the interval [-2..3]. The Newton’s method used for finding roots and extremums of the function which specified with the second argument.

The result of the analysis – function’s roots and extremums, presented on the picture below.

The TMS Analytics and Physics pack includes the library with 5 ready-to-use numerical tools totally integrated with analytical features, such as symbolic derivatives. The version can be downloaded from the product page. There is the source code example for all numerical tools with the new version download.

Bruno Fierens


Bookmarks: 

This blog post has received 2 comments. Add a comment.



Introducing functional derivatives is easy

Bookmarks: 

Monday, March 20, 2017

The unique feature of Analytics library is symbolic derivatives calculation. The library provides the simplest way to get a symbolic derivative expression for almost any complicated expression. There are predefined derivative for all standard functions: trigonometric, hyperbolic, logarithmic and exponential, inverse trigonometric and hyperbolic, and other.

Another fine thing is that any developer can easily define derivative for any other function. Let us consider how to define derivative for special Gamma function (https://en.wikipedia.org/wiki/Gamma_function#General). Its derivative is defined via another special function – Digamma (Polygamma function of order 0, https://en.wikipedia.org/wiki/Polygamma_function). Hereafter the Gamma function denoted as ‘G’ and the Digamma (and Polygamma) as ‘Y’. Then, the derivative formula is: d(G(x))/dx = G(x)*Y(x).

To introduce new functional derivative new class must be implemented. In our case, the class is inherited from ‘TSimpleFunctionalDerivative’ because the function has one argument only. Here is the code of the realized class:

TGammaDerivative = class sealed(TSimpleFunctionalDerivative)
protected
  function GetFunctionName(): string; override;
  function BaseDerivative(argument: TBaseExpression): TBaseExpression; override;
public
  class function IsRealized: boolean; override;
end;

function TGammaDerivative.GetFunctionName: string;
begin
  result:= 'G';
end;

class function TGammaDerivative.IsRealized: boolean;
begin
  result:= true;
end;

function TGammaDerivative.BaseDerivative(argument: TBaseExpression): TBaseExpression;
var
  ge: TBaseExpression;
  dge: TBaseExpression;
begin
  ge:= TFunctionExpression.CreateSimple('G', argument);
  dge:= TFunctionExpression.CreateSimple('Y', argument);

  result:= TProductExpression.MakeProduct(ge, dge);
end;
As can be seen from the code, there are only three methods to override. The GetFunctionName method defines that the derivative is for function with name ‘G’. The IsRealized functions returns ‘true’, that means the class is totally realized and can be used by the Analytics system. And the last method defines the derivation algorithm itself. Namely, it creates the Gamma and Digamma functions with the same argument and, according to the derivative formula above, their product.

Now we can calculate derivatives for expressions with the Gamma function. For the simplest case with input string ‘G(x)’ we get the expected output string ‘G(x)*Y(x)’. No surprise here, because we just wrote the code, which produces the output.

But the amazing thing is when we trying to calculate the derivative expression for the Gamma function with a composite argument. As example, for the input string ‘G(x^2+1)’ it produces the output string ‘(G(x^2+1)*Y(x^2+1))*(2*x)’ which is totally correct for this case. And the output is correct for all the expressions with the Gamma function. It is because the Analytics library cares about other things, like implementing the ‘chain rule’ for derivation process.

The source code of the example project is available here.

Nancy Lescouhier


Bookmarks: 

This blog post has not received any comments yet. Add a comment.




Previous  |  Next  |  Index