In the context of curve fitting, a linear curve
is a curve that has a linear dependence on the curve parameters.
Examples of linear curves are: lines, polynomials,
Chebyshev series, and any linear combination of a set of curves.
The linearity greatly simplifies the calculations,
as the solution can be expressed in terms of simple linear
algebra.
Most curve fitting is done using
linear combinations of a set of
basis functions. A linear combination of a set of functions,
f_{1},
f_{2}...,
f_{n}
is a function whose value is given by
f(x) = a_{1}f_{1}(x) +
a_{2}f_{2}(x) + ... +
a_{n}f_{n}(x)
where
a_{1},
a_{2}...,
a_{n} are the
coefficients. The advantage of using this type
of approximation is that the math is relatively simple.
The solution can be found by solving a standard linear algebra
problem. In many cases, this can be done very efficiently.
The basis functions
f_{1},
f_{2}...,
f_{n}
that are combined to create the approximation together
form a function basis.
A linear combination is therefore
defined by the function basis and the coefficients
of the basis functions in the combination.
Perhaps the most wellknown example of a linear combination
of basis functions is provided by polynomials.
A polynomial is a function that is a combination of the argument
raised to different integer powers:
f(x) = a_{0} + a_{1}x + ... +
a_{n}x^{n}
The integer powers are the basis functions. Chebyshev polynomials form an alternate basis for the polynomials.
They can be defined mathematically by the identity
T_{n}
(
cos x) = cos nx
Their mathematical properties make them particularly suited
for approximating functions.
In the Extreme Optimization Numerical Libraries for .NET, function bases are implemented by the
abstract FunctionBasis class and its derived
classes. Linear combinations are implemented using the LinearCombination class and its derived classes, including
Polynomial and ChebyshevSeries. The table below summarizes the classes
that implement function bases and the corresponding linear combinations.
Static curve fitting methods
Many classes expose static methods that allow you to obtain a least squares fit. The main drawback of the methods
discussed in this section is that they don't offer much control over the calculation of the least squares fit. There
is also no easy way to obtain information about the quality of the fit. However, they may be useful because they
offer a very quick way to get a result.
The LeastSquaresFit method of the
Polynomial class constructs a Polynomial that is the least squares fit of a specified degree
through a given set of points.
Vector<double> xValues = Vector.Create(7, i => Math.Cos(i * Constants.Pi / 6));
Vector<double> yValues = Vector.Create(7, i => Math.Sin(i * Constants.Pi / 6));
Polynomial p = Polynomial.LeastSquaresFit(xValues, yValues, 4);
Dim xValues As Vector(Of Double) =
Vector.Create(7, Function(i) Math.Cos(i * Constants.Pi / 6))
Dim yValues As Vector(Of Double) =
Vector.Create(7, Function(i) Math.Sin(i * Constants.Pi / 6))
Dim p As Polynomial = Polynomial.LeastSquaresFit(xValues, yValues, 4)
No code example is currently available or this language may not be supported.
let xValues = Vector.Create(7, fun i > cos((float i) * Constants.Pi / 6.0))
let yValues = Vector.Create(7, fun i > sin((float i) * Constants.Pi / 6.0))
let p = Polynomial.LeastSquaresFit(xValues, yValues, 4)
A similar method is available for the Line and ChebyshevSeries classes.
You can also create a function basis, and obtain the least squares fit using the functions in the function basis
for a set of data points using the LeastSquaresFit method. It returns
a LinearCombination curve. This allows you
to fit data to an arbitrary set of functions. You can also specify weights for each of the data points. The
weights specify how much weight each data point should have in the sum of squares that is to be minimized.
The following example fits a set of data points to a combination of the sine, cosine, and exponential
functions.
Vector<double> xValues = Vector.Create(0.0, 1.0, 2.0, 3.0, 4.0);
Vector<double> yValues = Vector.Create(4.0, 1.0, 4.0, 10.0, 8.0);
GeneralFunctionBasis basis = new GeneralFunctionBasis(Math.Sin, Math.Cos, Math.Exp);
LinearCombination f = basis.LeastSquaresFit(xValues, yValues);
Dim xValues = Vector.Create(0.0, 1.0, 2.0, 3.0, 4.0)
Dim yValues = Vector.Create(4.0, 1.0, 4.0, 10.0, 8.0)
Dim basis As GeneralFunctionBasis =
New GeneralFunctionBasis(AddressOf Math.Sin,
AddressOf Math.Cos, AddressOf Math.Exp)
Dim f As LinearCombination = basis.LeastSquaresFit(xValues, yValues)
No code example is currently available or this language may not be supported.
let xValues = Vector.Create(0.0, 1.0, 2.0, 3.0, 4.0)
let yValues = Vector.Create(4.0, 1.0, 4.0, 10.0, 8.0)
let basis = new GeneralFunctionBasis(
Func<_,_>(Math.Sin),
Func<_,_>(Math.Cos),
Func<_,_>(Math.Exp))
let f = basis.LeastSquaresFit(xValues, yValues)
The LinearCurveFitter class
The LinearCurveFitter class performs a linear
least squares fit. It offers greater control over the procedure, and gives more extensive results.
To perform the fit, a LinearCurveFitter needs data points, and a curve to fit. You must set the
Curve property to an instance of a LinearCombination object. A LinearCombination
object can represent any combination of functions. Other objects, for example of type Line, Polynomial
or ChebyshevSeries are allowed, as they inherit from
LinearCombination.
The data is supplied as VectorT objects. The
XValues and YValues properties specify the X and Y values of the data
points, respectively.
By default, the fit is unweighted. All weights are set equal to 1. Weights can be specified in one of two ways.
The first way is to set the WeightVector
property to a vector with as many elements as there are data points. Each component of the vector specifies the
weight of the corresponding data point.
Alternatively, the WeightFunction can be
used to calculate the weights based on the data points. The WeightFunctions class contains a number of predefined weight
functions listed in the table below. In the descriptions, x and y stand for the x and y values of
each data point.
Field  Description 

OneOverX 
The weight equals 1/x.

OneOverXSquared 
The weight equals 1/x^{2}.

OneOverY 
The weight equals 1/y.

OneOverYSquared 
The weight equals 1/y^{2}.

The Fit method performs the actual least
squares calculation, and adjusts the parameters of the Curve to correspond to the least squares fit. The
BestFitParameters property returns a
VectorT containing the parameters of the curve. The
GetStandardDeviations returns a
VectorT containing the standard deviations of each
parameter as estimated in the least squares calculation.
Example 1: Fitting a Polynomial
In the first example, we fit some data to a quadratic polynomial.
Vector<double> loadData = Vector.Create<double>();
Vector<double> deflectionData = Vector.Create<double>();
Polynomial poly = new Polynomial(2);
LinearCurveFitter fitter = new LinearCurveFitter();
fitter.Curve = poly;
fitter.XValues = loadData;
fitter.YValues = deflectionData;
fitter.Fit();
Dim loadData = Vector.Create(Of Double)()
Dim deflectionData = Vector.Create(Of Double)()
Dim poly As Polynomial = New Polynomial(2)
Dim fitter As LinearCurveFitter = New LinearCurveFitter()
fitter.Curve = poly
fitter.XValues = loadData
fitter.YValues = deflectionData
fitter.Fit()
No code example is currently available or this language may not be supported.
let loadData = Vector.Create<double>()
let deflectionData = Vector.Create<double>()
let poly = new Polynomial(2)
let fitter = new LinearCurveFitter()
fitter.Curve < poly
fitter.XValues < loadData
fitter.YValues < deflectionData
fitter.Fit()
First, the data is loaded into the loadData and deflectionData vectors. We then create
the polynomial, poly. That gives us what we need to create the LinearCurveFitter objects.
We set its properties, and perform the fit.
Example 2: Fitting arbitrary functions
In the next example, we fit measurements of the conductance of copper to a theoretical model:
We have measurements of k(T) for a series of temperatures. To estimate the parameters
c_{1} and c_{2}, we need to take the following steps:

Transform the dependent variable k(T) into y = 1/k(T). The
expression for y is then linear in the basis functions 1/T and T^{2}.
 Create a function basis with these two basis functions.

Create a LinearCombination curve using this function basis.

Create the LinearCurveFitter object and set its properties.
 Perform the fit and report the results.
The code below performs these steps. It assumes we have created two vector variables, temperature and
conductance, that contain the data of the observations.
Vector<double> y = Vector.Reciprocal(conductance);
Func<double,double>[] basisFunctions = new Func<double,double>[]
{ x => 1/x, x => x * x };
GeneralFunctionBasis basis = new GeneralFunctionBasis(basisFunctions);
LinearCombination curve = new LinearCombination(basis);
LinearCurveFitter fitter = new LinearCurveFitter();
fitter.Curve = curve;
fitter.XValues = temperature;
fitter.YValues = y;
fitter.Fit();
Vector<double> solution = fitter.BestFitParameters;
Vector<double> s = fitter.GetStandardDeviations();
Console.WriteLine("c1: {0,20:E10} {1,20:E10}", solution[0], s[0]);
Console.WriteLine("c2: {0,20:E10} {1,20:E10}", solution[1], s[1]);
Dim y = Vector.Reciprocal(conductance)
Dim basisFunctions As Func(Of Double, Double)() =
{Function(x) 1 / x, Function(x) x * x}
Dim basis As GeneralFunctionBasis = New GeneralFunctionBasis(basisFunctions)
Dim curve As LinearCombination = New LinearCombination(basis)
Dim fitter As LinearCurveFitter = New LinearCurveFitter()
fitter.Curve = curve
fitter.XValues = temperature
fitter.YValues = y
fitter.Fit()
Dim solution = fitter.BestFitParameters
Dim s = fitter.GetStandardDeviations()
Console.WriteLine("c1: 0,20:E10 1,20:E10", solution(0), s(0))
Console.WriteLine("c2: 0,20:E10 1,20:E10", solution(1), s(1))
No code example is currently available or this language may not be supported.
let y = Vector.Reciprocal(conductance)
let basisFunctions = [
Func<_,_>(fun x > 1.0 / x);
Func<_,_>(fun x > x * x) ]
let basis = new GeneralFunctionBasis(basisFunctions)
let curve = new LinearCombination(basis)
let fitter = new LinearCurveFitter()
fitter.Curve < curve
fitter.XValues < temperature
fitter.YValues < y
fitter.Fit() > ignore
let solution = fitter.BestFitParameters
let s = fitter.GetStandardDeviations()
Console.WriteLine("c1: 0,20:E10 1,20:E10", solution.[0], s.[0])
Console.WriteLine("c2: 0,20:E10 1,20:E10", solution.[1], s.[1])