Extreme Optimization >
User's Guide >
Mathematics Library >
Curve Fitting >
Nonlinear Curve Fitting
Extreme Optimization User's Guide
User's Guide
Up: Curve Fitting Next: Predefined Nonlinear Curves Previous: Linear Curve Fitting Contents
Nonlinear Curve Fitting
Nonlinear curve fitting extends linear curve fitting to curves whose parameters appear in the function expression
in arbitrary ways, not just linearly. Almost any function that can be expressed in closed form can be used for
nonlinear curve fitting. With this increased power comes the drawback that it is more difficult to estimate the
parameters. It may not be possible to guarantee the best solution is found.
Mathematical background
A nonlinear curve with parameters a1, a2...,
an is a curve of the form
f(x) = f(a1, a2, ..., an;
x).
Nonlinear least squares fitting follows the same general method as linear least squares. However, no closed form
algebraic expressions exist for the solution. The solution must be found by using a general multidimensional
optimization algorithm to find the minimum of the sum of the squares of the residuals.
Even though the objective function is not as simple as in the linear case, the fact that it equals a sum of
squares can be exploited to obtain more efficient methods. The Extreme Optimization Mathematics Library for
.NET uses a variant of the Levenberg-Marquardt method to compute the solution.
Nonlinear Curves
The Levenberg-Marquardt algorithm requires that the partial derivatives of the fitted curve with respect to the
curve parameters be available. However, a numerical approximation may be used if the closed form is not available. It
may also be advantageous to compute initial values for the parameters based on the data.
The NonlinearCurve class defines two methods,
FillPartialDerivatives and
SetInitialValues, that enable this
functionality.
The Extreme.Mathematics.Curves.Nonlinear namespace
defines a number of nonlinear curves. This includes exponential, rational and logistic curves. Details are discussed
later.
You can define your own non-linear curve by inheriting from NonlinearCurve and implementing the ValueAt and FillPartialDerivatives methods. The latter
takes two arguments. The first is the x value where the partial derivatives should be evaluated. The second
is a Vector whose elements are, on exit, the
partial derivatives with respect to each of the curve parameters. If you don't override
FillPartialDerivatives, then a numerical approximation is used.
The example below defines a hyperbola 1 / (x + c), with one parameter c.
| C# | Copy Code |
|---|
public class Hyperbola : NonlinearCurve
{
public Hyperbola() : base(1)
{
Parameters[0] = 1;
}
override public double ValueAt(double x)
{
return 1 / (x + Parameters[0]);
}
override public double DerivativeAt(double x)
{
x += Parameters[0];
return -1 / (x*x);
}
override public void FillPartialDerivatives(double x, GeneralVector derivatives)
{
x += Parameters[0];
derivatives[0] = -1 / (x*x);
}
} |
| Visual Basic | Copy Code |
|---|
Public Class Hyperbola : Inherits NonlinearCurve
Public Sub New()
MyBase.New(1)
Parameters(0) = 1
End Sub
Public Overrides Function ValueAt(ByVal x As Double) As Double
Return 1 / (x + Parameters(0))
End Function
Public Overrides Function DerivativeAt(ByVal x As Double) As Double
x = x + Parameters(0)
Return -1 / (x * x)
End Function
Public Overrides Sub FillPartialDerivatives(ByVal x As Double, _
ByVal derivatives As GeneralVector)
x = x + Parameters(0)
derivatives(0) = -1 / (x * x)
End Sub
End Class |
The curve constructor must call the base constructor with as its only argument the number of parameters of the
curve. For calculations, the curve parameters are accessed through the Parameters collection.
Depending on the application, it may be useful to override other methods of the Curve class. For
example, for the above curve, the Integral is easily
calculated as Math.Log(x+c).
The NonlinearCurveFitter class
The NonlinearCurveFitter class performs a
nonlinear least squares fit.
To perform the fit, a NonlinearCurveFitter needs data points, and a curve to fit. You must set the
Curve property to an instance of a NonlinearCurve object. This can be an instance of any of the
classes in the Extreme.Mathematics.Curves.Nonlinear namespace, or you can supply your own, as discussed
in the previous section.
The data is supplied as Vector objects. The
XValues and YValues properties specify the X and Y values of the data
points, respectively.
By default, the fit is unweighted. All weights are set equal to 1. Weights can be specified in one of two ways.
The first way is to set the WeightVector
property to a vector with as many components as there are data points. Each component of the vector specifies the
weight of the corresponding data point.
Alternatively, the WeightFunction can be
used to calculate the weights based on the data points. The WeightFunctions class contains a number of predefined weight
functions listed in the table below. In the descriptions, x and y stand for the x and y values of
each data point.
| Field |
Description |
| OneOverX |
The weight equals 1/x. |
| OneOverXSquared |
The weight equals 1/x2. |
| OneOverY |
The weight equals 1/y. |
| OneOverYSquared |
The weight equals 1/y2. |
If error estimates for the Y values are available, then the GetWeightVectorFromErrors of the
WeightFunctions class can be used to construct a vector with the weights that correspond to the errors.
Its only argument is a Vector that contains the error
for each observation.
The Optimizer property gives access
to the multidimensional optimizer that is used to compute the least squares solution. You can set tolerances and
control convergence as set out in the chapter on optimization.
The Fit method performs the actual least
squares calculation, and adjusts the parameters of the Curve to correspond to the least squares fit. The
BestFitParameters property returns a
Vector containing the parameters of the curve. The
GetStandardDeviations returns a
Vector containing the standard deviations of each
parameter as estimated in the least squares calculation.
The Optimizer property also gives access to details of the calculation. It is useful to inspect its
Status property to make sure the algorithm ended
normally. A value other than Converged indicates some kind of problem with the algorithm.
Example: Fitting a Dose Response Curve
In the first example, we fit dose response data to a four parameter logistic curve.
| C# | Copy Code |
|---|
Vector dose = new GeneralVector(/*...*/);
Vector response = new GeneralVector(/*...*/);
Vector error = new GeneralVector(/*...*/);
FourParameterLogisticCurve doseResponseCurve
= new FourParameterLogisticCurve();
NonlinearCurveFitter fitter = new NonlinearCurveFitter();
fitter.Curve = doseResponseCurve;
fitter.XValues = dose;
fitter.YValues = response;
fitter.WeightVector = WeightFunctions.GetWeightVectorFromErrors(error);
fitter.Fit();
Console.WriteLine("Initial value: {0,10:F6} +/- {1:F4}",
doseResponseCurve.InitialValue, s[0]);
Console.WriteLine("Final value: {0,10:F6} +/- {1:F4}",
doseResponseCurve.FinalValue, s[1]);
Console.WriteLine("Center: {0,10:F6} +/- {1:F4}",
doseResponseCurve.Center, s[2]);
Console.WriteLine("Hill slope: {0,10:F6} +/- {1:F4}",
doseResponseCurve.HillSlope, s[3]); |
| Visual Basic | Copy Code |
|---|
Dim dose As Vector = New GeneralVector(' ...
Dim response As Vector = New GeneralVector(' ...
Dim errors As Vector = New GeneralVector(' ...
Dim doseResponseCurve As FourParameterLogisticCurve _
= New FourParameterLogisticCurve
fitter.Curve = doseResponseCurve
fitter.XValues = dose
fitter.YValues = response
fitter.WeightVector = WeightFunctions.GetWeightVectorFromErrors(errors)
fitter.Fit()
Console.WriteLine("Initial value: {0,10:F6} +/- {1:F4}", _
doseResponseCurve.InitialValue, s(0))
Console.WriteLine("Final value: {0,10:F6} +/- {1:F4}", _
doseResponseCurve.FinalValue, s(1))
Console.WriteLine("Center: {0,10:F6} +/- {1:F4}", _
doseResponseCurve.Center, s(2))
Console.WriteLine("Hill slope: {0,10:F6} +/- {1:F4}", _
doseResponseCurve.HillSlope, s(3)) |
Two things are worth noting. One is the use of the GetWeightVectorFromErrors method to
transform the errors of the observations into their corresponding weights. Also, the FourParameterLogisticCurve class exposes
properties that name the curve properties. Many of the predefined curves define such named parameters.
Up: Curve Fitting Next: Predefined Nonlinear Curves Previous: Linear Curve Fitting Contents
Copyright 2004-2008,
Extreme Optimization. All rights reserved.
Extreme Optimization, Complexity made simple, M#, and M
Sharp are trademarks of ExoAnalytics Inc.
Microsoft, Visual C#, Visual Basic, Visual Studio, Visual
Studio.NET, and the Visual Studio Logo are registered trademarks of Microsoft Corporation