Extreme Optimization >
User's Guide >
Statistics Library >
Regression Analysis >
Nonlinear Regression
Extreme Optimization User's Guide
User's Guide
Up: Regression Analysis Next: Logistic Regression Previous: Polynomial Regression Contents
Nonlinear Regression
Multiple nonlinearregression is a technique to analyze a nonlinear
relationship between one or more independent variables and a dependent
variable. The values of the independent variables are considered to be exact,
while the values of the dependent variables are subject to error. The
Extreme Optimization Statistics Library for .NET supports nonlinear
regression through the NonlinearRegressionModel class.
This class currently supports only one independent variable.
Constructing Nonlinear Regression Models
The NonlinearRegressionModel class has five constructors.
The first constructor takes no arguments. You have to set the dependent and independent variables by
adding them to the model's DependentVariables
and IndependentVariables
collections.
The second constructor takes two parameters. The first is a NumericalVariable that
represents the dependent variable. The second is a NumericalVariable that
represent the independent variables.
| C# |  Copy Code |
|---|
NumericalVariable dependent = new NumericalVariable("y", yData);
NumericalVariable independent = new NumericalVariable("x1", x1Data);
NonlinearRegressionModel model1 = new NonlinearRegressionModel(dependent, independent); |
| Visual Basic | Copy Code |
|---|
Dim dependent As NumericalVariable = New NumericalVariable("y", yData)
Dim independent As NumericalVariable = New NumericalVariable("x1", xData)
Dim model1 As NonlinearRegressionModel = _
New NonlinearRegressionModel(dependent, independent) |
The third constructor takes two Vector parameters.
The first contains the data for the independent variable. The second contains the data for the dependent variable.
The fourth constructor takes 3 parameters. The first parameter is a
VariableCollection object that contains the variables to be used in the
regression. The second parameter is a string containing the name of the
dependent variable. The third parameter is a string containing
the name of the independent. All the names must exist in
the collection specified by the first parameter. All variables must be of type NumericalVariable.
| C# | Copy Code |
|---|
VariableCollection variables = new VariableCollection();
variables.Add(dependent);
variables.Add(independent);
NonlinearRegressionModel model2 = new NonlinearRegressionModel(variables, "y", "x1"); |
| Visual Basic | Copy Code |
|---|
Dim variables As VariableCollection = New VariableCollection()
variables.Add(dependent)
variables.Add(independent)
Dim model2 As NonlinearRegressionModel = _
New NonlinearRegressionModel(variables, "y", "x1") |
The fifth constructor also takes 3 parameters. The first parameter is
a DataTable object that contains the data for the regression analysis. The
second parameter is a string containing the name of the column that contains the
data for the dependent variable. The third parameter is a string containing the
name of the column that contains the data for the independent variable. Both columns
must be numerical or convertible to numerical values.
| C# | Copy Code |
|---|
// Fill data table with data from some datasource.
NonlinearRegressionModel model3 = new NonlinearRegressionModel(table, "y", "x1"); |
| Visual Basic | Copy Code |
|---|
Dim table As DataTable = New DataTable()
' Fill data table with data from some datasource.
Dim model3 As NonlinearRegressionModel = _
New NonlinearRegressionModel(table, "y", "x1") |
Defining the model
A nonlinear model is defined by a function that is nonlinear in the curve parameters and the independent variable.
Such a function is represented by the NonlinearCurve class
in the Extreme.Mathematics.Curves namespace
.
To define the model, set the model'sCurve
to an instance of the curve you want to use. The Extreme.Mathematics.Curves.Nonlinear
namespace defines a number of nonlinear curves. This includes exponential, rational and logistic curves.
You can also define your own nonlinear curves. For details, see the chapter on Curve Fitting in the
Mathematics Library for .NET User's Guide.
Computing the Regression
The Compute
method performs the actual analysis. Most properties and methods throw an exception when they are accessed before
the Compute method is called. You can verify that the model has been calculated by inspecting the
Computed property.
| C# | Copy Code |
|---|
model1.Compute(); |
| Visual Basic | Copy Code |
|---|
model1.Compute() |
The PredictedValues property
returns a Vector
that contains the values of the dependent variable as predicted by the model.
The Residuals property returns a vector containing the difference between the
actual and the predicted values of the dependent variable. Both vectors contain
one element for each observation.
Regression Parameters
The
NonlinearRegressionModel class' Parameters
property returns a ParameterCollection
object that contains the parameters of the regression model. The members of
this collection are of type Parameter.
Regression parameters are created by the model. You cannot create them
directly.
Parameters can be accessed by numerical index or by name. Parameters are automatically given the names A0, A1, and so on.
The Parameter class has four useful properties. The Value property
returns the numerical value of the parameter, while the StandardError
property returns the standard deviation of the parameter's distribution.
The Statistic
property returns the value of the t-statistic corresponding to the hypothesis
that the parameter equals zero. The PValue
property returns the corresponding p-value. A high p-value indicates that
the variable associated with the parameter does not make a
significant contribution to explaining the data. The p-value always corresponds
to a two-tailed test. The following
example prints the properties of the slope parameter of our
earlier example:
| C# | Copy Code |
|---|
Parameter x1Parameter = model1.Parameters["x1"];
Console.WriteLine("Name: {0}", x1Parameter.Name);
Console.WriteLine("Value: {0}", x1Parameter.Value);
Console.WriteLine("St.Err.: {0}", x1Parameter.StandardError);
Console.WriteLine("t-statistic: {0}", x1Parameter.TStatistic);
Console.WriteLine("p-value: {0}", x1Parameter.PValue); |
| Visual Basic | Copy Code |
|---|
Dim x1Parameter As = model1.Parameters("x1")
Console.WriteLine("Name: {0}", x1Parameter.Name)
Console.WriteLine("Value: {0}", x1Parameter.Value)
Console.WriteLine("St.Err.: {0}", x1Parameter.StandardError)
Console.WriteLine("t-statistic: {0}", x1Parameter.TStatistic)
Console.WriteLine("p-value: {0}", x1Parameter.PValue) |
The Parameter class has one method: GetConfidenceInterval.
This method takes one parameter: a confidence level between 0 and 1. A value of
0.95 corresponds to a confidence level of 95%. The method returns the confidence
interval for the parameter at the specified confidence level as an Interval
structure.
Verifying the Quality of the Regression
The ResidualSumOfSquares
property gives the sum of the squares of the residuals. The regression line was
found by minimizing this value.
The StandardError
property gives the standard deviation of the data.
The
RSquared
property returns the coefficient of determination. It is the ratio of the
variation in the data that is explained by the model compared to the total
variation in the data. Its value is always between 0 and 1, where 0 means the
model explains nothing and 1 means the model explains the data perfectly.
An entirely different assessment is available through an analysis of
variance. Here, the variation in the data is decomposed into a component
explained by the model, and the variation in the residuals. The FStatistic
property returns the F-statistic for the ratio of these two variances. The
PValue
property returns the corresponding p-value. A low p-value means that it is
unlikely that the variation in the model is the same as the variation in the
residuals. This means that the model is significant.
The results of the analysis of variance are also summarized in the regression
model's ANOVA table, returned by the AnovaTable
property.
Up: Regression Analysis Next: Logistic Regression Previous: Polynomial Regression Contents
Copyright 2004-2008,
Extreme Optimization. All rights reserved.
Extreme Optimization, Complexity made simple, M#, and M
Sharp are trademarks of ExoAnalytics Inc.
Microsoft, Visual C#, Visual Basic, Visual Studio, Visual
Studio.NET, and the Visual Studio Logo are registered trademarks of Microsoft Corporation