Logistic Regression | Extreme Optimization Numerical Libraries for .NET Professional |

Logistic regression is a technique to analyze a situation where the outcome can have two possible values (binomial logistic regression). The model estimates the probability that an observation leads to either of the outcomes. A generalization to more than two outcomes, in the form of nominal logistic regression, is possible. The LogisticRegressionModel class implements logistic regression.

The LogisticRegressionModel class has four constructors.

The first constructor takes two arguments. The first is a
ICategoricalVector
that represents the dependent variable. The second is a parameter array of
Vector

var dependent = Vector.CreateCategorical(yData); var independent1 = Vector.Create(x1Data); var independent2 = Vector.Create(x2Data); var model1 = new LogisticRegressionModel(dependent, independent1, independent2);

The second constructor takes 3 arguments. The first argument is a
IDataFrame
(a DataFrame

var dataFrame = DataFrame.FromColumns(new Dictionary<string, object>() { { "y", dependent }, { "x1", independent1 }, { "x2", independent2 } }); var model2 = new LogisticRegressionModel(dataFrame, "y", "x1", "x2");

The third and fourth constructor are like the first two, but take arrays instead of parameter arrays, and also take two optional arguments. The first optional argument specifies a vector with weights for the observations. The second optional argument specifies the method: binary or nominal.

The Compute method performs the actual analysis. Most properties and methods throw an exception when they are accessed before the Compute method is called. You can verify that the model has been calculated by inspecting the Computed property.

The Predictions
property returns a CategoricalVector

The LogisticRegressionModel class' Parameters property returns a ParameterVector object that contains the parameters of the regression model. The elements of this vector are of type Parameter. Regression parameters are created by the model. You cannot create them directly.

Parameters can be accessed by numerical index or by name. The name of a parameter is usually the name of the variable associated with it.

A logistic regression model has as many parameters as there are independent variables, plus one for the intercept (constant term) when it is included. The intercept, if present, is the first parameter in the collection, with index 0.

The parameters of a logistic regression are assumed to have a normal distribution. This is an approximation that is valid if the number of observations is large enough. Parameters for other models are assumed to have a student-t distribution, which for all practical purposes is equivalent to a normal distribution.

The Parameter class has four useful properties. The Value property returns the numerical value of the parameter, while the StandardError property returns the standard deviation of the parameter's distribution.

The Statistic property returns the value of the z-statistic corresponding to the hypothesis that the parameter equals zero. The PValue property returns the corresponding p-value. A high p-value indicates that the variable associated with the parameter does not make a significant contribution to explaining the data. The p-value always corresponds to a two-tailed test. The following example prints the properties of the slope parameter of our earlier example:

var dataFrame = DataFrame.FromColumns(new Dictionary<string, object>() { { "y", dependent }, { "x1", independent1 }, { "x2", independent2 } }); var model2 = new LogisticRegressionModel(dataFrame, "y", "x1", "x2");

The Parameter class has one method: GetConfidenceInterval. This method takes one argument: a confidence level between 0 and 1. A value of 0.95 corresponds to a confidence level of 95%. The method returns the confidence interval for the parameter at the specified confidence level as an Interval structure.

Because a logistic regression model is computed by directly maximizing the likelihood function, it does not have the same range of diagnostic values available for linear regression models.

The GetLogLikelihood method returns the logarithm of the likelihood of the computed model. The regression parameters were computed by maximizing this value.

The quality of the model can be measured using a likelihood ratio test, available through the GetLikelihoodRatioTest method. Without parameters, this method returns a test for the hypothesis that all parameters that correspond to independent variables are zero. It compares the likelihood of the current model with a model that only has a constant term. The resulting statistic follows a chi-squared distribution. This method returns a SimpleHypothesisTest that can be used to verify the hypothesis at any significance level.

The GetLikelihoodRatioTest method has a second overload that takes another LogisticRegressionModel as its only argument. This model must be nested inside the current model: the dependent variable must be the same, and all independent variables in the nested model must also be in the current model. The test indicates whether the inclusion of additional independent variables in the current model produced a significantly better model.

The Wald test is a generalization of the z-test on individual parameters to multiple parameters, and takes into account interactions between the parameters. The test statistic follows a chi-squared distribution with degrees of freedom equal to the number of parameters included in the test.

Without parameters, the GetWaldTest method returns the Wald test for the entire model, including the constant term. This method returns a SimpleHypothesisTest object that can be used to verify the hypothesis at any significance level.

The GetWaldTest method can take one parameter, which must be an array of integers specifying the indexes of the parameters that are to be included in the test.

Copyright Â© 2004-20116,
Extreme Optimization. All rights reserved.

*Extreme Optimization,* *Complexity made simple*, *M#*, and *M
Sharp* are trademarks of ExoAnalytics Inc.

*Microsoft*, *Visual C#, Visual Basic, Visual Studio*, *Visual
Studio.NET*, and the *Optimized for Visual Studio* logo

are
registered trademarks of Microsoft Corporation.