Regression Sample

The Extreme Optimization Numerical Libraries for .NET supports simple and multiple linear regression with optional transformation of variables. The Regress sample illustrates these capabilities.

About Linear Regression
How to use the program
The sample code
Handling the data
Performing the calculation

About Linear Regression

Linear regression is one of the basic techniques of statistical analysis. The somewhat peculiar name derives from the original investigations by sir Francis Galton in the 19th century.

Linear regression is a means of establishing a linear relationship between a dependent variable and one or more independent variables. In simple regression there is only one independent variable. In multiple regression, there are more than one independent variables.

The analysis gives information about the signficance of the contributions of the independent variables, and allows you to predict the values of the dependent variable for new values of the independent variables.

How to use the program

The Regress program has a simple tab based interface.

The first tab lets you select the dataset. It shows a dropdown list and a grid. When you select a dataset from the dropdown list, the data appears in the grid. The data is stored in an Access database, and are taken from the National Institute for Standards and Technology’s Statistical Reference Datasets website. The Institute provides certified results that were calculated using very high precision (500 digits).

Once a dataset has been selected, you can construct the model using the Model tab. This tab has three parts. On the left is a list of numerical variables in the selected dataset. In the middle are a text box for the dependent variable, a list of independent variables, and a series of companion buttons. On the right is a panel with several model options.

You select the dependent variable by selecting it in the variables list and clicking the ‘Set’ button next to the dependent variable box. You can clear the dependent variable by clicking on the clear button.

You select the independent variables by selecting them one at a time from the variables list and clicking the ‘Add’ button. To remove an independent variable, click the remove button.

The options panel has two checkboxes. The first lets you specify whether to include a constant term or intercept in the model. The second checkbox lets you perform a polynomial regression. You must also specify the degree of the polynomial, which must be less than the number of data points. There can only be one independent variable for this option to be available.

Clicking the ‘Calculate’ button performs the actual calculation, and brings you to the Regression tab. The global properties of the regression model such as residual error and the R-squared value are given in the top half of the page. The bottom half is taken up by a list of the regression parameters and their standard errors.

Finally, the Anova Table tab shows the ANOVA table for the regression model.

The sample code

Most of the code in this sample deals with the user interface.

Handling the data

A DataFrame<R,C> is the object that corresponds to a statistical dataset. A dataframe can be created in a variety of ways. One of the simplest is illustrated here: a DataTable or DataTableView object can be transformed directly into a VariableCollection as follows:

[C#]
// Load the DataTableDataTable dt = null;
OleDbDataAdapter da = new OleDbDataAdapter("Select * from " + tableName, cnn);
dt = new DataTable();
da.Fill(dt);

// Create the variables collection straight from the DataTable:
variables = new VariableCollection(dt);

[Visual Basic]
' Load the DataTable
Dim dt As DataTable
Dim da As OleDbDataAdapter = new OleDbDataAdapter("Select * from " + tableName, cnn)
dt = New DataTable()
da.Fill(dt)

' Create the variables collection straight from the DataTable:
variables = New VariableCollection(dt)

Variables can also be created from arrays or DataColumn objects, or they can be returned as the result of a calculation or transformation involving one or more other variables.

Performing the calculation

Most of the really interesting code is contained in the Calculate method:

[C#]
private void cmdCalculate_Click(object sender, System.EventArgs e)
{
    if (!CanCompute())
        return;

    // Set up the model:    LinearRegressionModel model;
    if (chkPolynomial.Checked)
    {
        // Get the degree of the polynomial:        int degree;
        try        {
            degree = Int32.Parse(txtDegree.Text);
            if (degree >= variables.RowCount)
            {
                MessageBox.Show(
                    "Degree must be an integer less than the number of data points.");
                return;
            }
        }
        catch (FormatException)
        {
            MessageBox.Show(
                "Degree must be an integer less than the number of data points.");
            return;
        }
        // Finally, create the model.        model = new PolynomialRegressionModel(variables,
            txtDependent.Text, lstIndependent.Items[0].ToString(), degree);
    }
    else    {
        // For a 'classic' multiple regression, we can use the variable names instead:        string[] independents = new string[lstIndependent.Items.Count];
        lstIndependent.Items.CopyTo(independents, 0);
        string dependent = txtDependent.Text;
        // Now create the model:        model = new LinearRegressionModel(variables, dependent, independents);
    }

    // Check if we want an intercept in the model:    model.NoIntercept = chkNoIntercept.Checked;
    // Compute the model    model.Compute();
    // Show results    // ...}

[Visual Basic]
 Private Sub cmdCalculate_Click(ByVal sender As Object, ByVal e As System.EventArgs)  _
    Handles cmdCalculate.Click

    If (Not CanCompute()) Then        Return    End If    ' Set up the model:    Dim model As LinearRegressionModel

    If (chkPolynomial.Checked) Then        ' Get the degree of the polynomial:        Dim degree As Integer        Try            degree = Int32.Parse(txtDegree.Text)
            If (degree >= variables.RowCount) Then                MessageBox.Show( _
                    "Degree must be an integer less than the number of data points.")
                Return            End If        Catch ex As FormatException
            MessageBox.Show( _
                "Degree must be an integer less than the number of data points.")
            Return        End Try        ' Finally, create the model.        model = New PolynomialRegressionModel(variables, _
            txtDependent.Text, lstIndependent.Items(0).ToString(), degree)
    Else        ' For a 'classic' multiple regression, we can use the variable names instead:        Dim independents As String() = New String(lstIndependent.Items.Count - 1) {}
        lstIndependent.Items.CopyTo(independents, 0)
        Dim dependent As String = txtDependent.Text

        ' Now create the model:        model = New LinearRegressionModel(variables, dependent, independents)
    End If    ' Check if we want an intercept in the model:    model.NoIntercept = chkNoIntercept.Checked

    ' Compute the model    model.Compute()
    
    ' Show results:    ' ...End Sub
    

The first step in computing the regression is creating the model. Regression models are implemented by the LinearRegressionModel class. The details depend on whether we want a polynomial regression or a simple linear regression.

In case of a polynomial regression, we use the PolynomialRegressionModel class, which inherits from LinearRegressionModel. In case of normal multiple regression, we can simply pass the names of the variables in the VariableCollection together with the collection to the constructor.

The actual calculation is performed by the Fit method.

The remainder of the Calculate method is spent filling in the data on the Regression and Anova Table tabs. Note that an AnovaTable object can be converted directly into a DataTable object for direct binding with a DataGrid.