Extreme Optimization™: Complexity made simple.

Math and Statistics
Libraries for .NET

  • Home
  • Features
    • Math Library
    • Vector and Matrix Library
    • Statistics Library
    • Performance
    • Usability
  • Documentation
    • Introduction
    • Math Library User's Guide
    • Vector and Matrix Library User's Guide
    • Data Analysis Library User's Guide
    • Statistics Library User's Guide
    • Reference
  • Resources
    • Downloads
    • QuickStart Samples
    • Sample Applications
    • Frequently Asked Questions
    • Technical Support
  • Blog
  • Order
  • Company
    • About us
    • Testimonials
    • Customers
    • Press Releases
    • Careers
    • Partners
    • Contact us
Introduction
Deployment Guide
Nuget packages
Configuration
Using Parallelism
Expand Mathematics Library User's GuideMathematics Library User's Guide
Expand Vector and Matrix Library User's GuideVector and Matrix Library User's Guide
Expand Data Analysis Library User's GuideData Analysis Library User's Guide
Expand Statistics Library User's GuideStatistics Library User's Guide
Expand Data Access Library User's GuideData Access Library User's Guide
Expand ReferenceReference
  • Extreme Optimization
    • Features
    • Solutions
    • Documentation
    • QuickStart Samples
    • Sample Applications
    • Downloads
    • Technical Support
    • Download trial
    • How to buy
    • Blog
    • Company
    • Resources
  • Documentation
    • Introduction
    • Deployment Guide
    • Nuget packages
    • Configuration
    • Using Parallelism
    • Mathematics Library User's Guide
    • Vector and Matrix Library User's Guide
    • Data Analysis Library User's Guide
    • Statistics Library User's Guide
    • Data Access Library User's Guide
    • Reference
  • Vector and Matrix Library User's Guide
    • Basic Concepts
    • Vectors
    • Matrices
    • Structured Matrix Types
    • Matrix Decompositions
    • Sparse Vectors and Matrices
    • Complex Linear Algebra
    • Single-Precision Linear Algebra
    • Distributed and GPU Computing
  • Matrices
    • Matrix Basics
    • Dense Matrices
    • Accessing Rows and Columns
    • Mathematical Properties
    • Operations on Matrices
    • Solving Systems Of Linear Equations
    • Solving Least Squares Problems
  • Solving Least Squares Problems

Solving Least Squares Problems

Extreme Optimization Numerical Libraries for .NET Professional

The Extreme Optimization Numerical Libraries for .NET support a simple mechanism for solving linear least squares problems.

The LeastSquaresSolver Class

The LeastSquaresSolverT class implements the calculation of least squares solutions. The constructor takes two arguments: the matrix of observations, and the vector of outcomes:

Least squares problems can be solved using a variety of techniques. The method to use is specified by the SolutionMethod property. This property is of type LeastSquaresSolutionMethod and can take on the following values:

Value

Description

NormalEquations

The solution is found by solving the normal equations. This method is often the fastest, but is also less stable.

QRDecomposition

The solution is found by first calculating the QR decomposition of the observation matrix. This is the default.

SingularValueDecomposition

The solution is found by first calculating the singular value decomposition of the observation matrix. This is the slowest but most robust method.

NonNegative

The solution is restricted to values that are positive or zero.

The Solve method performs the actual calculation. This method returns a VectorT containing the least squares solution. This vector is also available through the Solution property.

Various other methods let you retrieve more information about the least squares solution. The GetPredictions method returns the vector of the outcomes predicted by the solution. The GetResiduals method returns the residuals: the difference between the predicted and actual outcome.

By default, the least squares solver uses an algorithm based on the QR decomposition of the matrix of observations. Depending on the numerical properties of the observation matrix, it may be acceptable to calculate the solution using the normal equations. This is faster, especially for problems with many more observations than variables, but the numerical accuracy tends to be questionable for larger numbers of variables.

Methods using a complete orthogonal decomposition and the singular value decomposition will be supported in the future.

Directly Solving the Normal Equations

In some instances, it is convenient and cheap to accumulate the normal equations directly. If the condition of the observation matrix is good enough, a least squares solution may be obtained by solving the accumulated equations. The following example illustrates this procedure:

C#
VB
C++
F#
Copy
SymmetricMatrix aTa = SymmetricMatrix.FromOuterProduct(a);
Vector aTb = b * a;
Vector x = aTa.Solve(aTb);
Console.WriteLine("x = {0}", x.ToString("F4"));
Dim aTa As SymmetricMatrix = SymmetricMatrix.FromOuterProduct(a);
Dim aTb As Vector = b * a
Dim x As Vector = aTa.Solve(aTb)
Console.WriteLine("x = {0}", x.ToString("F4"))

No code example is currently available or this language may not be supported.

No code example is currently available or this language may not be supported.

Non-Negative Least Squares

It is sometimes necessary to restrict the solution to values that are positive or zero. To compute such a non-negative least squares solution, set the SolutionMethod property to NonNegative.

References

G. H. Golub, C. F. Van Loan, Matrix Computations (3rd Ed), Johns Hopkins University Press, 1996.

Copyright (c) 2004-2021 ExoAnalytics Inc.

Send comments on this topic to support@extremeoptimization.com

Copyright © 2004-2021, Extreme Optimization. All rights reserved.
Extreme Optimization, Complexity made simple, M#, and M Sharp are trademarks of ExoAnalytics Inc.
Microsoft, Visual C#, Visual Basic, Visual Studio, Visual Studio.NET, and the Optimized for Visual Studio logo
are registered trademarks of Microsoft Corporation.