Extreme Optimization >
User's Guide >
Vector and Matrix Library >
Matrices >
Solving Least Squares Problems
Extreme Optimization User's Guide
User's Guide
Up: Matrices Next: Structured Matrix Types Previous: Solving Systems of Linear Equations Contents
Solving Least Squares Problems
The Extreme Optimization Mathematics Library for .NET
supports a simple mechanism for solving linear least squares
problems.
The LeastSquaresSolver Class
The LeastSquaresSolver
class implements the calculation of least squares solutions. The
constructor takes two arguments: the matrix of observations, and
the vector of outcomes:
| C# |  Copy Code |
|---|
Matrix a = new GeneralMatrix(6, 4, new double[]
{
1, 1, 1, 1, 1, 1,
1, 2, 3, 4, 5, 6,
1, 4, 9, 16, 25, 36,
1, 2, 1, 2, 1, 2
});
Vector b = new GeneralVector(1, 3, 6, 11, 15, 21);
LeastSquaresSolver solver = new LeastSquaresSolver(a, b); |
| Visual Basic | Copy Code |
|---|
Dim a As Matrix = New GeneralMatrix(6, 4, New Double() _
{ _
1, 1, 1, 1, 1, 1, _
1, 2, 3, 4, 5, 6, _
1, 4, 9, 16, 25, 36, _
1, 2, 1, 2, 1, 2 _
})
Dim b As Vector = New GeneralVector(1, 3, 6, 11, 15, 21)
Dim solver As LeastSquaresSolver = New LeastSquaresSolver(a, b) |
Least squares problems can be solved using a variety of
techniques. The method to use is specified by the
SolutionMethod property. This property is of type
LeastSquaresSolutionMethod
and can take on the following values:
| Value |
Description |
NormalEquations |
The solution is found by solving the normal equations. This
method is often the fastest, but is also less stable. |
QRDecomposition |
The solution is found by first calculating the QR decomposition
of the observation matrix. This is the default. |
CompleteOrthogonalDecomposition |
The solution is found by first calculating the complete
orthogonal decomposition of the observation matrix. This is a
rank-revealing enhancement of the QR decomposition. (Reserved for
future use.) |
SingularValueDecomposition |
The solution is found by first calculating the singular value
decomposition of the observation matrix. This is the slowest but
most robust method. |
The Solve
method performs the actual calculation. This method
returns a Vector
containing the least squares solution. This vector is also
available through the Solution
property.
| C# | Copy Code |
|---|
solver.SolutionMethod = LeastSquaresSolutionMethod.QRDecomposition;
// The Solve method calculates the solution:
Vector x = solver.Solve();
Console.WriteLine("x = {0}", x.ToString("F4"));
// The Solution property also returns the solution:
Console.WriteLine("x = {0}", solver.Solution.ToString("F4")); |
| Visual Basic | Copy Code |
|---|
solver.SolutionMethod = LeastSquaresSolutionMethod.QRDecomposition
' The Solve method calculates the solution:
Dim x As Vector = solver.Solve()
Console.WriteLine("x = {0}", x.ToString("F4"))
' The Solution property also returns the solution:
Console.WriteLine("x = {0}", solver.Solution.ToString("F4")) |
Various other methods let you retrieve more information about
the least squares solution. The
GetPredictions method returns the vector of the
outcomes predicted by the solution. The
GetResiduals method returns the residuals: the
difference between the predicted and actual outcome.
By default, the least squares solver uses an algorithm based on
the QR decomposition of the matrix of observations. Depending
on the numerical properties of the observation matrix, it may be
acceptable to calculate the solution using the normal equations.
This is faster, especially for problems with many more observations
than variables, but the numerical accuracy tends to
be questionable for larger numbers of variables.
Methods using a complete orthogonal decomposition and the
singular value decomposition will be supported in the future.
Directly Solving the Normal Equations
In some instances, it is convenient and cheap to accumulate the
normal equations directly. If the condition of the observation
matrix is good enough, a least squares solution may be obtained by
solving the accumulated equations. The following example
illustrates this procedure:
| C# | Copy Code |
|---|
SymmetricMatrix aTa = SymmetricMatrix.FromOuterProduct(a);
Vector aTb = b * a;
Vector x = aTa.Solve(aTb);
Console.WriteLine("x = {0}", x.ToString("F4")); |
| Visual Basic | Copy Code |
|---|
Dim aTa As SymmetricMatrix = SymmetricMatrix.FromOuterProduct(a);
Dim aTb As Vector = b * a
Dim x As Vector = aTa.Solve(aTb)
Console.WriteLine("x = {0}", x.ToString("F4")) |
References
G. H. Golub, C. F. Van Loan, Matrix Computations
(3rd Ed), Johns Hopkins University Press, 1996.
Up: Matrices Next: Structured Matrix Types Previous: Solving Systems of Linear Equations Contents
Copyright 2004-2008,
Extreme Optimization. All rights reserved.
Extreme Optimization, Complexity made simple, M#, and M
Sharp are trademarks of ExoAnalytics Inc.
Microsoft, Visual C#, Visual Basic, Visual Studio, Visual
Studio.NET, and the Visual Studio Logo are registered trademarks of Microsoft Corporation