Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an additional OLS implementation based on Cholesky Decomposition[Feature Request] #162

Closed
Recktenwald opened this issue Oct 31, 2021 · 0 comments · Fixed by #164
Closed

Comments

@Recktenwald
Copy link

Is your feature request related to a problem? Please describe.
I noticed that the standard implementation was relatively slow. Here is a code example

open FSharp.Stats
open FSharp.Stats.Distributions
open FSharp.Stats.Fitting.LinearRegression
let normal = Continuous.normal 0.0 1.0
let n = 10000
let xs = [|for _ in [1..n] do 4.0 * normal.Sample()|]
let us = [|for _ in [1..n] do 36.0 * normal.Sample()|]
let yData = vector (Array.map2 (fun x u -> 3.0+2.0*x + u) xs us)
let xData = Matrix.ofJaggedArray (Array.zip xs us |> Array.map (fun (a,b) -> [|a;b|]))
#time
let olsCoeffs = OrdinaryLeastSquares.Linear.Multivariable.coefficients xData yData

olsCoeffs // Real: 00:00:10.390, CPU: 00:00:11.515, GC gen0: 143, gen1: 3, gen2: 2
#time
let X = Matrix.init (xData.NumRows) (xData.NumCols+1) (fun i j -> if j = 0 then 1.0 else xData.[i,j-1])
let XT = Matrix.transpose X
let lower = (XT * X) |> Algebra.LinearAlgebra.Cholesky
let gamma = Algebra.LinearAlgebra.SolveTriangularLinearSystem (Matrix.transpose lower) (XT * yData) false 
let beta = Algebra.LinearAlgebra.SolveTriangularLinearSystem  lower gamma true
beta // Real: 00:00:00.093, CPU: 00:00:00.000, GC gen0: 0, gen1: 0, gen2: 0

The result is already slightly unstable, the coefficients deviating further from the true value than in the standard implementation, but it is fast and useful for exploratory analysis.

Describe the solution you'd like
Add an implementation based on the Cholesky Decomposition.

Describe alternatives you've considered
Waiting longer for results?

Additional context
I have already done it and I'm only opening this issue because the contribution guidelines say to do it. Also as a side note the PR Template is leading to a 404 ( https://github.com/fslaborg/FSharp.Stats/blob/developer/PULL_REQUEST_TEMPLATE.md )

This was referenced Oct 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant