Improve numerical stability of the Kalman filter for ARIMA #4259
Conversation
Nyrio
added
2 - In Progress
Nyrio
added
3 - Ready for Review
There was a problem hiding this comment.
Hi @Nyrio , thanks for this PR! It looks good, I just some smaller suggestions. In case we would need to return to this in the future: are there any references / links / keywords about the Kalman filter stability problem that might be worth adding to the PR description?
|
@tfeher Thanks for the review. I have updated the PR.
I added a mention of the Joseph form. The main numerical instability source in the Kalman filter is the covariance update, and this is especially true with the simplified formula where the subtraction can result in a loss of positive-definiteness. Hence the Joseph formula is a possible (expensive) fix. |
Codecov Report
@@ Coverage Diff @@
## branch-21.12 #4259 +/- ##
===============================================
Coverage ? 85.98%
===============================================
Files ? 231
Lines ? 18577
Branches ? 0
===============================================
Hits ? 15974
Misses ? 2603
Partials ? 0
Flags with carried forward coverage won't be shown. Click here to find out more. Continue to review full report at Codecov.
|
|
@gpucibot merge |
…4259) This PR improves the numerical stability of the Kalman filter by enforcing that the state covariance matrix is always symmetric and only has positive elements along its diagonal. Note that statsmodels does the former and not the latter, but I have found both to improve the stability in cases I was testing. If stability problems arise in the future, a strategy to consider would be to use [Joseph form](https://en.wikipedia.org/wiki/Kalman_filter#Deriving_the_posteriori_estimate_covariance_matrix) for the covariance update. This avoids the subtraction which can result in a loss of positive-definiteness, but at the expense of performance because this formula is more complex. Authors: - Louis Sugy (https://github.com/Nyrio) Approvers: - Tamas Bela Feher (https://github.com/tfeher) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4259
This PR improves the numerical stability of the Kalman filter by enforcing that the state covariance matrix is always symmetric and only has positive elements along its diagonal. Note that statsmodels does the former and not the latter, but I have found both to improve the stability in cases I was testing.
If stability problems arise in the future, a strategy to consider would be to use Joseph form for the covariance update. This avoids the subtraction which can result in a loss of positive-definiteness, but at the expense of performance because this formula is more complex.