Document specific dynamic programming strategies used #483
Comments
|
HARK does not solve models.
You, the user, use the tools in HARK to solve your model, by writing the
.solve() method for your model.
HARK structures the problem for you in a generic and uniform way, with the
aim of making it as easy as possible to write your solution algorithm. And
provides a number of canonical examples of problems that have been solved.
But it up to the user whether to use policy function iteration, value
function iteration, or some other method.
We almost always use policy function iteration. One of the many things on
the to-do list is to do some toy examples of value function iteration, for
comparison to PFI. But for powerful mathematical reasons, policy function
iteration is better when you can use it. Like, it can be 100 or 1000 times
faster, for any given desired degree of accuracy.
PS. A "consumption function" a policy function for consumption. A
portfolio share rule is the policy function for your portfolio choice. etc.
…On Fri, Jan 31, 2020 at 4:21 PM Sebastian Benthall ***@***.***> wrote:
HARK advertises that it solves the economic models it builds using dynamic
programming.
For users with more general CS background, it would be most helpful to
document the functionality of HARK in terms of more generic kinds of
problems and algorithms.
It's taken me quite some time to realize that all the AgentType models are
variations on a Markov Decision Process (MDP). This can be stated more
directly in the documentation.
For a computer scientist, the next question would be, "What algorithms are
you using to solve this MDP?" The answer is "dynamic programming". But
there are several different dynamic programming algorithms for solving
MDPs. For reference, see Netos' textbook, section 2.2:
http://users.isr.ist.utl.pt/~mtjspaan/readingGroup/learningNeto05.pdf
DP methods for solving MDP's include:
- Value iteration
- Policy iteration
- Generalized policy iteration?
Which are HARK using? And when?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#483?email_source=notifications&email_token=AAKCK76W2RAADR75Z2KQDWTRAQ6YJA5CNFSM4KOIXACKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IKFMCJQ>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAKCK73AONXHD6XZGDU2IALRAQ6YJANCNFSM4KOIXACA>
.
--
- Chris Carroll
|
|
I feel like there are some split hairs here. I think I see what you're saying though:
It sounds like for a HARK 2.0, it would be ideal if the HARK core provided even more generic utilities, like a generic framework for policy iteration and value iteration. That way, it would be even easier to write new solutions. |
|
It sounds like for a HARK 2.0, it would be ideal if the HARK core
provided even more generic utilities, like a generic framework for policy
iteration and value iteration.
You're getting closer.
But the point is that you can't have a generic framework for the solution
of a problem, until you have a generic framework for saying what the
problem IS. DolARK is more about saying what the problem is, than it is
about how best to solve it. HARK is the opposite: It solves a bunch of
problems efficiently without stating/expressing them in a generic way
(well, the math is generic, but not the code). These two things need to be
merged.
…On Sat, Feb 1, 2020 at 12:56 AM Sebastian Benthall ***@***.***> wrote:
I feel like there are some split hairs here. I think I see what you're
saying though:
- HARK.core tries to provide a generic setup for problems. This has to
be extended with solution algorithms, the solve() method.
- Some problems have already been solved in HARK.ConsumptionSaving.
These have been solved with policy function iteration, for the reason you
say.
It sounds like for a HARK 2.0, it would be ideal if the HARK core provided
even more generic utilities, like a generic framework for policy iteration
and value iteration.
That way, it would be even easier to write new solutions.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#483?email_source=notifications&email_token=AAKCK75NREJNKU4MNG3R5GLRAS3EHA5CNFSM4KOIXACKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKQMEGI#issuecomment-580960793>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAKCK73UJT4SOZHVJJV5JETRAS3EHANCNFSM4KOIXACA>
.
--
- Chris Carroll
|
|
These are my notes on the 2.0 plan so far. |
|
This is some interesting work on generic solution algorithms for continuous MDP's. |
|
This work by Feng et a. 2004 discusses dynamic programming algorithms for continuous MDPs: |
|
My understanding is that HARK and Dolo both solve the continuous MDP problem through discretization. A general description of discretization for solving these kinds of problems is here: I wonder what language would best describe the particular discretization techniques used in HARK, and for which models, and how to standardize the inclusion of this language in the documentation. |
|
Interpolation is a key step here. |
|
Here is the best account of the methods Chris has found useful for optimization problems: |
|
See #362 -- this can go in the engineering "Journey" |
HARK advertises that it solves the economic models it builds using dynamic programming.
For users with more general CS background, it would be most helpful to document the functionality of HARK in terms of more generic kinds of problems and algorithms.
It's taken me quite some time to realize that all the AgentType models are variations on a Markov Decision Process (MDP). This can be stated more directly in the documentation.
For a computer scientist, the next question would be, "What algorithms are you using to solve this MDP?" The answer is "dynamic programming". But there are several different dynamic programming algorithms for solving MDPs. For reference, see Netos' textbook, section 2.2:
http://users.isr.ist.utl.pt/~mtjspaan/readingGroup/learningNeto05.pdf
DP methods for solving MDP's include:
Which are HARK using? And when?
The text was updated successfully, but these errors were encountered: