Complete deprecation of HARK.core.Solution

[sbenthall]

Trying to work through the core logic of HARK...it looks like there is a deprecated Solution class, but that there is some dead code still lying around because of it.

HARK/HARK/core.py

Lines 156 to 163 in eb8d1a6

	class Solution(HARKobject):
	'''
	A superclass for representing the "solution" to a single period problem in a
	dynamic microeconomic model.
	NOTE: This can be deprecated now that HARKobject exists, but this requires
	replacing each instance of Solution with HARKobject in the other modules.
	'''

This ticket is for completing the deprecation of HARK.core.Solution

[mnwhite]

I think this might have been placeholder code that I wrote in 2015. I think my reasoning was that there might eventually be methods/features that are universal across "solution" type objects, but that we don't want in our generic "HARKobject" extension of object. 4.5 years later, I haven't thought of any, so... deprecate away?

…

On Fri, Jan 31, 2020 at 9:55 AM Sebastian Benthall ***@***.***> wrote: Trying to work through the core logic of HARK...it looks like there is a deprecated Solution class, but that there is some dead code still lying around because of it. https://github.com/econ-ark/HARK/blob/eb8d1a683522a592ef8f1a29c4b4e4ec6d680b8a/HARK/core.py#L156-L163 This ticket is for completing the deprecation of HARK.core.Solution — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#481?email_source=notifications&email_token=ADKRAFMPW33LITRK4E5LROLRAQ3WNA5CNFSM4KOIJ7C2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IKFE5PQ>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADKRAFJKX5TT5OOHSJZ7QW3RAQ3WNANCNFSM4KOIJ7CQ> .

[llorracc]

@sbenthall are you joining the Zoom conv going on right now

[sbenthall]

@mnwhite That's interesting. I'm trying to understand the architecture of the core solution code.
This relates to #482

It looks like the class ConsumerSolution inherits from Solution.

HARK/HARK/ConsumptionSaving/ConsIndShockModel.py

Line 46 in ad83315

class ConsumerSolution(Solution):

This class has many parameters that are specific to the model which could be abstracted away in a more general class.

For example, rather than hard-coding the consumption function, taking market resources as input, as cFunc, in general MDP's support a "policy" that maps* from state variables to actions.

I think it would be smart to back up as much general logic to more abstract classes, if possible. I'm still trying to work out to what degree that's possible.

[mnwhite]

Yes, that's one of the main things we realized was a mistake in like 2016, and that we want to change in "HARK 2.0". Our model solutions should have a generic object that does something like *controls = solution[t].PolicyFunc(*states), where both controls and states are generic lists, or possibly named tuples. This will also make it easier to have HARK interface with some way to specify the mathematical form of the model in a machine-readable way, e.g. by specifying the names of states, controls, shocks, and post-states. I think this will require a significant rewrite, and I just haven't had the bandwidth for it. But it's important.

…

On Fri, Jan 31, 2020 at 2:46 PM Sebastian Benthall ***@***.***> wrote: @mnwhite <https://github.com/mnwhite> That's interesting. I'm trying to understand the architecture of the core solution code. This relates to #482 <#482> It looks like the class ConsumerSolution inherits from Solution. https://github.com/econ-ark/HARK/blob/ad833152145529dc0f0f47d6c575a95a542967cd/HARK/ConsumptionSaving/ConsIndShockModel.py#L46 This class has many parameters that are specific to the model which could be abstracted away in a more general class. For example, rather than hard-coding the consumption function, taking market resources as input, as cFunc, in general MDP's support a "policy" that maps* from state variables to actions. I think it would be smart to back up as much general logic to more abstract classes, if possible. I'm still trying to work out to what degree that's possible. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#481?email_source=notifications&email_token=ADKRAFJVVX6NSPKQCOMZ5G3RAR5ZPA5CNFSM4KOIJ7C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKPZD2I#issuecomment-580882921>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADKRAFOCW756D4HR4JYJ373RAR5ZPANCNFSM4KOIJ7CQ> .

[sbenthall]

Ok, gotcha. I'm glad to hear this is the intended direction.

I may have the bandwidth to work towards this. I think it's going to be necessary for any kind of significant integration with Dolo.

As you say, it may require a big rewrite to really effect this change. But I wonder if there's any more incremental steps towards it. I'll see what I can do.

[llorracc]

Basically, this is exactly what integration with Dolo is all about. Dolo didn't exist when we started HARK (except maybe in a very embryonic state). And many of the models in HARK involve mathematical structures or tools or approaches that cannot yet be described in Dolo or in any other computer language. That is, in good part, what the DolARK project is about: Enriching the syntax of Dolo so that it can handle the kinds of problems we are solving in HARK. Dolo's origins were in the class of representative agent models, and there are many aspects of microeconomic reality (kinks, discontinuities, etc) that can't really happen in RA models. The way things have traditionally been done in HA modeling is to describe your problem, in your paper, in the rigorous, clear mathematical language of MDP's (any kind of MDP) with the further assumption that the decision rule should be the optimal solution to the problem. That's what Bellman's equation describes. But then, everybody would make up their own computer code from scratch to solve the problem themselves in whatever way they thought of. That is the wilderness we are trying to bring some structure and order to. So the first step in integrating HARK and DolARK is to take each HARK model and provide the corresponding representation of that model using the DolARK syntax. That is exactly what our chimeras do. Unlike HARK, DolARK DOES have generic solution algorithms, but they are too simple to solve many of the problems that need to be solved.

…

On Sat, Feb 1, 2020 at 12:19 AM Sebastian Benthall ***@***.***> wrote: Ok, gotcha. I'm glad to hear this is the intended direction. I may have the bandwidth to work towards this. I think it's going to be necessary for any kind of significant integration with Dolo. As you say, it may require a big rewrite to really effect this change. But I wonder if there's any more incremental steps towards it. I'll see what I can do. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#481?email_source=notifications&email_token=AAKCK72SCCYXWNWAHPIYMJ3RASWYFA5CNFSM4KOIJ7C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKQKHWA#issuecomment-580953048>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKCK72BMPXT65V4VVWWAE3RASWYFANCNFSM4KOIJ7CQ> .

-- - Chris Carroll

[sbenthall]

Is DolARK effectively HARK 2.0?
In other words, rather than trying to patch HARK, would it be better to work on expanding the functionality of DolARK, as indicated by the chimeras?

[llorracc]

Not really. DolARK is a language for describing models. Econ-ARK is a place where models can be built and stored and shared. Your question is kind of like asking "Is the English language (=DolARK) basically the same as the Penguin Classics of English Literature (=Econ/ARK -- except that we are far from having many of the classics yet)"?

…

On Sat, Feb 1, 2020 at 1:05 AM Sebastian Benthall ***@***.***> wrote: Is DolARK effectively HARK 2.0? In other words, rather than trying to patch HARK, would it be better to work on expanding the functionality of DolARK, as indicated by the chimeras? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#481?email_source=notifications&email_token=AAKCK7Z64XE4WBRVSPFPZE3RAS4E5A5CNFSM4KOIJ7C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKQMTZI#issuecomment-580962789>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKCK75SBB6R3EEFWE5SNP3RAS4E5ANCNFSM4KOIJ7CQ> .

-- - Chris Carroll

[mnwhite]

To abuse this metaphor, our print quality is also on par with Penguin. On Sat, Feb 1, 2020, 3:42 PM Christopher Llorracc Carroll < [email protected]> wrote:

…

Not really. DolARK is a language for describing models. Econ-ARK is a place where models can be built and stored and shared. Your question is kind of like asking "Is the English language (=DolARK) basically the same as the Penguin Classics of English Literature (=Econ/ARK -- except that we are far from having many of the classics yet)"? On Sat, Feb 1, 2020 at 1:05 AM Sebastian Benthall < ***@***.***> wrote: > Is DolARK effectively HARK 2.0? > In other words, rather than trying to patch HARK, would it be better to > work on expanding the functionality of DolARK, as indicated by the chimeras? > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > < #481?email_source=notifications&email_token=AAKCK7Z64XE4WBRVSPFPZE3RAS4E5A5CNFSM4KOIJ7C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKQMTZI#issuecomment-580962789 >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AAKCK75SBB6R3EEFWE5SNP3RAS4E5ANCNFSM4KOIJ7CQ > > . > -- - Chris Carroll — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#481?email_source=notifications&email_token=ADKRAFNFXG2KUBFMCVKNSL3RAXNCRA5CNFSM4KOIJ7C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKRGFLI#issuecomment-581067437>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADKRAFPUMHPJR5WYF7SD4WLRAXNCRANCNFSM4KOIJ7CQ> .

[llorracc]

I think that's a little unfair to Penguin. Yes, the paper for the Penguin pages is very thin, but still readable. We have no paper at all! And whether our code is readable is still up for debate ... On Sat, Feb 1, 2020 at 10:04 PM Matthew N. White <[email protected]> wrote:

…

To abuse this metaphor, our print quality is also on par with Penguin. On Sat, Feb 1, 2020, 3:42 PM Christopher Llorracc Carroll < ***@***.***> wrote: > Not really. DolARK is a language for describing models. Econ-ARK is a > place where models can be built and stored and shared. > > Your question is kind of like asking "Is the English language (=DolARK) > basically the same as the Penguin Classics of English Literature (=Econ/ARK > -- except that we are far from having many of the classics yet)"? > > > > > > On Sat, Feb 1, 2020 at 1:05 AM Sebastian Benthall < > ***@***.***> > wrote: > > > Is DolARK effectively HARK 2.0? > > In other words, rather than trying to patch HARK, would it be better to > > work on expanding the functionality of DolARK, as indicated by the > chimeras? > > > > — > > You are receiving this because you commented. > > Reply to this email directly, view it on GitHub > > < > #481?email_source=notifications&email_token=AAKCK7Z64XE4WBRVSPFPZE3RAS4E5A5CNFSM4KOIJ7C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKQMTZI#issuecomment-580962789 > >, > > or unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/AAKCK75SBB6R3EEFWE5SNP3RAS4E5ANCNFSM4KOIJ7CQ > > > > . > > > > > -- > - Chris Carroll > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > < #481?email_source=notifications&email_token=ADKRAFNFXG2KUBFMCVKNSL3RAXNCRA5CNFSM4KOIJ7C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKRGFLI#issuecomment-581067437 >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/ADKRAFPUMHPJR5WYF7SD4WLRAXNCRANCNFSM4KOIJ7CQ > > . > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#481?email_source=notifications&email_token=AAKCK73AWHXKMJZISLTAUKLRAXPXBA5CNFSM4KOIJ7C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKRGSXQ#issuecomment-581069150>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKCK72Y5ILOWGRGQSI5XMDRAXPXBANCNFSM4KOIJ7CQ> .

-- - Chris Carroll

[mnwhite]

Yeah, it was supposed to be a knock on our/my code quality. On Sat, Feb 1, 2020 at 4:11 PM Christopher Llorracc Carroll < [email protected]> wrote:

…

I think that's a little unfair to Penguin. Yes, the paper for the Penguin pages is very thin, but still readable. We have no paper at all! And whether our code is readable is still up for debate ... On Sat, Feb 1, 2020 at 10:04 PM Matthew N. White ***@***.*** > wrote: > To abuse this metaphor, our print quality is also on par with Penguin. > > On Sat, Feb 1, 2020, 3:42 PM Christopher Llorracc Carroll < > ***@***.***> wrote: > > > Not really. DolARK is a language for describing models. Econ-ARK is a > > place where models can be built and stored and shared. > > > > Your question is kind of like asking "Is the English language (=DolARK) > > basically the same as the Penguin Classics of English Literature > (=Econ/ARK > > -- except that we are far from having many of the classics yet)"? > > > > > > > > > > > > On Sat, Feb 1, 2020 at 1:05 AM Sebastian Benthall < > > ***@***.***> > > wrote: > > > > > Is DolARK effectively HARK 2.0? > > > In other words, rather than trying to patch HARK, would it be better to > > > work on expanding the functionality of DolARK, as indicated by the > > chimeras? > > > > > > — > > > You are receiving this because you commented. > > > Reply to this email directly, view it on GitHub > > > < > > > #481?email_source=notifications&email_token=AAKCK7Z64XE4WBRVSPFPZE3RAS4E5A5CNFSM4KOIJ7C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKQMTZI#issuecomment-580962789 > > >, > > > or unsubscribe > > > < > > > https://github.com/notifications/unsubscribe-auth/AAKCK75SBB6R3EEFWE5SNP3RAS4E5ANCNFSM4KOIJ7CQ > > > > > > . > > > > > > > > > -- > > - Chris Carroll > > > > — > > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub > > < > #481?email_source=notifications&email_token=ADKRAFNFXG2KUBFMCVKNSL3RAXNCRA5CNFSM4KOIJ7C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKRGFLI#issuecomment-581067437 > >, > > or unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/ADKRAFPUMHPJR5WYF7SD4WLRAXNCRANCNFSM4KOIJ7CQ > > > > . > > > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > < #481?email_source=notifications&email_token=AAKCK73AWHXKMJZISLTAUKLRAXPXBA5CNFSM4KOIJ7C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKRGSXQ#issuecomment-581069150 >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AAKCK72Y5ILOWGRGQSI5XMDRAXPXBANCNFSM4KOIJ7CQ > > . > -- - Chris Carroll — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#481?email_source=notifications&email_token=ADKRAFLQUYORICYLK2RV5P3RAXQQ7A5CNFSM4KOIJ7C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKRGW4I#issuecomment-581069681>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADKRAFJIDIX4ANPXIQD3QVDRAXQQ7ANCNFSM4KOIJ7CQ> .

[sbenthall]

Is there a document anywhere where you lay out the anticipated design for HARK 2.0 with integrated DolARK? I believe I have a pretty sound and sympatico idea of what the different functionalities are that need to fit together to make software like this work well. Here and elsewhere I seem to be getting tripped up on choices that you've made about how to carve that functionality up into different repositories/packages.

If there is no such design document, I can try to write one based on these conversations and run it by you.

As the main substantive issues left for how to link DolARK and HARK in the combined "2.0 vision", I'm thinking:

• DolARK will have a parser that reads in model configuration files. But it will have to parse these into a (generic) model class. Will this model class be Dolo's Model class, be a new class in DolARK, or a new class in HARK?
• Generic solvers will operate on this generic model class. Will these generic algorithms be in DolARK or in HARK?

I'm talking about the ideal design we want to get to, I understand HARK's current limitations.

[sbenthall]

These are my notes on a 2.0 plan so far.

https://github.com/econ-ark/HARK/wiki/Towards-2.0

[sbenthall]

Closed with #772