x/gov: Change proposal state storage, so votes do not read static data

[ValarDragon]

Summary

A serious problem for cosmwasm upload governance proposals, is that the x/gov module has every vote, read the entire governance proposal. Votes should not require reading static data from a proposal, e.g. the Content. The current design makes votes for wasm upload proposals require 1million+ gas.

This applies in SDK v0.46 with the message based governance module as well. This is because proposals are written in state with the entire proposal content at one state entry.

Thank you @ethanfrey for pointing this out

Problem Definition

Fix gas consumption for custom governance proposals. This applies outside of just cosmwasm proposals, and even applies in the 46 framing of Messages in the proposal.

I'd classify the current proto structure of having the content in the same key in state as a bug.

Proposal

Change the proto structure of governance proposals to separate out static proposal content into another struct, and that is written in state at a different key than the core mutable data. (Both still indexed by governance proposal ID)

Then in votes, we do not read the static content.
I'd even argue that we should consider having three state entries per proposal:

• Proposal Static Metadata
• Proposal Content for execution
• Proposal vote data

And adjust the static metadata to not have the content in the same entry, as the content can have extremely high length

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned

[ethanfrey]

I agree with this strongly. It has been an issue with both Osmosis and Statgaze. Especially as some clients fix a cost for voting, less than 1 million obviously.

I believe this same issue hits proposals with very long text descriptions, albeit an order of magnitude less, still much unnecessary reading and writing and gas burn

[tac0turtle]

This sounds good to me. This would also coincide with the idea of using the file system to store votes instead of the tree.

[alexanderbez]

This would also coincide with the idea of using the file system to store votes instead of the tree.

I think we'd need to use some sort of Merkle tree, like a Merkle BTree (see osmosis-labs#294), because we need this data to still be state sync-able, but we don't need proofs for them. Opposed to flat files, which cannot be state synced AFAIK.

[tac0turtle]

Flat files can be state syncable. The same way CosmWasm contract code is state syncable.

[ethanfrey]

This can also be a two step process. First to pull votes into separate (iavl) struct, which solve the immediate problem.

Second, later, to pull them out of iavl tree completely if that is needed for performance (I would have good benchmarks of real data to show this is a slowdown and the solution would make a benefit)

[alexanderbez]

Flat files can be state syncable. The same way CosmWasm contract code is state syncable.

ohh, well this isn't what @ValarDragon told me :P