F1 Spec to Markdown / tex->png utility

[rigelrozanski]

we ought to stay consistent with our specs and maintain them in a common format - thus the (great) F1 spec should be converted into markdown.

As a part of this process, the spec should be broken out into separate markdown files as described in docs/spec/SPEC-SPEC.md - I imagine due to the scope and layout of F1 we may choose to adapt the SPEC-SPEC.md (as well as the other specs) to be more in line with some of the features of the current F1 spec if we find there are superior sectionings during this conversion.

One of the nice things about the F1 spec is the use of .tex formulas I'd recommend that we include all these formulas in the markdown document in the form of png images - however we should also include all the formulas in the original .tex format in the spec as well and simply build a .tex -> .png conversion script which could exist in the spec document - this way one would not need to have LaTex installed to update the equations.

This script could be whipped up as a golang program using ImageMagick. See:
https://tex.stackexchange.com/questions/35690/how-to-export-a-equation-as-a-image-without-background
https://github.com/gographics/imagick

CC @ValarDragon @cwgoes

[ValarDragon]

I personally prefer this to remain a pdf, with implementation notes / details maintained separately in a markdown format alongside the code. I do like reading pdfs though ;)

It allows the spec for what we implement to be specific, and whats in the paper to be more general. We need to specify and justify the actual inflation algorithms, commission bounds, what features are enabled which aren't, separately from what the algorithm can do. Another example is explaining that the F1 specs validator iteration in the implementation does not "come for free" since it can't re-use the same staking validator / delegation structs. (Due to the SDK's lack of support for cross-module leaf-combining / column hashing)

[jackzampolin]

I prefer leaving it in the current format as well.

[alexanderbez]

The main thing is that we are consistent here...

[rigelrozanski]

Yeah I think we need to stay consistent - I'm not totally opposed to converting all the specs to LaTex, but I do think in the long run it will be:

• a maintainability burden due to installation requirements
• a burden to integrating our specs into our docs website.
  This which is why we're chosen to use markdown for literally everything else.

With regards to having the .pdf I actually agree with @ValarDragon that it's really nice, however I think having LaTex as the source is burdensome. I'm quite sure we could easily auto-generate pdfs from the markdown as a part of the build process (and have it look quite neat and nice as F1 currently looks). This pdf conversion could combine all the markdown documents in a linear order into a single pdf per spec folder (as is suggested in #3300)

[rigelrozanski]

./ValarDragon [Yesterday at 11:56 PM]
I don't agree with the conversion to markdown. I think that will cause it to be a single spec of what we are doing, as opposed to an abstract algorithm and our instantiation of it

14 replies
fp4k [3 hours ago]
Well first off… it could literally be identical to what it is right now but written in MD and be more maintainable. Second, it would be possible to have two layers of spec, one being more a bit more abstract (current) as well as an SDK implementation spec, which is actually what we should require and have in the spec directory. This implementation spec would include all the elements necessary to re-write a compatible module for an SDK in another language… aka we want all the standard stuff such as state.md which is missing here. (edited)

fp4k [2 hours ago]
(just posted in 3299 we should continue discussion there)

Also sent to the channel
./ValarDragon [38 minutes ago]
If this were more complex of a scheme, I definitely think it should be a paper form. (That is the standard)

From my perspective, the actual F1 scheme is really simple, so maybe it isn't really an academic contribution to the space. The fact that all delegate/withdraw/slash operations can be done with no iteration isn't immediate, but is also not that complex.

I think making it a published paper would be nice if we think its a meaningful contribution to the space of fee distribution mechanisms.

fp4k [24 minutes ago]
Oh yeah publishing is definitely on the table here - the paper would need to undergo several edit rounds internally first, not a priority ATM but in 2019 I could see something being put together! - But paper or not, I still sustain that we should organize our documentation in to md and simultaneously have pdf exports. It’s important to make the spec update process as open as possible - which means removing as many barriers to entry, I reckon we tex is one on a few levels. (edited)

cwgoes [12 minutes ago]
Maybe, but if and only if Markdown can retain the clarity (and e.g. mathematical notation) possible in LaTeX. Do you have a tool in mind? I've used Pandoc before successfully (not sure about complex notation though).

fp4k [11 minutes ago]
As I explained in the issue, we script export latex equations to png to use within markdown, alternatively, we can also use mathdown which effectively combines these two

[cwgoes]

What about Pandoc?

That's capable of converting Markdown to LaTeX, and the output looks decent (I've used it before).

I agree about separate algorithmic / implementation specs. I think the former are more important.

[rigelrozanski]

My main concerns/evaluations are as follows:

• low barrier to entry for the team and community to contribute to the specs
  • requiring downloading LaTex (3.5GB) or requiring use of proprietary web-tools in inhibiting
  • easiest is export pngs from tex in a script as explained earlier in this issue
  • maybe could use mathdown
• good looking documentation for printing
  • we can autogen nice compiled pdfs from any of the options this shouldn't be an issue
• good looking documentation on website / ease of website integration.
  • markdown is going to to be the easiest I reckon

[jackzampolin]

Didn't we come up with something good here for the ICS repo @cwgoes ?

[cwgoes]

The ICS repo uses Markdown and occasional LaTeX for diagrams.

Future papers will be written in Markdown, then converted to PDFs with pandoc.

[fedekunze]

closing this issue as outdated