Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add KSY syntax diagram generated by Docson #37

Merged
merged 4 commits into from
Jul 1, 2020

Conversation

generalmimon
Copy link
Member

@generalmimon generalmimon commented Jul 1, 2020

Resolves kaitai-io/kaitai_struct#655 (comment)

Adding the KSY language syntax diagram generated by Docson, as discussed on Gitter. If you don't want to run it locally, here's how it looks (note that this online demo is slow, it might take up to 10-15 seconds to render). I've only integrated it into our documentation at https://doc.kaitai.io/, it is based on a normal .adoc page, so it has the same header, footer and look as the other pages.

Moreover, I added a simple loading spinner, which is shown along with a descriptive message and disappears when the diagram is fully rendered. But according to the local testing, the loading time isn't too long as in case of the online demo linked above, it's 1-3 seconds at most, but still can probably improve the user experience a bit.

Note that if you want to test it locally, it won't work just by opening the local file in the browser, the page has to be served through a HTTP server. If you have Python installed, it is a matter of one command (python3 -m http.server or python -m SimpleHTTPServer in Python 2), see Running a simple local HTTP server at MDN. I used php -S localhost:3000 for example, it works just as well.

Copy link
Contributor

@KOLANICH KOLANICH left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👎 for Node.js and npm
👎 :-100500: for relying on JS on client.

@generalmimon
Copy link
Member Author

👎 for Node.js and npm

Where do you see it? I don't use it at all here, neither Node.js nor npm ever runs here. I just copy the generated files from this directory, these are everything that's needed for it to work. All JavaScript runs on the client-side.

@KOLANICH
Copy link
Contributor

KOLANICH commented Jul 1, 2020

All JavaScript runs on the client-side.

Not using npm and node is good, at least the precompiled code fetched can (in theory, not in practice!) be checked for backdoors.

@generalmimon
Copy link
Member Author

generalmimon commented Jul 1, 2020

👎 :-100500: for relying on JS on client.

@KOLANICH Well, if you find a better JSON Schema documentation generator suitable for our use which doesn't use Node.js, npm and generates everything on the server-side, here with that. Or if you are willing to write such a tool by yourself, even better.

I spent a few hours looking for a good documentation generator from JSON Schema, and Docson, regardless of its unfortunate approach, is still the best that I could find. I tried at least 10 of them, and trust me, the fact that they mostly use npm is the least of the problem. I describe the situation here.

Running JS on client side is clearly inacceptable. It's better to serve pregenerated pure HTML+CSS.

Again, if you know something better, please share it. I fully agree with you that it would be of course better to generate everything on the server side and send just the static files to the browser. But back to the real world: AFAIK such tool doesn't exist, or at least isn't public. (I mean a tool that would work for our JSON Schema, and wouldn't have too ugly output.)

@generalmimon
Copy link
Member Author

generalmimon commented Jul 1, 2020

The best I know is https://github.com/coveooss/json-schema-for-humans , but it needs some improvements (coveooss/json-schema-for-humans#4)

Do you consider it best just because it uses your holy Python? I have tried it too of course, and I recall that the JSON Schema we use has been rejected by this tool, probably due to the recursive references. And even if I temporarily removed them to see how it looks, I'm still more satisfied with the Docson appearance, it's far nicer and more clear.

@generalmimon
Copy link
Member Author

@KOLANICH So if your only concern is the ideological problem with the architecture, I think it's OK and we can merge this PR with a clear conscience. Let's just wait what will @GreyCat say.

@KOLANICH
Copy link
Contributor

KOLANICH commented Jul 1, 2020

Hmm, I wonder if it can be possible to use stuff like https://github.com/jsdom/jsdom or maybe a full-featured browser to emulate the browser in node.js and get the resulting HTML (can work only if there is no interactivity, but may be hackable with CSS).

@KOLANICH
Copy link
Contributor

KOLANICH commented Jul 1, 2020

Do you consider it best just because it uses your holy Python?

Yes, the goals are

  • no JS on client, all should be rendered into static documents without any active content
  • to stay away of Node.js and its ecosystem, and also away of kther backdoors-plagued ecosystems

So precompiled tools written in C++ are also OK, but most of the tools for this purpose are written in scripting languages, and among them the most are written in JS.

we use has been rejected by this tool, probably due to the recursive references.

Do you mean coveooss/json-schema-for-humans#8 ?

@generalmimon
Copy link
Member Author

generalmimon commented Jul 1, 2020

Hmm, I wonder if it can be possible to use stuff like https://github.com/jsdom/jsdom or maybe a full-featured browser to emulate the browser in node.js and get the resulting HTML

Sure, feel free to experiment with that and I will be glad to accept the better server-side approach instead of the client-side one. I just don't think it's worth the effort, when you consider that it will take you 10-20 hours to make it work, and if all works well, the end-user won't notice any change, at best it'll be just one second faster, and that's it.

I mean: your concern about the ideological dislike is absolutely fine, and I'd probably raised the same thing if someone else did the same PR, but it's just doesn't have the highest priority for me. The most important thing for me is the user experience, and I'm not going to accept a worse solution only because it uses a better language or environment. Remember: the end user always decides, and has the supreme power by deciding whether or not to use the product. If you have an ugly documentation (which has a very nice internal code though, runs on the most secure and state-of-the-art environments, and it's able to fully work completely without JavaScript and CSS), the odds of getting many users aren't very high.

Such a shame that the users don't care about the beauty of the internal code, the security of the environment that runs the server code (to some extent of course, but I think we can agree that it's not the first factor for the normal end users) and don't see hundreds of hours that were necessary to put everything together, but that's simply the reality.

@KOLANICH
Copy link
Contributor

KOLANICH commented Jul 1, 2020

the end-user won't notice any change, maybe it'll be just one second faster.

JS-based stuff intended to be run client-side doesn't work at all, if NoScript is properly set up and configured or if javascript.enabled is set to false.

@generalmimon
Copy link
Member Author

generalmimon commented Jul 1, 2020

JS-based stuff intended to be run client-side doesn't work at all, if NoScript is properly set up and configured or if javascript.enabled is set to false.

Yes, that's right. But I think that the amount of users worldwide today that have JavaScript disabled is somewhere close to 0.1% (This article states 0.2% global, but the measurements are 4 years old, so let's assume some Netscape users passed away since then (although even Netscape had supposedly some JavaScript support since version 2.x., so just substitute other prehistoric browser without JS.)) Given that the https://github.com/kaitai-io/kaitai_struct repo has ~2000 stars, I'll assume that the number of our users is similar. This would mean that in average 2 users come to our site with JavaScript disabled. Are you willing to spend 20-30 hours to modify Docson for this single documentation page to work without JavaScript on the client side for these 2 users? Or would you rather work on implementing serialization which will be appreciated by the vast majority of Kaitai Struct users?

Another thing that is important to understand is that there is a difference between a website

  1. fully relying on client-side JS (which means that no meaningful content is accessible without it),
  2. where you can seamlessly access the content, but some features don't work,
  3. where all features work without JavaScript.

As the third option is uneconomical and often next to impossible to achieve (imagine trying to apply it to the Web IDE - just insane), it's natural even for large-budget products to use the second option. But even the 1st option is fine for low budget projects, as they clearly don't have the money and the reason why debug the application for such a marginal percentage of users having the JS (often intentionally) disabled.

And as there would be only one page which doesn't work without JS, our documentation would still fall into the 2nd category.

I'm sure that you've heard this quote:

There are no solutions. There are only trade-offs.

I think it fits here very well. In the commercial sphere, you can spend time only on those things for which your client will pay. In an open-source project, one would say that a contributor can in theory spend an unlimited number of hours on not-much-needed features, and he surely can, but eventually a contributor that spent significantly less hours on implementing a widely-wanted feature that will help many more users will help the project a lot more.

Copy link
Member

@GreyCat GreyCat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good to me! Thanks for contributing this big chunk of work!

@generalmimon
Copy link
Member Author

@GreyCat Thanks for the review! I'm merging it then.

@generalmimon generalmimon merged commit 2af7a5e into master Jul 1, 2020
@generalmimon generalmimon deleted the docson-ksy-diagram branch July 1, 2020 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

KSY schema: generate human-readable documentation
3 participants