-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
POD6 to RakuDoc revision process #375
Comments
I know you said hold back from making comments, but something that was always missing in Rakudo and probably roast, is not recognizing formatting codes in declarator blocks: #| Some B<subroutine>
sub foo {} I think it should be supported(along with I don't know the new RakuDoc supports this though. |
Just my 2c
Everyone is entitled to spend their energy and resources whichever way they want. There's a whole ecosystem to pursue any kind of project you want, and Raku fortunately has the support for it. Involving the community, however, is already a resource-consumption item by itself, not to mention if it was actually accepted and it needed implementation in the core. My position, then, is thanking you a lot for your passion and contributions, but firmly against any redefinition of Pod6 or inclusion of a RakuDoc along Pod6 in the core (which would be even worse)
|
Agree that this should be possible. But in order to be able to do that in any user-friendly and future-proof manner, is still a long way off. I gave up on my attempts to even make the RakuAST pod parsing a slang (fwiw, the legacy pod parsing is most definitely NOT a slang).
More technically: parsing for Pod6 runs the
Both have their pros and cons. The con of the second approach, is that some kind of code parsing would still be needed to handle configs. So I'm leaning towards the first option.
Agree that is a problem. But this effort is not about the content.
I know that @finanalyst and @thoughtstream are not primarily interested in helping with the documentation content problem. And to be honest, at the moment I'm not that interested in it either, other than making sure any additions to Raku have at least a bare documentation nugget (case in point:
This is not a redefinition of Pod6: it's a clarification / codification of many of Pod6's implicitly accepted features:
RakuDoc = Pod6++. There will be no side-by-side in 6.e when RakuAST completely lands in core. Legacy features, such as @JJ I hope this will alleviate some of your concerns. |
@JJ Some of the changes needed to move POD6 to RakuDoc were to meet long-standing issues. I started on this journey in response to an issue that you created, namely the ability to allow for templates rather than hard-coding HTML for each block type. That led to a close examination of the original design of POD6, which has not changed in RakuDoc. That led to new possibilities being recognised. Once we have RakuDoc locked down, I will be writing a new renderer somewhat like Raku::Pod::Render, which will process The difference is that Further, there are two use cases for POD6 and RakuAST, one interacts with code, which is useful for IDEs but leads to behaviour that Github does not like. The other use case is for standalone documentation, which is what is needed in README's. I think what @lizmat is suggesting is that the In a real sense, we are not modifying POD6 so much as updating it to take into account long-term issues that are preventing POD6 from being more widely used. @JJ whether you are happy with this or not, the RakuDoc revision has come about because of your energy and the work you put into the documentation suite. |
I do see several reasons why GitHub will probably never turn into a very good Raku module documentation viewer:
I guess for similar reasons GitHub doesn't support JavaDoc either. But now that our module index sites have evolved more, I see no reason why someone would primarily view Raku distributions on GitHub instead of visiting some module index website. So I see a different way forward that I believe would push adoption of RakuDoc more than GitHub support: Have raku.land (or some other module index site) render module documentation and have a working CLI module documentation viewer. For me personally raku.land already is the main entry point to modules. I only go from raku.land to GitHub if I'm specifically interested in the version history. This parallels how I use metacpan.org vs GitHub to view Perl module documentation.
I agree this is a problem that we should solve some day, but I don't think this is the most important one. (I do understand that this is blocking GitHub support.)
I suspect decalarator blocks will prove to be a pain point here as they reference a piece of code. I think the first option has more chance to keep declarator blocks working.
I really hope we can have a |
Yes, you totally right. thank you |
I don't see a reason why a RakuAST based renderer wouldn't be able to do that. I think all of the hooks are already there. |
Reason it could not be implemented before is that Rakudo concatenated
output input and code into pod-code-block so a renderer could not
distinguish between the three. I did try. Not much point in changing this
now.
RakuAST keeps the distinction, so a RakuAST renderer will do this
immediately.
…On Tue, 4 Jul 2023, 13:59 Elizabeth Mattijsen, ***@***.***> wrote:
I love the example in S26
<https://github.com/Raku/old-design-docs/blob/master/S26-documentation.pod#L384-L406>
and dream to one day have a renderer that supports this.
I don't see a reason why a RakuAST based renderer wouldn't be able to do
that. I think all of the hooks are already there.
—
Reply to this email directly, view it on GitHub
<#375 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACYZHFWK2JWDEPHKFO5QBLXOQHUDANCNFSM6AAAAAAZ3SZZ2I>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
@CIAvash rakudo/rakudo@5f488afb56 implements opt-in Markup support for declarator docs. See the commit description for more information. |
Is RakuDoc (a.k.a POD6) 'Literate Programming'? https://www.perl.com/pub/tchrist/litprog.html/ |
RakuDoc is not identical to literate programming. However, it was influenced by that concept (i.e. stole ideas and mechanisms from it) |
I'll comment in a 'food-for-thought' manner... . TL;DR RakuDoc/POD6 as a literate-programming Markdown replacement will have a ready supply of Data Science users. I believe the concept of literate programming stretches back many decades, with various implementations in the meanwhile. I cited a Knuth paper earlier. I recall studying a bit about All these concepts have been turned on their head with the advent of Data Science. The primary objective of literate programming today is to produce Data Science Reports, not code documentation. In the R-programming language, this has meant the original implementation in All this work took a gigantic leap forward with 1) the advent of Markdown, and 2) release of the Looking at the literature, and tools created, it seems that the focus now is NOT in creating a language that produces stand-alone static documentation, but either 1) generating reports, or 2) running code (with plaintext serving as inline documentation). See: "Using knitr and pandoc to create reproducible scientific reports". So maybe we're looking at this issue backwards. There IS dissatisfaction in the Data Science community with Markdown, and (truth-be-told), a number of Extended Markdown Syntaxes have been developed, including CommonMark, GitHub Flavored Markdown (GFM), Markdown Extra, MultiMarkdown, and R Markdown. My question to those reading this far: is RakuDoc/POD6 better than Markdown (and assorted flavors)? If so (to address JJ's point), I would say full steam ahead. Markdown suffers from being anything-but-WYSIWYG. RakuDoc/POD6 doesn't seem to have that problem. However, I've never been able to insert "code-fences" into RakuDoc/POD6 and get code output. Here All this...seems like a lot of work. But maybe less than anticipated, because R-programmers have moved on to the Rmarkdown package, which uses a Pandoc backend. So why can't RakuDoc/POD6 use a Pandoc backend as well? For those who don't like to follow links, Pandoc (written in Haskell), inter-converts approximately 50 document formats. |
For what it's worth, I don't think Markdown has ever replaced (or will replace, for that matter) javadoc, doxygen kind of means of documentation. That's the kind of thing we failed to properly utilize with Raku POD so far. Also, I think the sole fact that there is a need for so many "flavors" shows that Markdown is not a great fit. I suppose part of the reason for that is that these communities (also) didn't want to inter-depend and thought they knew better what they needed for themselves. In the light of this, I think it's apparent that designing something with Raku in mind, both syntactically and semantically, yields a better tool, and the question should be posed the other way around: what's so great about creating yet another semi-compatible Markdown dialect that one should give up on all POD stuff? |
I am a contrarian on POD6 and have refrained from chipping in on this thread since my plan has been to side step POD6 anyhow and didn't want to rain on the parade. But, then again, maybe this is a chance to get what I want. My beef is that, coming from a Data Science and wider language perspective (Python in particular)...
[Now I would slightly prefer doc embedded in code as opposed to code embedded in doc ... but I also see the value in using vanilla Jupyter for the vast number of possible converts to Raku] So, my ideal is a tool that lets me adorn my Raku source with MD and then has bidirectional compatibility with .ipynb format. I would like my [Comma]IDE to be able to selectively hide doc &/ code please. If this can be achieved with pandoc ... and my superficial reading suggests that pandoc may be a neat way to/from .ipynb, then count me in! Edit: I am not advocating MD as a replacement for POD6, but I am hoping for a simple POD6 tag set that makes it possible to use MD in a .raku script (so legal, but not preferred) in a way that can round trip to an .ipynb ... and that's just a softish request since I guess I can continue to sidestep POD6 altogether. |
That's why I would be curious if @antononcube has something to say about this, as somebody who has worked towards literate programming, notebook features and conversion between POD and Markdown. On the other hand, I personally don't like how much this topic "gets owned" by the data science/literate programming discourse. If we think of either the old POD from Perl 5, or the name RakuDoc itself, it shines through that one of the main use cases (if not the main use case) would be the "case 1" of finanalyst's description, @Leont's 2022 conference presentation also mostly discusses the eventual RakuDoc from this perspective. And indeed, one can see that we are not doing a lot with it; we barely even generate a README.md from it (I think that's about all App::Mi6 does in this regard). Compare that to even good old doxygen, for example. And then, unsurprisingly, people barely use it, even the most proficient module authors. I really don't think markdown is an answer to that problem in any sense, and at the same time, I think it's more just a beneficial side effect if RakuDoc turns out to be a powerful tool for code generation or notebook description... |
POD6 was never fully implemented as originally intended. RakuDoc will be
implemented fairly quickly and will be customisable from the start.
Also, the tools needed to generate MD and epubs as well as HTML should
follow more easily too.
The customisation allows for Graphviz and Latex equations (Collection
plugins already exist for these). This will allow for far more flexible
documentation needs, for example showing the equation the code is
implementing.
…On Sun, 23 Jul 2023, 12:57 Márton Polgár, ***@***.***> wrote:
That's why I would be curious if @antononcube
<https://github.com/antononcube> has something to say about this, as
somebody who has worked towards literate programming, notebook features and
conversion between POD and Markdown.
On the other hand, I personally don't like how much this topic "gets
owned" by the data science/literate programming discourse. If we think of
either the old POD from Perl 5, or the name RakuDoc itself, it shines
through that one of the main use cases (if not *the* main use case) would
be the "case 1" of finanalyst's description
<https://new-raku.finanalyst.org/language/rakudoc>, @Leont
<https://github.com/Leont>'s 2022 conference presentation
<https://youtu.be/2mkIZesnDl8?t=1110> also mostly discusses the eventual
RakuDoc from this perspective. And indeed, one can see that we are not
doing a lot with it; we barely even generate a README.md from it (I think
that's about all App::Mi6 does in this regard). Compare that to even good
old doxygen, for example. And then, unsurprisingly, people barely use it,
even the most proficient module authors. I really don't think markdown is
an answer to that problem in any sense, and at the same time, I think it's
more just a beneficial side effect if RakuDoc turns out to be a powerful
tool for code generation or notebook description...
—
Reply to this email directly, view it on GitHub
<#375 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACYZHGQ67MEKTJZ36UKZMLXRUGRRANCNFSM6AAAAAAZ3SZZ2I>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
I'm not advocating for a change to the POD6/RakuDoc specification, except adding the option to create reports with inline code output such as tables and graphs. @2colours there isn't a reason to add another Markdown flavor. If POD6/RakuDoc is a better specification it will win out. But you raise a fair point: can POD6/RakuDoc replace both Doxygen and Markdown? Right now R-programmers write their documentation with roxygen2, and their Data Science reports with knitr or rmarkdown. If POD6/RakuDoc is more powerful than Doxygen I don't see why POD6/RakuDoc can't replace both Doxygen and Markdown for Raku module authors. @librasteve I've used Jupyter notebooks, but never liked the closure model. It seemed you could be at the bottom of a document working with variable @finanalyst I've used Graphviz with R (external to the @thoughtstream I'm sorry I couldn't get your links to work. But maybe my comments are too far afield? It seems reading the Pandoc repo that we would need some Haskell and/or Lua expertise to get Pandoc conversion from POD6/RakuDoc to other documentation formats. Thanks everyone for the lively discussion! |
@jubilatious1 @librasteve @2colours @thoughtstream @lizmat RakuDoc update.As explained in the first post in this thread, RakuDoc is currently being revised. @jubilatious1 the links by @thoughtstream were to a private repo, an on-line version can be found at new-raku. This is still not the final version, although close. Most of the revision has been completed, and we are working our way through an issue related to markup in code blocks, and specifically whether RakuDoc instructions should be handled by the Raku parser, or the RakuDoc renderer. Although most of RakuDoc can be rendered by The RakuDoc document includes some significant explanations and examples that are missing from the POD6 document. As mentioned in the first post, the design and wording of RakuDoc will be open to community discussion from August 1. I think we are on schedule for that date. Community discussion is slated to last until November. @jubilatious1 When I mentioned GraphViz, Equations, and Leaflet maps, I meant they can be easily exposed using Custom blocks in RakuDoc.
@librasteve I understand that you may not like POD6, aka RakuDoc. I also sympathise completely with the pain of a learning curve for a new language. My pain is significantly higher now because I have to implement the additions to RakuDoc (some of which were in POD6, but not realised in a renderer). Unlike you, I do not like MarkDown. It is too simple and constrained for content rich documents, eg. the Raku Documentation suite. I wanted the custom blocks I cite above for my own website. I put some websites together using award winning CMS systems, and I had to write the plugins for Maps for one of them. I found that their reliance on MarkDown as the underlying text component was inflexible. I also found that writing custom plugins was convoluted too. MarkDown does not have any extensibility specified, which is why there are so many flavours of it. @jubilatious1 RakuDoc was not written for Literate programming, but it borrows from its concepts. From what I have read, RakuDoc could be used to accomplish the goals set out for literate programming. A challenge for the future will be to take some of the examples in the literate programming specification and show how to accomplish them in RakuDoc - perhaps by using some custom blocks. As for a pandoc translator, that too sounds like an interest project for the future. |
Closing in lieu of #380 |
@codesections some commentary would be helpful, in light of: https://www.codesections.com/blog/weaving-raku/ Can "Technical Reports" be generated with Raku, Markdown, POD6, RakuDoc? Thx. |
Yes, although I am not sure what "technical report" generation means.
Large fractions of the functionalities above are demonstrated and discussed in the TRC-2023 talk "Integrating Large Language Models with Raku". |
@antononcube I'm looking for the Raku/POD6/RakuDoc equivalent of the following ( https://learning.nceas.ucsb.edu/2023-04-coreR/session_04.html |
@antononcube RMarkdown example with output: https://learning.nceas.ucsb.edu/2023-04-coreR/session_04.html |
This is the primary use case of for implementing "Text::CodeProcessing".
As for Pod6 notebook (interactive) solutions:
As for notebook solutions in general:
Here is a screenshot of a Jupyter notebook using a Raku kernel: (It is a fragment of the LLM-workflows notebook I mentioned above.) |
Please see the Pod6 text and code chunks tests here of the package "Text::CodeProcessing". |
@antononcube @jubilatious1 This thread was closed when the gamma revision repo was set up. |
With the move to RakuAST, it seemed a good time to review the POD6 specification/documentation. There is more detail on this in a blog post I wrote on DEV.
There is also a minute by the Raku Steering Council about the process for the revision The aim is to find a way to get a timely change to a standard, whilst making the process transparent and open to the community.
We have just started the BETA period, which will last to 1 August. Damian Conway, who wrote most of the design of POD6, is taking an active part in this process.
Around the beginning of August, an issue will be opened in Raku/docs with the new RakuDoc document, and a link here. Hopefully a more stable version of the document will be available for the Raku community to comment on by that time. The GAMMA period will last to 1 November to give enough time for everyone to comment and for us to reach a consensus about the new specification.
The current state of the document is still a bit fluid because the structure has been changed to make parts of it clearer, to document things that existed before in POD6 but were unknown, RakuAST offers more that just
$=pod
and there are typos.....But some things are beginning to clarify. For (a somewhat trivial) example, the markup language will be called
RakuDoc
(note the camel case) and the file extension will be.rakudoc
(which it has been for some time)For those who are interested, the latest iteration in HTML can be seen at new-raku rakudoc
Please hold back from making comments / suggestions yet. Everyone will have a chance to comment in August. The online version is DEFINITELY going to change in some way, so your red flag might disappear.
The document is quite long, and hopefully explains why RakuDoc offers more possibilities than just documenting code.
The text was updated successfully, but these errors were encountered: