docs: bootstrap API reference docs generation #119

aajtodd · 2021-05-03T14:03:18Z

Issue #, if available:
closes smithy-lang/smithy-kotlin#267

Description of changes:

I HIGHLY suggest pulling down this branch and looking at the generated output before providing feedback.

Bootstrap generation of the reference API docs using Dokka. There is still A LOT of improvements and surface area to explore with Dokka, including but not limited to:

improved code blocks / syntax highliting (Document status of syntax highlighting Kotlin/dokka#823)
Grouping runtime and services separately in nav bar (Suggestion: Group Methods or Classes Kotlin/dokka#84)
Including the version information somewhere
Ensuring that the KMP target information is displayed (e.g. JVM, Native, JS, etc)
Custom Markdown pages that give an introduction to the API, getting started, etc that have their own entry in the nav bar (I see no way to do this in the vanilla Dokka plugin but should be doable in our custom plugin...)
Getting smithy-kotlin docs merged in (Merge multiproject (composite build) docs Kotlin/dokka#1864)
Fixing our processing of the embedded model docs to spit out valid KDoc (currently we just pre-process the docs to strip tags, etc such that the generated documentation results in valid code. We do not however (e.g.) process HTML lists and convert markdown, etc).

NOTE: I did start updating the changelog but it's not finalized or anything.

I'd like to include some version of the API docs with the M1 release. I fully expect to continue iterating on this as we move forward. I would like our docs to be top notch.

You can include markdown pages with a specific module which for instance allows us to include documentation for specific services. I've shown an example of this for S3.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

ianbotsf

High-level feedback about the generated docs themselves:

In general, I like the overall look. The font choices are clean, the font weights and color contrast make for easy reading, and the whitespace feels balanced.
The left pane is always blank. I would expect to see a tree or a package/class list or something over there.
The All modules list on the top-level index page only gives a description for S3, not the other services I built or the common modules.
The copy link button in the All modules list copies an anchor link rather than a target link. Is that useful? It seems anchor links for section headers (e.g., Giving Feedback, All modules, etc.) would be more useful.
The package view seems broken for some services (e.g., aws.sdk.kotlin.services.lambda.model has some weird nesting behaviors going on like tags aren't being closed).
We can only view one of Types, Functions, or Properties at a time. It would be much nicer to see them all at the same time.
Many classes/members have no accompanying docstrings.
The docstrings for some items are poorly formatted. For example, s3/aws.sdk.kotlin.services.s3/S3Client/getObject has no linebreaks, doesn't vary font styles for file paths, and hyperlinks are missing closing tags.
Some referenced types are missing links (e.g., s3/aws.sdk.kotlin.services.s3.model/GetObjectResponse's Properties page is missing links for ByteStream and Instant).
Favicon only seems to be applied for the top-index page.

ianbotsf · 2021-05-03T16:57:05Z

CHANGELOG.md

+* S3
+    * NOTE: S3 is a complicated service, this initial release **DOES NOT** have complete support for all S3 features.


Suggestion: Can we note which features this release does not support? Or if it's easier, which features it does support?

I'm not sure we can articulate it well yet but we can try. Let's leave review of the CHANGELOG to another PR though. I started mucking with it here when I was testing some things. We'll have a "bump versions" PR before M1.

ianbotsf · 2021-05-03T17:01:24Z

README.md

+```
+./gradlew -Paws.services=lambda  :codegen:sdk:bootstrap
+```


Nit: ```sh

ianbotsf · 2021-05-03T17:10:45Z

dokka-aws/build.gradle.kts

+tasks.withType<org.jetbrains.kotlin.gradle.tasks.KotlinCompile> {
+    kotlinOptions.jvmTarget = "1.8"
+}


Question: Can the JVM target version be read from the parent project?

It could be made into a variable but it honestly doesn't matter, this is more dictated by what works with the DokkaPlugin than choices we make for our own code we publish.

Oh good point. I mistakenly thought these should be unified.

ianbotsf · 2021-05-03T17:14:33Z

dokka-aws/src/main/kotlin/aws/sdk/kotlin/dokka/transformers/FilterInternalApis.kt

+        val isInternal = when (d) {
+            is DClass -> d.isInternalSdk()
+            is DObject -> d.isInternalSdk()
+            is DTypeAlias -> d.isInternalSdk()
+            is DFunction -> d.isInternalSdk()
+            is DProperty -> d.isInternalSdk()
+            is DEnum -> d.isInternalSdk()
+            is DEnumEntry -> d.isInternalSdk()
+            is DTypeParameter -> d.isInternalSdk()
+            else -> false
+        }


Question: Why separate when branches for each subclass? Do we not want to check for the internal annotation on some subclasses?

Well the annotation itself only applies to certain types but also not every subclass of Documentable inherits the properties necessary to check if it has an annotation.

I noticed that the annotation itself is visible in docs. Is there a reason to keep it?

http://localhost:8000/client-runtime/aws-client-rt/aws-client-rt/aws.sdk.kotlin.runtime/-internal-sdk-api/index.html

Ahh good point we can suppress it. I don't think it matters either way I was just trying to get most of the things people shouldn't be using or care about out of the docs.

ianbotsf · 2021-05-03T17:16:17Z

services/s3/API.md

+
+```
+
+See [aws.sdk.kotlin.services.s3.model.GetObjectResponse]


Question: This link (and at least one other) doesn't go anywhere. Is it expected to yet?

It should link appropriately in the actual generated HTML

So it does!

aajtodd · 2021-05-03T17:59:05Z

High-level feedback about the generated docs themselves:

* In general, I like the overall look. The font choices are clean, the font weights and color contrast make for easy reading, and the whitespace feels balanced.

All defaults :)

* The left pane is always blank. I would expect to see a tree or a package/class list or something over there.

You might have to show me this, I'm not sure I follow. I see a navigation bar.

* The **All modules** list on the top-level index page only gives a description for S3, not the other services I built or the common modules.

Yeah this is because we have an API.md for that module which is hand-written. Not sure how to improve status quo for this at the moment.

* The copy link button in the **All modules** list copies an anchor link rather than a target link. Is that useful? It seems anchor links for section headers (e.g., **Giving Feedback**, **All modules**, etc.) would be more useful.

Not sure what you mean, this sounds like a bug in Dokka?

* The package view seems broken for some services (e.g., `aws.sdk.kotlin.services.lambda.model` has some weird nesting behaviors going on like tags aren't being closed).

* We can only view one of **Types**, **Functions**, or **Properties** at a time. It would be much nicer to see them all at the same time.

Dokka default, not sure if it's configurable out of the box or not (I did not see a knob for it but could be wrong)

* Many classes/members have no accompanying docstrings.

Runtime or generated? Generated you'd have to look at the model to verify we don't have a codegen bug. Runtime we just need to check we actually documented it.

* The docstrings for some items are poorly formatted. For example, `s3/aws.sdk.kotlin.services.s3/S3Client/getObject` has no linebreaks, doesn't vary font styles for file paths, and hyperlinks are missing closing tags.

Yup. Known issue. See smithy-lang/smithy-kotlin#136

* Some referenced types are missing links (e.g., `s3/aws.sdk.kotlin.services.s3.model/GetObjectResponse`'s **Properties** page is missing links for `ByteStream` and `Instant`).

Yup. Known issue, these are types from smithy-kotlin. It's unclear how to merge documentation across repos.

* Favicon only seems to be applied for the top-index page.

There is an open issue in Dokka github.

@ianbotsf

kggilmer

Excellent

kggilmer · 2021-05-03T23:00:07Z

README.md

+./gradlew --no-daemon --no-parallel dokkaHtmlMultiModule
+```
+
+This will output HTML formatted documentation to `build/dokka/htmlMultiModule`


suggestion

Mention python3 -m http.server here for people wanting to view local content. I looked for a Kotlin equivalent but there is no one liner I could find.

kggilmer · 2021-05-03T23:00:54Z

build.gradle.kts

+}
+
+dependencies {
+    dokkaPlugin(project(":dokka-aws"))


question

with the name is it your intent that this would be shared outside of the Kotlin SDK?

Nope just needed a name. I don't expect it to be published.

kggilmer · 2021-05-03T23:05:00Z

build.gradle.kts

+    tasks.withType<org.jetbrains.dokka.gradle.DokkaTaskPartial>().configureEach {
+        // each module can include their own top-level module documentation
+        // see https://kotlinlang.org/docs/kotlin-doc.html#module-and-package-documentation
+        if (project.file("API.md").exists()) {


question

Do you see this mainly being used to describe APIs? Perhaps something like docs.md or supplemental-info.md would be more descriptive?

I do, or rather only included in the generated API reference docs. This is because it has to follow a particular format for KDoc.

kggilmer · 2021-05-03T23:10:33Z

build.gradle.kts

+
+        val year = java.time.LocalDate.now().year
+        val pluginConfigMap = mapOf(
+            "org.jetbrains.dokka.base.DokkaBase" to """


comment

Maybe this JSON would be more maintainable as a file rather than embedded here? I see you resolve file paths though, maybe it wouldn't work with relative paths?

If there was a lot of JSON then maybe but this is literally the only 3 things you can configure with this JSON.

kggilmer · 2021-05-03T23:13:31Z

docs/GettingStarted.md


 2. Unzip the repository somewhere on your local machine

 ```sh
-> unzip aws-sdk-kotlin-0.1.0-M0.zip
+> unzip aws-sdk-kotlin-0.2.0-M1.zip


nice you've already written the M1 quick start guide!

kggilmer · 2021-05-03T23:14:26Z

dokka-aws/src/main/kotlin/aws/sdk/kotlin/dokka/AwsDokkaPlugin.kt

+ */
+class AwsDokkaPlugin : DokkaPlugin() {
+    init {
+        println("AwsDokkaPlugin loaded!")


question

do you still need this or was part of getting things working initially?

Both? So I was going to remove it but I had so many issues figuring out if it was actually being applied I think it's still useful (it gets printed out per/module when loaded).

kggilmer · 2021-05-03T23:15:40Z

services/s3/API.md

@@ -0,0 +1,63 @@
+# Module s3


comment

It's really cool that we can add additional content like this. I wonder if the doc team would be interested in collaboration here, in supplying some of this for us per-service?

Yeah potentially! This was mostly an example (feel free to critique) but I could see us adding additional docs for particular services, linking to the developer guide, etc.

aajtodd added 8 commits April 30, 2021 08:31

docs playground

d1d6e5f

fix styles and allow per/module includes

13fd3f9

feat(docs): register custom dokka plugin

4a73abd

add logo as local asset

1a9f0ad

cleanup

b935236

improve samples styling

a7976a7

update docs

7798a34

update readme

5b687ea

aajtodd requested review from kggilmer, ianbotsf and kiiadi May 3, 2021 14:03

ianbotsf approved these changes May 3, 2021

View reviewed changes

Merge remote-tracking branch 'origin/main' into api-docs-bootstrap

ec5aebe

kggilmer approved these changes May 3, 2021

View reviewed changes

aajtodd added 3 commits May 4, 2021 08:20

add snippet type annotation

ce6e3ce

suppress internal annotation from docs

cd9a8bb

update readme

1d1bdc2

aajtodd merged commit e14c101 into main May 4, 2021

aajtodd deleted the api-docs-bootstrap branch May 4, 2021 12:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: bootstrap API reference docs generation #119

docs: bootstrap API reference docs generation #119

aajtodd commented May 3, 2021 •

edited

Loading

ianbotsf left a comment

ianbotsf May 3, 2021

aajtodd May 3, 2021

ianbotsf May 3, 2021

ianbotsf May 3, 2021

aajtodd May 3, 2021

ianbotsf May 3, 2021

ianbotsf May 3, 2021

aajtodd May 3, 2021

kggilmer May 3, 2021

aajtodd May 4, 2021

ianbotsf May 3, 2021

aajtodd May 3, 2021

ianbotsf May 3, 2021

aajtodd commented May 3, 2021

kggilmer left a comment

kggilmer May 3, 2021

kggilmer May 3, 2021

aajtodd May 4, 2021

kggilmer May 3, 2021

aajtodd May 4, 2021

kggilmer May 3, 2021

aajtodd May 4, 2021

kggilmer May 3, 2021

kggilmer May 3, 2021

aajtodd May 4, 2021

kggilmer May 3, 2021

aajtodd May 4, 2021

		* S3
		* NOTE: S3 is a complicated service, this initial release DOES NOT have complete support for all S3 features.

docs: bootstrap API reference docs generation #119

docs: bootstrap API reference docs generation #119

Conversation

aajtodd commented May 3, 2021 • edited Loading

ianbotsf left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aajtodd commented May 3, 2021

kggilmer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aajtodd commented May 3, 2021 •

edited

Loading