Fix DGP/KMP integration, so Dokka can 'see' code from shared source sets in target source sets #3814

adam-enko · 2024-09-19T11:14:36Z

There are 3 main fixes in this PR:

Update determining the Kotlin target for each source set. (See KotlinAdapter#determineKotlinPlatform). Filter out metadata targets (they're not relevant for target determination), and only fall back to KotlinPlatform.Common if necessary.
Correctly resolve Konan dependencies: Use compilation.konanTarget.name instead of compilation.target.name.
Fix fetching the classpath: For Android projects, fetch both jar and android-classes-jar dependencies. For KGP projects, use compileDependencyFiles.

The remaining changes are to support these fixes:

Add functional test to verify that Dokka can correctly generate docs when 'leaf' source sets (e.g. linuxX64) reference 'branch' source sets (e.g. commonMain, nativeMain).
Update DGPv2 AGP integration test data.
Create a factory for creating DokkaSourceSetSpecs, which means DGP can consistently use convention() to set default values.
Add/update KDoc.

Should fix KT-70857

…ndents

…ndents # Conflicts: # dokka-runners/dokka-gradle-plugin/src/main/kotlin/internal/uriUtils.kt

- Filter out legacy KMP metadata compilations (they were retained to support Dokka v1). - Fetch both `jar` and `android-classes-jar` from AGP (probably not necessary, but why not?) - Only use `compilation.compileDependencyFiles` in non-AGP projects. - Add more docs

…ntion set in KotlinAdapter

- implement new JUnit extension to help set up and re-run DGP integration tests with different versions

…ses, tidy Android projects, fix suppress/analysisPlatform defaults.

`fileTree.kt` contains a utility for logging pretty file trees, which is useful in providing human-readable context for test failures. This PR updates the utility to only use NIO Path, rather than the older `java.io.File`. This change was split off from #3814 to make the PR smaller

…ndents # Conflicts: # .gitattributes # build.gradle.kts # dokka-integration-tests/gradle/build.gradle.kts # dokka-runners/dokka-gradle-plugin/src/main/kotlin/tasks/DokkaGenerateTask.kt # dokka-runners/dokka-gradle-plugin/src/testFixtures/kotlin/GradleTestKitUtils.kt # dokka-runners/dokka-gradle-plugin/src/testFixtures/kotlin/fileTree.kt

- Filter out metadata compilations when determining the source set target. - Remove KotlinPlatform warning - it's always triggered for commonMain. - Fetch _all_ dependent source sets, including transitives.

whyoleg

Nice!
Changes to defaults are looks logical for me, only the test placement worries me :)

dokka-runners/dokka-gradle-plugin/src/main/kotlin/adapters/KotlinAdapter.kt

dokka-runners/dokka-gradle-plugin/src/testFixtures/kotlin/kotestFiles.kt

whyoleg · 2024-11-22T09:19:44Z

...ers/dokka-gradle-plugin/src/testFunctional/kotlin/KmpCommonSourceSharedWithDependentsTest.kt

+import kotlin.io.path.walk
+
+@Ignored("KMP: References is not linked if they are in shared code and there is an intermediate level between them https://github.com/Kotlin/dokka/issues/3382")
+class KmpCommonSourceSharedWithDependentsTest : FunSpec({


I believe this test should be placed into dokka-integration-tests so that we will be able to check it with different Kotlin versions.
In my understanding, functional tests in DGPv2 are more about plugin behavior while this test is more about overall Dokka behavior and so should be in IT

I would probably except here a test which checks that we set correct defaults for DokkaSourceSetSpec platform, dependsOn and etc

I believe this test should be placed into dokka-integration-tests

Good idea, done.

Could you check the expected data in dokka-integration-tests/gradle/projects/it-kotlin-multiplatform/expectedData/html? There are errors, but I'm not sure if any are DGP's responsibility, or are caused by #3382

Hmm, it doesn't look like TeamCity likes the KMP IT. Some of the tests are failing.

https://ge.jetbrains.com/s/ndo3qeb66whvy/tests/overview?outcome=FAILED

I've checked expectedData/html and it looks reasonable and the error is most likely caused by #3382.
let's ignore test and re-check it in scope of #3930 (DGPv2 + K2)

Hmmmmm it looks like locally I get Error type: Unresolved type for NativeMainCls, but not on TeamCity?? Why??

https://ge.jetbrains.com/s/2cdhjngaftnhk/tests/task/:dokka-integration-tests:gradle:test/details/org.jetbrains.dokka.it.gradle.KotlinMultiplatformIT/generate%20dokka%20HTML(DokkaGradleProjectRunner)%5B1%5D?focused-exception-line=2-100&top-execution=4

The difference in OS (linux vs macOS) and so probably the difference is in dependencies passed inside KGP. E.g iOS targets are not compilable on linux and so probably the classpath (or some other configuration) is somehow different, which causes this behavior.

In this case, I think that for this test probably having just linux targets should be enough to reproduce the issue, correct?
We could also try to add mingw target (or androidNative targets), or add additional intermediate sourceSets somewhere, so that all targets will be cross-compiled on all OSs.

I could understand it if locally, on my Mac, I would get no errors and on Linux the macOS targets would cause Unresolved type errors, but I don't see why it's the other way around. I'd like to understand the problem more before trying to work around it, but I'm not sure how.

I'll check with my team.

I think that it's better to compare configurations on CI (linux) vs on macOS and then think about what could go wrong.

Also, I would propose in any case add second targets for iOS and linux, as Dokka doesn't work well with single target source sets - may be it could be also cause difference in produced configuration in this case.
E.g, we have an issue with K2: #3386, may be something similar also exists in K1.
Again, the first step will be to obtain configuration used to run Dokka and compare them

From looking at the Dokka configuration, it looks correct (both locally and on TeamCity), so I'm going to chalk the failures up to issues with K1 analysis and disable the HTML output validation tests. They're not critical for this PR anyway.

…src-to-dependents

…to integration test, to support testing different KGP versions

…ndents

…ingle-target source sets

Fix common source sets not propogated

ecc43a5

adam-enko added the runner: Gradle plugin An issue/PR related to Dokka's Gradle plugin label Sep 19, 2024

adam-enko self-assigned this Sep 19, 2024

adam-enko added 19 commits September 23, 2024 09:01

Add some kdoc

fc6319b

refine test

1482ecf

Fix fetching platform sources

1522d04

Merge branch 'master' into adam/feat/KT-70336/pass-common-src-to-depe…

e45e4d9

…ndents

Merge branch 'master' into adam/feat/KT-70336/pass-common-src-to-depe…

c41a249

…ndents

wip

a639d51

Merge branch 'master' into adam/feat/KT-70336/pass-common-src-to-depe…

148f821

…ndents # Conflicts: # dokka-runners/dokka-gradle-plugin/src/main/kotlin/internal/uriUtils.kt

Sort DokkaSourceSets (helps with debugging)

ad6a532

Fix convention for DokkaSourceSetSpec.suppress

169a859

Fix convention for DokkaSourceSetSpec.suppress

0c8605d

Add 'Dokka' to warn message

586d603

remove analysis platform convention, otherwise it overrides the conve…

f80f5b4

…ntion set in KotlinAdapter

Deprecate unused KotlinPlatform.DEFAULT

ead23da

IT Android with DGPv2

a3919ed

- implement new JUnit extension to help set up and re-run DGP integration tests with different versions

test Android Compose project

22cd3d5

Tidy DokkaGradlePluginTestExtension, split up file into separate clas…

e895ca1

…ses, tidy Android projects, fix suppress/analysisPlatform defaults.

Tidy GradleTestProjectInitializer

651cdf8

run tests with Java 17

91ee7d0

whyoleg mentioned this pull request Oct 31, 2024

[Dokka 2] References from a module description to module's classes are no longer rendered as links #3890

Closed

adam-enko mentioned this pull request Oct 31, 2024

Update fileTree test util to only use NIO Path #3892

Merged

adam-enko added 4 commits November 4, 2024 14:55

tidy after merge

7fe86c9

add project path to log message

c65a390

remove unused code... (maybe use it again in the future?)

718cc3a

adam-enko added 5 commits November 21, 2024 14:52

Add kdoc to dokkaSourceSetSpecFactory

d5d394f

Improve KmpCommonSourceSharedWithDependentsTest

35f960e

Update KotlinAdapter:

6f299d5

- Filter out metadata compilations when determining the source set target. - Remove KotlinPlatform warning - it's always triggered for commonMain. - Fetch _all_ dependent source sets, including transitives.

copy old comment for DSS.suppress

31d521c

revert adding arg name

e62424b

adam-enko changed the title ~~Fix common source sets not propogated~~ Fix DGP/KMP integration, so Dokka can 'see' code from shared source sets in target source sets Nov 21, 2024

adam-enko added 3 commits November 21, 2024 15:07

extract KotlinPlatform computation to a function

36eb7d7

revert depending on all transitive source sets

aca1c58

disable KmpCommonSourceSharedWithDependentsTest, because of #3382

c1d803c

adam-enko requested a review from whyoleg November 21, 2024 18:54

adam-enko added this to the Dokka 2.0.0 milestone Nov 21, 2024

adam-enko marked this pull request as ready for review November 21, 2024 18:55

whyoleg approved these changes Nov 22, 2024

View reviewed changes

adam-enko added 2 commits November 25, 2024 11:52

Merge branch 'refs/heads/master' into adam/feat/KT-70336/pass-common-…

15c0bcb

…src-to-dependents

convert KmpCommonSourceSharedWithDependentsTest from functional test …

f0bbb1b

…to integration test, to support testing different KGP versions

antohaby approved these changes Nov 25, 2024

View reviewed changes

adam-enko added 6 commits November 25, 2024 13:45

Add test for DokkaSourceSetSpec conventions

b8b5f8b

update assertion

3c52f0b

update dokkaGenerateModuleHtml assertion

82774ae

log dokka config

fe97896

log dokka config

f582d0d

log dokka config

b58d42f

adam-enko force-pushed the adam/feat/KT-70336/pass-common-src-to-dependents branch from 31d28b3 to b58d42f Compare November 26, 2024 12:15

adam-enko added 5 commits November 26, 2024 13:15

Merge branch 'master' into adam/feat/KT-70336/pass-common-src-to-depe…

e163110

…ndents

add more targets to workaround issue where Dokka gets confused with s…

d38aece

…ingle-target source sets

improve logging dokka config

a04eae9

disable KMP IT HTML comparison test

302ae58

grammar fix

bb77c17

adam-enko merged commit b7102d4 into master Nov 26, 2024
14 checks passed

adam-enko deleted the adam/feat/KT-70336/pass-common-src-to-dependents branch November 26, 2024 15:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix DGP/KMP integration, so Dokka can 'see' code from shared source sets in target source sets #3814

Fix DGP/KMP integration, so Dokka can 'see' code from shared source sets in target source sets #3814

adam-enko commented Sep 19, 2024 •

edited

Loading

whyoleg left a comment

whyoleg Nov 22, 2024

whyoleg Nov 22, 2024

adam-enko Nov 25, 2024 •

edited

Loading

adam-enko Nov 25, 2024

whyoleg Nov 25, 2024

adam-enko Nov 25, 2024

whyoleg Nov 25, 2024

adam-enko Nov 26, 2024

whyoleg Nov 26, 2024

adam-enko Nov 26, 2024 •

edited

Loading

Fix DGP/KMP integration, so Dokka can 'see' code from shared source sets in target source sets #3814

Fix DGP/KMP integration, so Dokka can 'see' code from shared source sets in target source sets #3814

Conversation

adam-enko commented Sep 19, 2024 • edited Loading

whyoleg left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adam-enko Nov 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adam-enko Nov 26, 2024 • edited Loading

Choose a reason for hiding this comment

adam-enko commented Sep 19, 2024 •

edited

Loading

adam-enko Nov 25, 2024 •

edited

Loading

adam-enko Nov 26, 2024 •

edited

Loading