0.45.0 backwards incompatibility: KtLint.Params#userData gets ignored, unclear migration path + more feedback

[mateuszkwiecinski]

Initially reported on Kotlin slack: https://kotlinlang.slack.com/archives/CKS3XG0LS/p1647897839331439 but I was asked to create issue here.

Expected Behavior

0.45.0 is backward compatible with 0.44.0, or has clear migration path

Observed Behavior

I'll copy-paste what I wrote on Slack:

1. Can someone share what’s the future of userData map, does it only change its purpose, or will it be removed completely at some point? The code I was upgrading had been passing indent_size and disabled_rules within userData map. The migration path for indent_size seemed to be hidden but I was able to identify DefaultEditorConfigProperties.indentSizeProperty which looked legitimate and worked in my case, but I couldn’t find a replacement for disabled_rules. The fact ExperimentalParams still takes userData suggests that’s still the “recommended” option, am I wrong here? What’s the proper way to pass rules that I expect to be disabled?
2. It seems like there is a behavior change introduced in 0.45.0, where values passed as userData using the now deprecated lint method are ignored. Maybe I misread the release notes, but they had vibe you strongly advise to migrate now, because later it will be more difficult, but in overall 0.45.0 is still backwards compatible. If breaking the compatibility was intended, then I’ll share my personal opinion: I think it would be better for api consumers if you removed the method already in this release so api consumers would get a compile-time error, instead of runtime one (or bug reports from end users 😛). If breaking the compatibility wasn’t intended, then FYI it did, at least the case described above 😛
   And just for context: I attempted to bump ktlint in Gradle Plugin, so I might be missing something here, as the release notes weren’t 100% clear for me (for example: I’ve never seen ASTNode.getEditorConfigValue(editorConfigProperty) or EDITOR_CONFIG_USER_DATA_KEY mentioned there 😬 )
3. I noticed isUnitTestContext = true gets passed when calling any of KtLint#lint methods. It doesn’t break anything, but looks suspicious, so I thought it might be worth pointing out, especially when seeing todos like this 👇
   AFAIU both ktlint-gradle and kotlinter-gradle use lint and format in regular usage, not in tests soo… 👀
   

Feel free to split them into 3 separate issues if it's more convenient for you :)

Steps to Reproduce

Run KtLint.lint(KtLint.Params(userData = mapOf("indent_size" to "5"), ...)) on a file with respective indentation and observe the lint reporting failures.

Your Environment

• Version of ktlint used: 0.45.1
• Relevant parts of the .editorconfig settings: none
• Name and version (or code for custom task) of integration used (Gradle plugin, Maven plugin, command line, custom Gradle task): I tried to upgrade kotlinter-gradle and the tests failed: https://github.com/jeremymailen/kotlinter-gradle/pull/243/files I had to switch to the ExperimentalParams class which introduced even more confusion - hence all the questions above
• Version of Gradle used (if applicable): it's not
• Operating System and version: macos-latest, windows-latest

[paul-dingemans]

Can someone share what’s the future of userData map, does it only change its purpose, or will it be removed completely at some point? The code I was upgrading had been passing indent_size and disabled_rules within userData map. The migration path for indent_size seemed to be hidden but I was able to identify DefaultEditorConfigProperties.indentSizeProperty which looked legitimate and worked in my case, but I couldn’t find a replacement for disabled_rules. The fact ExperimentalParams still takes userData suggests that’s still the “recommended” option, am I wrong here? What’s the proper way to pass rules that I expect to be disabled?

Unfortunately userData is not a single clear concept in ktlint.

UserData is set in the data class Params

public data class Params(
        val fileName: String? = null,
        val text: String,
        val ruleSets: Iterable<RuleSet>,
        val userData: Map<String, String> = emptyMap(),
        val cb: (e: LintError, corrected: Boolean) -> Unit,
        val script: Boolean = false,
        val editorConfigPath: String? = null,
        val debug: Boolean = false
    )

In a rule, the userData can be retrieved from the node:

override fun visit(
    node: ASTNode,
    autoCorrect: Boolean,
    emit: (offset: Int, errorMessage: String, canBeAutoCorrected: Boolean) -> Unit
) {
   val editorConfig = node.getUserData(KtLint.EDITOR_CONFIG).
}

The userData which is added to the Params class is not exactly the same as what can be retrieved in the rule which can be unexpected. Internally in Ktlint the userData contains data from different sources:

1. Properties extracted from the .editorConfig file
2. Command line properties like the "--android" flag
3. Path of the file being processed
4. Possibly, other things that I might not yet have discovered.

Data which is retrieved via node.getUserData(KtLint.EDITOR_CONFIG) contains anything except the android flag and the file path because they are filtered out. Those fields can be retrieved via node.getUserData(ANDROID_USER_DATA_KEY)' and node.getUserData(FILE_PATH_USER_DATA_KEY). Now, lets assume that node.getUserData(KtLint.EDITOR_CONFIG)indeed contains.editorconfig` properties.

As the userData is defined as Map<String, String> we have to deal with the problem of properties which have special value in each rule where those properties are used. For example, the max_line_length property might be set to value "off" or an integer. This kind of logic should not be implemented in multiple rules because it violates the DRY principle. A Ktlint rule now has to implement interface UsesEditorConfigProperties in order to the encapsulated property with node.getEditorConfigValue(...). A rule can either reuse a property which is defined in DefaultEditorConfigProperties or a property which is defined in another rule. The rule in which the property is defined, is considered to be the owner of the property.

In future releases, the userData parameter might be stripped down further by removing the --android and thefile path and provide another way to retrieve this data inside the rule. The userData will only be removed when it is not needed anymore inside Ktlint and our API users have not provided a uses cases for its existence.

Properties from .editorconfig can now best be passed via editorConfigOverride in ExperimentalParams:

public data class ExperimentalParams(
    val fileName: String? = null,
    val text: String,
    val ruleSets: Iterable<RuleSet>,
    val userData: Map<String, String> = emptyMap(),
    val cb: (e: LintError, corrected: Boolean) -> Unit,
    val script: Boolean = false,
    val editorConfigPath: String? = null,
    val debug: Boolean = false,
    val editorConfigOverride: EditorConfigOverridesMap = emptyMap(),
    val isInvokedFromCli: Boolean = false
)

It seems like there is a behavior change introduced in 0.45.0, where values passed as userData using the now deprecated lint method are ignored. Maybe I misread the release notes, but they had vibe you strongly advise to migrate now, because later it will be more difficult, but in overall 0.45.0 is still backwards compatible. If breaking the compatibility was intended, then I’ll share my personal opinion: I think it would be better for api consumers if you removed the method already in this release so api consumers would get a compile-time error, instead of runtime one (or bug reports from end users 😛). If breaking the compatibility wasn’t intended, then FYI it did, at least the case described above 😛

Breaking the API was definitively not intended. I have tried to deprecate while keeping support for old behavior but it seems that I missed a use case. I am affraid that our test coverage for API usage is insufficient. It would help, if you could share some unit tests that demonstratie what currently fail for you as API user.

The recommendation for not skipping upgrades is that the deprecation periods will be shortened to maybe as short as 1 release. I would like to evolve the API fast instead of having to spend a lot of effort in keeping backwards compatibility over a series of releases.

And just for context: I attempted to bump ktlint in jeremymailen/kotlinter-gradle#243, so I might be missing something here, as the release notes weren’t 100% clear for me (for example: I’ve never seen ASTNode.getEditorConfigValue(editorConfigProperty) or EDITOR_CONFIG_USER_DATA_KEY mentioned there 😬 )

Perhaps, you have missed section "Retrieving ".editorconfig" property value" in the release notes.

I noticed isUnitTestContext = true gets passed when calling any of KtLint#lint methods. It doesn’t break anything, but looks suspicious, so I thought it might be worth pointing out, especially when seeing todos like this 👇
AFAIU both ktlint-gradle and kotlinter-gradle use lint and format use this in regular usage, not in tests soo…

Ow, that could lead to problem when rules are using the VisitiorModifier.RunAfterRule incorrectly. Now, the check whether rules can be properly ordered is not working. I expect that the impact with the standard and experimental rules of Ktlint is minimal to none as both rules that are using this modifier are also tagged with VisitiorModifier.RunAsLateAsPossible. But it can result in some problems when rules are provided via custom rulesets and are not executed in the expected order.

Previously this lint method using the ExperimentalParams class was only used for unit tests.

@OptIn(FeatureInAlphaState::class)
public fun lint(experimentalParams: ExperimentalParams) {
    lint(
        experimentalParams,
        VisitorProvider(
            ruleSets = experimentalParams.ruleSets,
            debug = experimentalParams.debug,
            isUnitTestContext = true
        )
    )
}

I hope that I have properly answered your questions. If not, feel free to ask more questions. But it might be a better idea to continue in separate issues...

[ZacSweers]

This change has made KtLint unusable in essentially every third party plugin for it (gradle plugins, etc) because they all relied on userData. While I understand it's complicated, I think it should be temporarily restored until there's a proper path forward for integrations

[paul-dingemans]

This change has made KtLint unusable in essentially every third party plugin for it (gradle plugins, etc) because they all relied on userData. While I understand it's complicated, I think it should be temporarily restored until there's a proper path forward for integrations

I don't think restoring is the way forward. We need to figure out how we can move forward. But this can only be done with input from the community. So lets work together to identify problems like @mateuszkwiecinski did and define possible solutions for it. Those solutions should then be release as 0.45.x till we have a working release for the API consumers again. So I welcome proposals to the different problems mentioned above.

[paul-dingemans]

Most problems seem not that difficult to fix. I am working on it and expect it to fix next week.

[romtsn]

tbh, given that ktlint is still not at 1.0.0, I think it's probably fine to just break API and document the migration path properly. I'd pretty much prefer a compile/runtime exception rather than this hidden behavior of userData

[mateuszkwiecinski]

Unfortunately userData is not a single clear concept in ktlint.

Thanks, Paul, for explaining what the userData is and how it's used 🙏

It would help, if you could share some unit tests that demonstratie what currently fail for you as API user.

Here's something that passes on 0.44.0 and fails on 0.45.1 (with Unexpected indentation (2) (should be 4) error logged in the list)

    @Test
    fun repro() {
        val lintErrors = mutableListOf<LintError>()
        KtLint.lint(
            KtLint.Params(
                userData = mapOf("indent_size" to "2"),
                text = """
                    class WrongFileName {
        
                      fun someFun() = 2
                    }
                    
                    """.trimIndent(),
                cb = { lintError, _ ->
                    lintErrors.add(lintError)
                },
                ruleSets = listOf(StandardRuleSetProvider()).map(StandardRuleSetProvider::get)
            )
        )
        check(lintErrors.isEmpty())
    }

hope that's helpful :)

Perhaps, you have missed section "Retrieving ".editorconfig" property value" in the release notes.

I'm only the API consumer (I hope we agree the public methods of Ktlint object can be called the public API). I've never "retrieved" any editor config value, I've never operated on raw ASTNodes. It feels like when prototyping the latest api changes, you focused mainly on people who write custom rules and prepared migration path for them, but people maintaining ktlint integrations or wrappers (like Gradle plugins) were left out 👀

If not, feel free to ask more questions

I have one more question, about my third point, I'm not sure if I understood what should I/we do to properly use ktlint. You mentioned lint method which takes ExperimentalParams but the isUnitTestContext = true gets passed for the old Params overload too, see here:

ktlint/ktlint-core/src/main/kotlin/com/pinterest/ktlint/core/KtLint.kt

Line 144 in 8809f48

isUnitTestContext = true

Does it mean basically all integrations use ktlint in a wrong, unexpected way? 🤔

[paul-dingemans]

Here's something that passes on 0.44.0 and fails on 0.45.1 (with Unexpected indentation (2) (should be 4) error logged in the list)

    @Test
    fun repro() {
        val lintErrors = mutableListOf<LintError>()
        KtLint.lint(
            KtLint.Params(
                userData = mapOf("indent_size" to "2"),
                text = """
                    class WrongFileName {
        
                      fun someFun() = 2
                    }
                    
                    """.trimIndent(),
                cb = { lintError, _ ->
                    lintErrors.add(lintError)
                },
                ruleSets = listOf(StandardRuleSetProvider()).map(StandardRuleSetProvider::get)
            )
        )
        check(lintErrors.isEmpty())
    }

hope that's helpful :)

In this test the userData does contains the .editorconfig property indent. To make this more clear, the next version throws an exception with message: UserData should not contain '.editorconfig' properties [indent_size]. Such properties should be passed via the 'ExperimentalParams.editorConfigOverride' field. Note that this is only required for properties that (potentially) contain a value that differs from the actual value in the '.editorconfig' file.

The test above has to be rewritten as follows:

    @Test
    fun repro() {
        val lintErrors = mutableListOf<LintError>()
        KtLint.lint(
            KtLint.ExperimentalParams(
                userData = emptyMap(),
                editorConfigOverride = EditorConfigOverride.from(DefaultEditorConfigProperties.indentSizeProperty to 2),
                text = """
                    class WrongFileName {

                      fun someFun() = 2
                    }

                    """.trimIndent(),
                cb = { lintError, _ ->
                    lintErrors.add(lintError)
                },
                ruleSets = listOf(StandardRuleSetProvider()).map(StandardRuleSetProvider::get)
            )
        )
        check(lintErrors.isEmpty())
    }

I'm only the API consumer (I hope we agree the public methods of Ktlint object can be called the public API). I've never "retrieved" any editor config value, I've never operated on raw ASTNodes. It feels like when prototyping the latest api changes, you focused mainly on people who write custom rules and prepared migration path for them, but people maintaining ktlint integrations or wrappers (like Gradle plugins) were left out 👀

"Retrieving" an .editorconfig value is something that is done by KtLint. When putting is very simplified, it means that for each file that is processed by Ktlint, the corresponding .editorconfig' file is detected and read. If a rule takes settings from the .editorconfig` into account, those settings are to be retrieved from that file.

I agree with you that public methods in general should be considered the public API. Currently Ktlint exposes public methods which we want to remove from the public API. We want to offer a clean, small and well documented API. We have some iterations to go before we will arrive at that point.

Indeed, I am not sure how people maintaining ktlint integrations or wrappers (like Gradle plugins) are affected by those changes. Can you help me to understand this? What does your project do and how does KtLint fit into that picture? Why would you add a test like above to your project?

I have one more question, about my third point, I'm not sure if I understood what should I/we do to properly use ktlint. You mentioned lint method which takes ExperimentalParams but the isUnitTestContext = true gets passed for the old Params overload too, see here:

ktlint/ktlint-core/src/main/kotlin/com/pinterest/ktlint/core/KtLint.kt

Line 144 in 8809f48

isUnitTestContext = true

Does it mean basically all integrations use ktlint in a wrong, unexpected way? 🤔
The way you want to use it is correct. At the time that I added the isUnitTestContext parameter, I was not aware of how this would affect the API Consumers. In the next release, this is fixed.

Tnx again for your questions and remarks.

[mateuszkwiecinski]

Thanks for a detailed response 🙏

Can you help me to understand this? What does your project do and how does KtLint fit into that picture? Why would you add a test like above to your project?

I can only describe my use-case, I won't represent all other applications. So, kotlinter-gradle (which I occasionally contribute to) is a Gradle wrapper around ktlint (that's the plugin mentioned in ktlint's readme). It allows customizing ktlint behaviour by passing custom settings via Gradle extension:

kotlinter {
    ignoreFailures = false
    indentSize = 4
    reporters = ["checkstyle", "plain"]
    experimentalRules = false
    disabledRules = ["no-wildcard-imports"]
}

plugin users don't interact with ktlint classes directly, all the configuration happens via dedicated Gradle extensions.
As you can see, one of the properties is indentSize, which I think was added only because historically ktlint had much poorer .editorconfig support.
So, the answer to your question:

Why would you add a test like above to your project?

Would be: to verify if Gradle plugin consumer can still override indentSize in a backwards compatible way (taking Gradle Plugin APIs into account)

Now, seeing ktlint integrates much better with .editorconfig and lerning that overriding .editorconfig properties via userData won't be supported - I'll probably propose to drop the indentSize argument from kotlinter-gradle plugin.

I want to raise one more topic.
Context: Gradle Plugins, both kotlinter-gradle and ktlint-gradle allow overriding runtime ktlint version which will be used when Gradle tasks are invoked. AFAIU this implicitly relies on the fact ktlint API was stable, and they interact with single Ktlint.lint(Params) method as it didn't change since 0.34.0. By deprecating Params and forcing Gradle plugins to use ExperimentalParams such feature would be broken - they won't be able to create ExperimentalParams on ktlint version0.34.0). And while it's perfectly fine to occasionally break backwards compatibility, my concern I wanted to voice here is it doesn't feel light to drop the previous "stable" solution Params in favour of a class with a Experimental prefix, requiring explicit OptIn annotation.
So I'd ask if you'd consider any other approach, to make sure there is at least one "stable" API at any given time, because after your PR gets merged we'll have the choice either to use deprecated, but backwards compatible Params or unstable ExperimentalParams, which then feels like there is no way to provide reliable software anymore, as its behaviour is about to break regardless of which choice one picks :/
(☝️ the above assumes maintaining a backwards compatible solution doesn't come at a significant cost, and you're willing to commit to ensure such. And a reminder: I'm only sharing feedback, feel free to ignore it 😄)

[jeremymailen]

Thank you for all your work @paul-dingemans and recognize you're shepherding inherited code with some wrinkles.
I agree 100% that moving forward is the way to go. The old userData mechanism was quite vague and you're making things better by providing explicit arguments.

For kotlinter-gradle at least, we're quite fine having a compatibility boundary where kotlinter-gradle version, say 4.0.0, can only be used with ktlint 0.45.2 or later. Also quite fine removing some of the params like indentSize which only existed from the times when this wasn't configured by editorconfig :).

With this in mind, could we move forward with more confidence as @mateuszkwiecinski implies -- ExperimentalParams could just become the new Params and remove any alpha status?

Generally a gradle wrapper's needs in a ktlint API are:

• specify the relevant parameters to drive the lint or format task (which you are improving on)
• receive back a usable results structure (which is there)
• be able to get a list of resolved editorconfig files and other ktlint configuration in use (nice to have new request -- sometimes have to bust gradle cache if configuration has changed or otherwise have some insight into what ktlint is configured to do). Cheers 👍

[paul-dingemans]

Thank you for all your work @paul-dingemans and recognize you're shepherding inherited code with some wrinkles.

Tnx!

With this in mind, could we move forward with more confidence as @mateuszkwiecinski implies -- ExperimentalParams could just become the new Params and remove any alpha status?

In next release (0.46.0), I will definitely clean up a lot of the alpha and beta markings. I am sure that more changes will follow in the future but both maintainers and consumers now need an API which is less bloated. Hopefully this also sheds light on the next steps. Having said that, I expect following to be included in the release:

• remove current deprecations including 'Params'
• deprecating/removal ExperimentalParams. I have not yet decided on the new name but the structureborobably won't change

• be able to get a list of resolved editorconfig files and other ktlint configuration in use (nice to have new request -- sometimes have to bust gradle cache if configuration has changed or otherwise have some insight into what ktlint is configured to do).

Please file a separate issue. But please make more clear what you need. It can take a while before it will be implemented. My own backlog is also full of stuff that I want to achieve. And as winter is almost gone, I will have less time to spend.

[paul-dingemans]

@mateuszkwiecinski @jeremymailen Changes have been merged to master. It would be great if you could check out against the snapshot whether the compatibility issues have been resolved. If so, then I will request the patch release to be build.

[mateuszkwiecinski]

I tested the latest snapshot against kotlinter-gradle code from the moment when I created this issue. I wouldn't call it "compatibility issues have been resolved" as the build fails with "UserData should not contain '.editorconfig' properties ..." message, but I understood that's expected, and that's a noticeable improvement as the migration path is now clear.

So if I was asked if 0.46.0-SNAPSHOT is backwards compatible with i.e. 0.44.0 - I'd say it's not. But if you asked if 0.46.0-SNAPSHOT is something we could move forward than yes, after kotlinter-gradle dropped the indentSize it doesn't pass any userData and I don't see any other issues 👍 Thank you!

[jeremymailen]

Yes, thanks I think we're good to move ahead.

[paul-dingemans]

Thnx! @shashachu Would you be so kind to release current master as version 0.45.2?

[shashachu]

0.45.2 is released. Thanks everyone!