inspect should show some useful info about modules (500USD Bounty)
#2929
Comments
lihaoyi
changed the title
inspect should show some useful info about modulesinspect should show some useful info about modules (500USD Bounty)
|
@lefou can I take a stab at this? are there any expected challenges/hurdles, or should the task be relatively straightforward? |
|
@Shri333 go for it. It should be relatively straightforward. It will probably involve changes to the mill-moduledefs plugin to save the scaladoc for |
|
I am trying to build mill's source (main). It failed with JDK 21 but worked with JDK 17 ( |
|
@Shri333 for IDE support you can set up a Scala <-> Mill is currently only built and tested using Java 17, so not surprising it doesn't build cleanly on newer versions. We'll get to upgrading our CI at some point |
|
It's not Mill, that's not running with Java 21, it's only some test cases, which exercise older libraries/tools not yet compatible with Java 21.
|
|
Is there a particular debug configuration that you use in IDEA? |
|
@Shri333 typically I run tests from the terminal and use println debugging. I'm not actually that familiar with usage of the IntelliJ debugger haha |
|
It seems to no longer output an error anymore (the |
|
@Shri333 that's because it resolves to the |
|
Ok that makes sense. Thanks! |
|
@lihaoyi when transforming to copy scala doc into java annotations in the module defs plugin, which type in |
|
objects are ModuleDef, lazy vals are ValDef with a flag iirc |
|
So, there are no changes required to this compiler plugin? |
|
I don't know, you'll have to tell me haha |
|
Looks like it works for objects. I got the following working: /**
* test scaladoc
*/
object hello extends Module {
} |
|
with the following changes: case r: Resolved.Module => {
val a = r.cls.getAnnotation(classOf[mill.moduledefs.Scaladoc])
println(mill.util.Util.cleanupScaladoc(a.value).map("\n " + _).mkString) |
|
What |
|
The point of |
|
Yeah I was thinking about doing that. I kind of wanted to stick to the existing pattern though. I will try that. |
|
I am trying to push my branch with local changes to create a draft PR, but I do not have access. @lihaoyi can you add me? |
|
you need to fork the repo on github and push to your fork to create a pr from that |
|
What is the easiest way to get file/line info? |
|
For example, I tried the following, but it is not perfect |
|
@Shri333 is the module's own |
|
Actually, it looks like it is working for now. It was not working before for some reason (maybe some change fixed it?). |
|
added support for filename/linenum, scaladoc, inherited modules, module dependencies, default task, and all tasks if you want to take a look @lihaoyi Example outputs: /**
* scaladoc for foo
* foo is a module that extends MyModule
*/
object foo extends MyModule {
def moduleDeps = Seq(bar)
def ivyDeps = Agg(
ivy"net.sourceforge.argparse4j:argparse4j:0.9.0",
)
}/**
* test scaladoc
*/
object hello extends Module {
} |
|
@Shri333 I think that looks great! Some thoughts: a lot of the verbosity comes from transitively resolving stuff that isn't directly written in the code. Let's try
For testing, you should be able to re-use the |
|
@lihaoyi also, can I use scala reflect instead of java reflect? It was added in scala 2.10, but I am not sure about backwards compat issues. |
|
lets avoid scala reflect; using it for macros is fine, but at rintime it has a pretty big performance hit from running the scala compiler |
|
Hmm, java reflect works for everything except filtering out methods directly declared on an object. It seems to be an issue with how scala compiles objects into bytecode. Instead of getting the methods directly declared on an object (like |
|
@Shri333 what are you trying to do? If you're not sure how to use java-reflect for Scala/Mill, you can see |
|
I was trying to get all tasks that are explicitly declared on a module. For example, for the module below: /**
* scaladoc for foo
* foo is a module that extends MyModule
*/
object foo extends MyModule {
def moduleDeps = Seq(bar)
def ivyDeps = Agg(
ivy"net.sourceforge.argparse4j:argparse4j:0.9.0",
)
}I used Reflect.reflect(t.cls, classOf[Target[_]], Function.const(true), noParams = true)But, I got a bunch of extra methods: |
|
Also, I added tests and fixed some of the resolve tests, but other resolve tests don't seem to be working correctly. I fixed some of them, but the ones that use selectors do not seem to filter properly |
@Shri333 That is because As for the resolve tests, you'll have to look into your code changes and see why they broke. |
|
It's the same with val tasks = Reflect.reflect(
cls,
classOf[Target[_]],
Function.const(true),
noParams = true,
declaredMethods = true
).map(_.getName).distinct def reflect(
outer: Class[_],
inner: Class[_],
filter: String => Boolean,
noParams: Boolean,
declaredMethods: Boolean = false
): Seq[java.lang.reflect.Method] = {
val res = for {
m <- if (declaredMethods) outer.getDeclaredMethods else outer.getMethods
n = decode(m.getName)
if filter(n) &&
isLegalIdentifier(n) &&
(!noParams || m.getParameterCount == 0) &&
(m.getModifiers & Modifier.STATIC) == 0 &&
inner.isAssignableFrom(m.getReturnType)
} yield m
...
...
...
}Tasks: allIvyDeps, allLocalMainClasses, allSourceFiles, allSources, artifactId, artifactName, artifactNameParts, artifactSuffix, assembly, bspCompileClassesPath, bspCompileClasspath, bspCompiledClassesAndSemanticDbFiles, bspLocalClasspath, bspLocalRunClasspath, bspTransitiveCompileClasspath, bspTransitiveLocalClasspath, compile, compileClasspath, compileIvyDeps, compileResources, compiledClassesAndSemanticDbFiles, docJar, docJarUseArgsFile, docResources, docSources, finalMainClass, finalMainClassOpt, forkArgs, forkEnv, forkWorkingDir, generatedSources, ideaCompileOutput, ivyDeps, jar, javacOptions, javadocOptions, launcher, localClasspath, localCompileClasspath, localRunClasspath, mainClass, mandatoryIvyDeps, mandatoryJavacOptions, manifest, platformSuffix, prependShellScript, resolvedIvyDeps, resolvedRunIvyDeps, resolvedSemanticDbJavaPluginIvyDeps, resources, runClasspath, runIvyDeps, runUseArgsFile, semanticDbData, semanticDbEnablePluginScalacOptions, semanticDbJavaVersion, semanticDbPluginClasspath, semanticDbPluginIvyDeps, semanticDbScalaVersion, semanticDbVersion, sourceJar, sources, transitiveCompileClasspath, transitiveCompileIvyDeps, transitiveIvyDeps, transitiveLocalClasspath, unmanagedClasspath, upstreamAssembly, upstreamAssembly2, upstreamAssemblyClasspath, upstreamCompileOutput, zincAuxiliaryClassFileExtensions, zincIncrementalCompilation, zincReportCachedProblems |
|
Maybe some of them are synthetic trait forwarders? do they have any naming convention or modifiers thst distinguish them? |
|
Doesn't seem so: Tasks: public mill.define.Target build_.package_$foo$.allIvyDeps(), public mill.define.Target build_.package_$foo$.allLocalMainClasses(), public mill.define.Target build_.package_$foo$.allSourceFiles(), public mill.define.Target build_.package_$foo$.allSources(), public mill.define.Target build_.package_$foo$.artifactId(), public mill.define.Target build_.package_$foo$.artifactName(), public mill.define.Target build_.package_$foo$.artifactNameParts(), public mill.define.Target build_.package_$foo$.artifactSuffix(), public mill.define.Target build_.package_$foo$.assembly(), public mill.define.Target build_.package_$foo$.bspCompileClassesPath(), public mill.define.Target build_.package_$foo$.bspCompileClasspath(), public mill.define.Target build_.package_$foo$.bspCompiledClassesAndSemanticDbFiles(), public mill.define.Target build_.package_$foo$.bspLocalClasspath(), public mill.define.Target build_.package_$foo$.bspLocalRunClasspath(), public mill.define.Target build_.package_$foo$.bspTransitiveCompileClasspath(), public mill.define.Target build_.package_$foo$.bspTransitiveLocalClasspath(), public mill.define.Target build_.package_$foo$.compile(), public mill.define.Target build_.package_$foo$.compileClasspath(), public mill.define.Target build_.package_$foo$.compileIvyDeps(), public mill.define.Target build_.package_$foo$.compileResources(), public mill.define.Target build_.package_$foo$.compiledClassesAndSemanticDbFiles(), public mill.define.Target build_.package_$foo$.docJar(), public mill.define.Target build_.package_$foo$.docJarUseArgsFile(), public mill.define.Target build_.package_$foo$.docResources(), public mill.define.Target build_.package_$foo$.docSources(), public mill.define.Target build_.package_$foo$.finalMainClass(), public mill.define.Target build_.package_$foo$.finalMainClassOpt(), public mill.define.Target build_.package_$foo$.forkArgs(), public mill.define.Target build_.package_$foo$.forkEnv(), public mill.define.Target build_.package_$foo$.forkWorkingDir(), public mill.define.Target build_.package_$foo$.generatedSources(), public mill.define.Target build_.package_$foo$.ideaCompileOutput(), public mill.define.Target build_.package_$foo$.ivyDeps(), public mill.define.Target build_.package_$foo$.jar(), public mill.define.Target build_.package_$foo$.javacOptions(), public mill.define.Target build_.package_$foo$.javadocOptions(), public mill.define.Target build_.package_$foo$.launcher(), public mill.define.Target build_.package_$foo$.localClasspath(), public mill.define.Target build_.package_$foo$.localCompileClasspath(), public mill.define.Target build_.package_$foo$.localRunClasspath(), public mill.define.Target build_.package_$foo$.mainClass(), public mill.define.Target build_.package_$foo$.mandatoryIvyDeps(), public mill.define.Target build_.package_$foo$.mandatoryJavacOptions(), public mill.define.Target build_.package_$foo$.manifest(), public mill.define.Target build_.package_$foo$.platformSuffix(), public mill.define.Target build_.package_$foo$.prependShellScript(), public mill.define.Target build_.package_$foo$.resolvedIvyDeps(), public mill.define.Target build_.package_$foo$.resolvedRunIvyDeps(), public mill.define.Target build_.package_$foo$.resolvedSemanticDbJavaPluginIvyDeps(), public mill.define.Target build_.package_$foo$.resources(), public mill.define.Target build_.package_$foo$.runClasspath(), public mill.define.Target build_.package_$foo$.runIvyDeps(), public mill.define.Target build_.package_$foo$.runUseArgsFile(), public mill.define.Target build_.package_$foo$.semanticDbData(), public mill.define.Target build_.package_$foo$.semanticDbEnablePluginScalacOptions(), public mill.define.Target build_.package_$foo$.semanticDbJavaVersion(), public mill.define.Target build_.package_$foo$.semanticDbPluginClasspath(), public mill.define.Target build_.package_$foo$.semanticDbPluginIvyDeps(), public mill.define.Target build_.package_$foo$.semanticDbScalaVersion(), public mill.define.Target build_.package_$foo$.semanticDbVersion(), public mill.define.Target build_.package_$foo$.sourceJar(), public mill.define.Target build_.package_$foo$.sources(), public mill.define.Target build_.package_$foo$.transitiveCompileClasspath(), public mill.define.Target build_.package_$foo$.transitiveCompileIvyDeps(), public mill.define.Target build_.package_$foo$.transitiveIvyDeps(), public mill.define.Target build_.package_$foo$.transitiveLocalClasspath(), public mill.define.Target build_.package_$foo$.unmanagedClasspath(), public mill.define.Target build_.package_$foo$.upstreamAssembly(), public mill.define.Target build_.package_$foo$.upstreamAssembly2(), public mill.define.Target build_.package_$foo$.upstreamAssemblyClasspath(), public mill.define.Target build_.package_$foo$.upstreamCompileOutput(), public mill.define.Target build_.package_$foo$.zincAuxiliaryClassFileExtensions(), public mill.define.Target build_.package_$foo$.zincIncrementalCompilation(), public mill.define.Target build_.package_$foo$.zincReportCachedProblems() I think it's a problem with the nature of scala objects itself and the corresponding bytecode that is produced. We may be able to do this with scala reflection: trait MyTrait1 {
def bar(): Unit = println("bar")
}
trait MyClass {
def baz(): Unit = println("baz")
}
object MyObject extends MyClass with MyTrait1 {
def foo(): Unit = println("foo")
}
typeOf[MyObject.type].members.filter(_.isMethod).toList
// List(method foo, constructor MyObject, method bar, method $init$, method baz, method synchronized, method ##, method !=, method ==, method ne, method eq, method notifyAll, method notify, method clone, method getClass, method hashCode, method toString, method equals, method wait, method wait, method wait, method finalize, method asInstanceOf, method isInstanceOf)
typeOf[MyObject.type].decls.filter(_.isMethod).toList
// List(constructor MyObject, method foo) |
|
@Shri333 Is this complete? If not does it bother you if I decide to attempt this 24 hours from now? |
|
@llvee this is not complete, but close to completion; there's prob one main thing to fix (displaying tasks) and task resolution |
|
@Shri333 Would you like some help with completing the rest? |
|
@llvee I have run into a few blockers, but I would like to work on them for a little bit. If I am not able to resolve them, I will mention you on this thread. |
|
@Shri333 Ok, I will continue checking notifications daily. I am up to help in the case that you would like some assistance. |
From the maintainer Li Haoyi: I'm putting a 500USD bounty on this issue, payable by bank transfer on a merged PR implementing this.
Currently,
mill inspectonly works for targets, but not for modules. For module, it just outputs an error message "inspect Cannot find default task to evaluate for module foo".Instead, it should provide some useful information:
The text was updated successfully, but these errors were encountered: