Complex plugin setup fix

[scphantm]

Need to write unit tests for the two new tasks but wanted to get this out there and have people start looking at it now. I needed groovydoc and scaladoc. I also had issues with the dreaded maven-publish bug of "can not modify publish after its been accessed" i'm building something on par with what netflix has and our publish is very complex. The bug has to do with the order that gradle executes the closures for the publish task, its very brittle. The fix is rather simple, instead of doing publishing{publish{}}, switch it to publishing.publish{} and you are now accessing as a field property, not a closure and the problem goes away. This code reflects that.

I also figured while im in there copying and pasting the two new tasks, i may as well refactor things to make them more in-line with how i do things that i know people are going to extend in their projects as well. for example, im going to need this to expand to sphinx doc, jsdoc, whatever the hell .net uses, etc. The way i refactored the code here allows me to quickly create those other tasks in my project with minimal code copying and fall into the nebula publishing scheme.

I will be testing in the morning and add some more unit tests for this.

[scphantm]

How do i get the test result reports from travis? I see in the log that its failing, but i don't understand, those tests run locally. id like to take a look at the report, its says its under /home/travis/build/nebula-plugins/nebula-publishing-plugin/build/reports/tests/test but that doesn't do me much good. Is there a way to get those reports from Travis?

[scphantm]

Speaking of testing, you may want to activate appveyor as well. Tests ran 100% on my mac, then moved to my windows work machine and 24 failed. the reason for the fail, ivy { url 'C:\....... backslashes can't be in the url scheme. works fine for Posix file systems, doesn't work for FAT32/NTFS file systems. Ill address the tests tomorrow, but you really should activate appveyor so the system is tested on both linux and windows systems across the entire nebula suite.

[rspieldenner]

From my understanding publishing.publications accesses the publishing block, where as the series of closures will apply later when a publishing task is run.

[scphantm]

it has something to do with the timing. ive heard several discussions on what causes it. What i do know for sure is this is universally referred to as the fix and Gradle's answer for the past 2 or so years is publish-maven is in incubation.

[rspieldenner]

Need to see a test that fails with

project.publishing {
  publications {
    // config
  }
}

[scphantm]

wow, thats going to be tough because its buried about three layers of custom plugins deep in a CI server. It has to do with the core publish being in the core plugin that everything else inherits, with child plugins and such. Its really tough, I do know its a well known groovy/gradle issue because it only took me a minute or two to find the solution when i hit it. I can track down the research when im on my work machine tomorrow.

[scphantm]

researchgate/gradle-release#125

https://stackoverflow.com/questions/28020520/custom-gradle-plugin-causes-cannot-configure-the-publishing-extension

are where i got the fix for the "can not modify publish after its been accessed" error. As i said, its a well known gradle/groovy issue in their maven-publish plugin.

Still trying to figure out a way to boil it down to a unit test. the conditions are pretty specific and not easy to replicate.

[scphantm]

http://stackoverflow.com/questions/21190230/how-can-plugin-programmatically-configure-maven-publish-publishing-and-allow-bui

Apparently Justin Ryan built a class in a very early version of nebula publish to try to handle this specific issue. but just changing the syntax is far easier than his fix.

It looks like i have no choice but to build an additional custom plugin (a build.gradle file is nothing more than a plugin). It seams you can't replicate the problem from a single plugin closure, like a build.gradle file. It seems like you can only replicate it from a parent/child plugin relationship, where the parent introduces publishing and finishes its apply method, then a child method attempts to access the publishing from its apply method. At least i think so.

[scphantm]

Would it be ok if I put the prelim stuff in https://github.com/nebula-plugins/gradle-nothing-plugin

create a new plugin in that project to use to replicate this here? I can't figure out any other way to replicate it than with an external plugin. but ill keep thinking

[scphantm]

this latest commit will fail until the nebula-test and nebula-plugin projects are updated. Once those are updated, than i can strip out my hackery from this build and it should pass. i submitted the bug fixes to the nebula-test project already, im assuming whoever does that merge will update nebula-plugins

[scphantm]

this integration is waiting for my PR from nebula-test. Once that is merged in, and hopefully whomever can merge that new library into nebula-plugin, then i can get this wrapped up. the bugs that caused the tests to fail were not in this plugin. they were in nebula-test.

[scphantm]

waiting for this PR so i can finish this
nebula-plugins/nebula-test#77

[scphantm]

Ok, this is maddening. I have windows machines and mac machines in front of me, and i can't get the tests to fail, yet they fail on travis. Is there a way i can get the junit report from travis? I can't find where in the stack its screwing up, something with nebula.info it looks like, which makes no sense.

[rspieldenner]

tomorrow i'll take a look at if any of our plugins went and published dynamic versions and are pulling in something odd for the info plugin here

[scphantm]

I appreciate that because i just finished installing a linux vm on my home mac and it just ran clear. The only errors we are getting is on travis, Windows, MacOS, and Ubuntu all run clear for me.

[scphantm]

HA!!, i dumped my local maven cache and realized the project was pulling both version 3.1.0 and 3.5.0 of nebula.info. Why gradle didn't catch that and use 3.5.0 is beyond me. but it passed. Now, to build that unit test you want, the only way i can think of to do it is create an external plugin that publishes. I can add an additional plugin to that "do nothing" plugin that i can use to replicate it (i think) but im not sure if you want that project used like that. And ive never tried to have a unit test pull in an external plugin, that will be new. Thoughts?

[rspieldenner]

Test setup create plugin publish to repo in test directory, have code under test consume simple plugin

…

On Wed, Feb 15, 2017 at 7:26 PM Steel City Phantom ***@***.***> wrote: HA!!, i dumped my local maven cache and realized the project was pulling both version 3.1.0 and 3.5.0 of nebula.info. Why gradle didn't catch that and use 3.5.0 is beyond me. but it passed. Now, to build that unit test you want, the only way i can think of to do it is create an external plugin that publishes. I can add an additional plugin to that "do nothing" plugin that i can use to replicate it (i think) but im not sure if you want that project used like that. And ive never tried to have a unit test pull in an external plugin, that will be new. Thoughts? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#71 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAvvFo9QnWs1OFL3bXs_1sf0gqAcDR0_ks5rc8HqgaJpZM4L6Vqw> .

[scphantm]

I can't get it to replicate outside of my build system. My build system has dozens of different plugins, 30 of them proprietary in a 6 layer stack (shockingly, i actually counted today) depending on what you apply at what time, we even do plugin injection from the CI server directly into the product build file. I tried to replicate a small chunk of that by adding some subprojects to this in order to replicate the 3 primary plubshing stacks, and one of my JVM ninjas just walked by and told me i was wasting my time because he thinks testkit actually can't replicate it due to the subtle differences in how testkit initializes a project vs command line. Makes a lot of sense because the problem skipped past our unit testing and didn't show up until we rolled out the first piece of the new publishing system, all i know for sure is this fixed the problem internally and I have a deadline looming and have been working on this since 1am.

[rspieldenner]

So it was originally the complicated way Justin showed in the post you linked. We were having all kinds of issues whenever anybody called project.properties() (which resolves the model). Or if people wanted to manually tweak any of the publications (having their own publishing { publications { nebula(MavenPublication) { //config } } }) Or if they did something like project.publishing.publications.each {}

Swapping to the simpler way here which echoes what is done in a build.gradle cleared up the issues for us. I'm willing to give this other syntax (publishing.publications) a shot, but if our internal acceptance tests show issues with it I'm going to roll it back to the other way (publishing { publications {}}).

[scphantm]

I have a half dozen different plug types (some binary some script) that are each intended to tweak the publishing to something very specific for that "classification" of project. so if its a properties project, it publishes a zip file and NOTHING else, if its a java, blow up if there isn't a javadoc publication, etc, etc. Sounds like we have similar problems.

Run it thru internal i suppose. If it works, great, if not, i will roll those lines back and do a touch of refactoring to this and instead of pulling it in as a plugin, i pull it in as a library and use it to build my own internal publish plugin. then we can both have it our way.

[scphantm]

How is the testing going? I have been thinking about this a lot and i can refactor this to turn it from a plugin to a plugin/library. I can then extend that library into my own plugin thats setup the way i need it to be if its causing issues for others.