travis and classes.bak

[fommil]

I see this a lot

[error] (scoverage:compile) Could not create directory /home/travis/build/ensime/ensime-server/target/scala-2.11/classes.bak

But I'm also creating a lot of directories in my CI, so I know that travis doesn't just barf when directories are created.

Does the SBT file API perhaps throw if the dir already exists? This might just need a little bit of error handling to avoid this common race / io.

[sksamuel]

I see that error too. I don't know why it happens. I don't know who is
creating classes.bak either.

On 4 September 2014 21:58, Sam Halliday [email protected] wrote:

I see this a lot

error Could not create directory /home/travis/build/ensime/ensime-server/target/scala-2.11/classes.bak

But I'm also creating a lot of directories in my CI, so I know that travis
doesn't just barf when directories are created.

Does the SBT file API perhaps throw if the dir already exists? This might
just need a little bit of error handling to avoid this common race / io.

—
Reply to this email directly or view it on GitHub
#47.

[fommil]

given that sbt is basically 💩 I'm guessing it's sbt.

[sksamuel]

I saw your recent thread on sbt. I had to laugh it was fucking brilliant.

On 4 September 2014 22:56, Sam Halliday [email protected] wrote:

given that sbt is basically [image: 💩] I'm guessing it's sbt.

—
Reply to this email directly or view it on GitHub
#47 (comment)
.

[fommil]

And today somebody has gone and added MORE fucking magic to the sbt parser. They are on another planet.

[sksamuel]

I'll write typelevel build tool this weekend.
On 5 Sep 2014 08:07, "Sam Halliday" [email protected] wrote:

And today somebody has gone and added MORE fucking magic to the sbt
parser. They are on another planet.

—
Reply to this email directly or view it on GitHub
#47 (comment)
.

[fommil]

we use https://github.com/cage433/maker at work, which would be utterly amazing if it had the resources to bring it to the masses.

Imagine instead of abstract interfaces, being handed case classes containing real information.

Imagine instead of an "axis of configurations, scopes, tasks and monad state transformers", you had Tasks that depend on other Tasks, and the build tool tries its damndest to run as much stuff in parallel as possible. It took me 2 hours to write the ensime plugin... it took me a week to do the same for sbt.

[sksamuel]

Since I don't work I have the resources to do loads of plugins :)

I might have a bash this weekend dunno.

On 5 September 2014 09:26, Sam Halliday [email protected] wrote:

we use https://github.com/cage433/maker at work, which would be utterly
amazing if it had the resources to bring it to the masses.

Imagine instead of abstract interfaces, being handed case classes
containing real information.

Imagine instead of an "axis of configurations, scopes, tasks and monad
state transformers", you had Tasks that depend on other Tasks, and the
build tool tries its damndest to run as much stuff in parallel as possible.
It took me 2 hours to write the ensime plugin... it took me a week to do
the same for sbt.

—
Reply to this email directly or view it on GitHub
#47 (comment)
.

[fommil]

the author is currently working on add transitive dep support

[sksamuel]

Says in his readme.he doesnt like that
On 5 Sep 2014 12:01, "Sam Halliday" [email protected] wrote:

the author is currently working on add transitive dep support

—
Reply to this email directly or view it on GitHub
#47 (comment)
.

[fommil]

he has been persuaded otherwise. would you like me to put you in contact?

[sksamuel]

Contact for ...

On 5 September 2014 12:05, Sam Halliday [email protected] wrote:

he has been persuaded otherwise. would you like me to put you in contact?

—
Reply to this email directly or view it on GitHub
#47 (comment)
.

[fommil]

sorry, I misunderstood. You meant looking at this at the weekend, not looking at maker.

[sksamuel]

I meant I might dick about and write my own. Maker seems good but i want
one thats baaically a scala maven. Maven gets slagged off but its always
worked. Get rid of the xml and make it a bit more flexible. Winner.
On 5 Sep 2014 12:06, "Sam Halliday" [email protected] wrote:

sorry, I misunderstood. You meant looking at this at the weekend, not
looking at maker.

—
Reply to this email directly or view it on GitHub
#47 (comment)
.

[sksamuel]

My priority right now though is this breaded chicken sandwich that's taking
forever to cook
On 5 Sep 2014 12:09, "Stephen Samuel (Sam)" [email protected] wrote:

I meant I might dick about and write my own. Maker seems good but i want
one thats baaically a scala maven. Maven gets slagged off but its always
worked. Get rid of the xml and make it a bit more flexible. Winner.
On 5 Sep 2014 12:06, "Sam Halliday" [email protected] wrote:

sorry, I misunderstood. You meant looking at this at the weekend, not
looking at maker.

—
Reply to this email directly or view it on GitHub
#47 (comment)
.

[fommil]

actually, I know what you mean. I think a simple DSL that converts a terse language spec (like what SBT keep aiming for) into maven files would be awesome. That way, the entire maven infrastructure can be pulled in... but I don't think maven is very efficient at compiling scala and it doesn't do parallelism very well.

maker is very stable on our project. It just lacks a lot of features you'd normally want.

[sksamuel]

It's actually not hard to write a build tool. You pull in some deps using
Aether. And then you invoke scalac with the classpath pointing to them.

What's hard is coming up with the worst API I've ever seen in the history
of computing. That took some real talent and whoever designed SBT is an
expert.

On 5 September 2014 12:19, Sam Halliday [email protected] wrote:

actually, I know what you mean. I think a simple DSL that converts a terse
language spec (like what SBT keep aiming for) into maven files would be
awesome. That way, the entire maven infrastructure can be pulled in... but
I don't think maven is very efficient at compiling scala and it doesn't do
parallelism very well.

maker is very stable on our project. It just lacks a lot of features
you'd normally want.

—
Reply to this email directly or view it on GitHub
#47 (comment)
.

[fommil]

hehe. zinc is the way to do the compile.

If you're serious about it, I really recommend looking at maker. It's a solid foundation that has been hardened with real world experience and both its domain objects and philosophy are easy to understand. It could do with some topping and tailing for use in the open source world.

[fommil]

my wishlist for maker cage433/maker#96

[sksamuel]

Yeah I don't want to start something if there's no interest (ie the only
people who hate it are plugin writers), and maker seems like he has it
licked. Although development seems to have slowed on it.

The most annoying thing about SBT is that most of the bugs on scoverage are
SBT related. I can't fix most of them as I can't figure out how to get SBT
to do what I want, and no one (except Jacek) answers my questions on SO.

On 5 September 2014 12:36, Sam Halliday [email protected] wrote:

my wishlist for maker cage433/maker#96
cage433/maker#96

—
Reply to this email directly or view it on GitHub
#47 (comment)
.

[fommil]

dev happens when it is needed by the project it supports ;-) github doesn't tell the full story

[fommil]

I think the demand for a better build tool is quite strong. Any substantially large project starts to need to touch the build tool, and the dependency resolution times are just insane (which again impacts larger projects the most).

[sksamuel]

Why is it insane? Slow to download, slow to find out what it needs,...?

On 5 September 2014 13:50, Sam Halliday [email protected] wrote:

I think the demand for a better build tool is quite strong. Any
substantially large project starts to need to touch the build tool, and the
dependency resolution times are just insane (which again impacts larger
projects the most).

—
Reply to this email directly or view it on GitHub
#47 (comment)
.

[fommil]

very slow to work out what it needs. 10 minutes in some cases.

[fommil]

it uses ivy, and it is single threaded. also the downloads are very slow... I think it's also serial.

[fommil]

@sksamuel would it help the stability if you used a different target directory entirely for the instrumented binaries?

[sksamuel]

not sure I can because I'd have to update all the keys in sbt to point there. I do point to scoverage-classes rather than just classes though.

Does this error only happen with scoverage?

[sksamuel]

Doesn't seem to be an scoverage thing.

typesafehub/sbt-multi-jvm#15

[sksamuel]

Well we write to scoverage-classes instead of classes
On 5 Sep 2014 17:59, "Sam Halliday" [email protected] wrote:

@sksamuel https://github.com/sksamuel would it help the stability if
you used a different target directory entirely for the instrumented
binaries?

—
Reply to this email directly or view it on GitHub
#47 (comment)
.

[RichardBradley]

Doesn't seem to be an scoverage thing.

typesafehub/sbt-multi-jvm#15

It's not directly an scoverage bug, but only plugins that do compilation in parallel tickle this SBT bug -- notice that the linked bug is an issue in sbt-multi-jvm (does multiple compilations in parallel), not in core sbt.

I'll raise a core SBT issue to see if they can help diagnose.

[RichardBradley]

(Hmm - not sure whether to link to this "sbt is shit" thread from my new SBT bug ;-)

[sksamuel]

Hahah
On 14 Oct 2014 11:50, "Richard Bradley" [email protected] wrote:

(Hmm - not sure whether to link to this "sbt is shit" thread from my new
SBT bug ;-)

—
Reply to this email directly or view it on GitHub
#47 (comment)
.

[fommil]

heh, oh they know my thoughts :-)

[sksamuel]

Maybe in the meantime, Scoverage should just disable parallel compilation?

[RichardBradley]

That would be quite helpful. Do you know how? I tried to figure out how to disable parallel tasks for scoverage builds, but leave them on for normal builds, but gave up and disabled it for all my builds (noticeably slower :( )

[sksamuel]

Not off top of my head but perhaps setting it directly in the plugin for
the ScoverageTest scope will do it.
On 15 Oct 2014 11:08, "Richard Bradley" [email protected] wrote:

That would be quite helpful. Do you know how? I tried to figure out how to
disable parallel tasks for scoverage builds, but leave them on for normal
builds, but gave up and disabled it for all my builds (noticeably slower :(
)

—
Reply to this email directly or view it on GitHub
#47 (comment)
.

[RichardBradley]

Did that commit fix the issue for you?
ISTR trying that change and finding that it didn't make any difference, as the parallel stuff was happening in a different scope ("compile" or something).

[sksamuel]

Well it worked when I ran the tests a few times, since this error barely
happens for me anyway, 1 in 100 say, it's hard to confirm for sure.

On 5 November 2014 22:16, Richard Bradley [email protected] wrote:

Did that commit fix the issue for you?
ISTR trying that change and finding that it didn't make any difference, as
the parallel stuff was happening in a different scope ("compile" or
something).

—
Reply to this email directly or view it on GitHub
#47 (comment)
.