Temp disk space usage

[Iristyle]

Bower doesn't seem to clean up after itself very well.

In addition to the space required for the cache, which is understandable, I just noticed that there are many stray copies of packages in the temp directory

On our build server, this amounted to about 10GB and was enough to use up all the disk space, preventing jobs from executing (we have a large disk for Jenkins jobs, but temp is on the same volume as our smaller OS disk).

For now, I'm going to setup a job to purge this stuff, but I think that Bower should trash whatever stuff it might use temporarily.

[necolas]

Out of interest, what are you using Bower for on your build server?

[Iristyle]

https://github.com/yatskevich/grunt-bower-task

[necolas]

But why aren't you checking the Bower packages into source control?

[Iristyle]

I do a lot of build engineering work w/ Jenkins, and I'm firmly in the camp of never checking packages / dependencies into source control. I always lock versions and restore using a package manager, whether it's NPM / npm-shrinkwrap.json, Bundler / Gemfile.lock, Nuget / packages.config, Bower / component.json, etc, etc. It keeps repo sizes down (admittedly a bigger concern with binary deps) amongst other advantages.

Regardless of which camp you fall into, Bower should still clean up its temp files 😺 I had the other devs on the team scope out their temp directories and they all had the same issue -- anywhere between 500MB and 5 GB of temp junk hanging around from Bower 😲

[necolas]

Yes, it should definitely clean up. We also strongly recommend that you check-in components installed by Bower (for similar reasons to those outlined here: http://www.futurealoof.com/posts/nodemodules-in-git.html).

[satazor]

@Iristyle bower uses tmp package that has an option to cleanup all the created temporary directories (https://github.com/raszi/node-tmp#graceful-cleanup). Bower enables that but it seems it's not working sometimes?

Still, to prevent future issues, the folders could be manually cleaned with a give probability (e.g.: each 5% of the times bower is run, it would kick off the manual cleanup of the temp directory).

[sindresorhus]

If it's a bug with the tmp module I would rather see it reported and fixed there rather than us working around it.

[wibblymat]

@sindresorhus @satazor I think that the graceful cleanup option only removes files created by the module - so it will only remove a temp directory if it is empty.

https://github.com/raszi/node-tmp#directory-creation:

If the directory still contains items on process exit, then it won't be removed.

[satazor]

Oh, I see. Looks strange to me that node-tmp has this behaviour enabled by default and do not have an option for this.

@raszi Do you think it would be good to add an option to delete the created tmp dirs even if they are not empty (globally or per created dir?)

[raszi]

@satazor It's not that good idea but I can do that and I even have a pull request related to this. So I'll implement this unsafe delete option and release the new version in a few days.

[raszi]

@satazor and others: I've released the new version which supports the recursive delete option.

If you create a directory with enabled unsafeCleanup option then it will remove it recursively even though the files inside were not created by the tmp library itself.

[Iristyle]

@necolas yup, seen that (dated) post from Mikeal before on committing packages -- I don't agree ;0 Note that it was also written before shrinkwrap existed in NPM (1.1.2).

@satazor thanks for checking this out!

[Iristyle]

@satazor just had a chance to check this out ... it's not quite 100%. All of the guid based directories disappear, and most, but not all of the bower- prefixed directories disappear.

I flushed out temp, then ran my build to watch would happen.

I'm going to guess that this might have something to do with package dependencies. My Bower output includes

Please note that
    ClientSide Dependencies requires jquery 1.8.3
    angular-strap requires jquery >= 1.8.0
    bootstrap-datepicker requires jquery >=1.7.1
    bootstrap-timepicker requires jquery >=1.8.3
    angular-ui-select2 requires jquery >=1.6.4
    jquery.cookie requires jquery >=1.0
    bootstrap requires jquery ~1.8.0
    bootstrap requires jquery ~1.8.0
    bootstrap requires jquery ~1.8.0
    bootstrap requires jquery ~1.8.0
    select2 requires jquery ~1.4.4
    select2 requires jquery ~1.4.4

Resolved to jquery v1.8.3, which matches the requirement defined in the project's component.json.
Conflicts may occur.

The packages kicking around in temp still are the following...

bower-bootstrap-17980awrtmwg
bower-bootstrap-17980ni5220g
bower-jquery-17980bxdyuvk
bower-jquery-17980feoh7ao
bower-jquery-17980gb7sb4g
bower-jquery-17980mqz62og
bower-jquery-17980ncvm628
bower-jquery-17980p7oeb0w
bower-jquery-17980pyyop5c
bower-jquery-17980zpjun4
bower-jquery-1798055ntww
bower-select2-17980tswz9b4

For ref, this is my component.json if it helps to repro locally

{
  "name": "ClientSide Dependencies",
  "version": "1.0.0",
  "main": "",
  "dependencies": {
    "AngularJS": "git+https://github.com/Iristyle/bower-angular.git#1.1.4",
    "angular-auth": "git+https://github.com/Iristyle/bower-angularAuth.git#0.0.3",
    "bwizard": "git+https://github.com/gilluminate/Bootstrap-Wizard.git",
    "jquery": "1.8.3",
    "jquery.cookie": "v1.3.1",
    "chai": "1.5.0",
    "mocha": "1.8.1",
    "bootstrap": "2.2.2",
    "modernizr": "git+https://github.com/Iristyle/bower-modernizr.git#2.6.2",
    "requirejs": "2.1.6",
    "lawnchair": "0.6.3",
    "lodash":"v1.2.1",
    "toastr": "1.1.5",
    "font-awesome": "3.1.0",
    "angular-strap": "0.7.1",
    "angular-placeholders": "git+https://github.com/Iristyle/bower-angular-placeholders.git#0.0.1a",
    "requirejs-glob": "0.1.2",
    "select2": "3.3.2",
    "angular-ui-select2": "latest"
  }
}