Approaches for ulimit setting

[davidski]

With #319 now in play, is there a suggested way to properly adjust ulimits to useful values? Working towards using the 2.x branches of both this cookbook and ES in general, I've found my ES nodes on AWS Linux are coming up with only 30K nofiles available while older ES 1.x/coobook 0.13.x nodes have 64k courtesy of the 10-cookbook.conf file.

Mocking up a new template for /etc/security/limit.d is easy enough, but I don't want to go reinventing a bad wheel if that's been determined to be a bad practice.

[martinb3]

Hi @davidski -- we figured there are lots of Chef cookbooks out there to set ulimit, and the packaged init scripts also do it now:

https://github.com/elastic/elasticsearch/blob/master/distribution/rpm/src/main/packaging/init.d/elasticsearch#L97-L102

https://github.com/elastic/elasticsearch/blob/master/distribution/deb/src/main/packaging/init.d/elasticsearch#L156-L162

And if you'd rather use Chef, here are some cookbooks:
https://supermarket.chef.io/cookbooks/ulimit
https://supermarket.chef.io/cookbooks/ulimit2

[davidski]

Thanks for the detailed response, @martinb3! That was super helpful. In testing, I've found that AWS Linux supplies a /etc/security/limits.d/default.conf file with a nofile setting of 30,000. This appears to block the ability of the init script to override this to a higher volume. This default doesn't appear on either Centos 6.4 or Centos 7.1, so I presume this is an AWS specific tweak.

Is having this limit set to the package desired default of 65,535 on AWS Linux something you'd care for this cookbook to do? Again, it's easy enough for me to drop a file in that directory via my cookbook leveraging the elasticsearch resources, so no offense taken if that's something you'd rather not do, though it may be worth a note in the docs for folks on this platform.

[davidski]

I found the limits.d file was being dropped by an upstream cookbook and not part of the base image, so this is definitely not a problem for this cookbook. Sorry for the noise. Revoking my issue permissions for $SOME_PERIOD. 😭

[martinb3]

It's all good! I prefer noise over silence! :D