Add support for .lintignore.

Lint-ing all of Weave Net is currently not desirable because:

* it would create conflicts on many branches,
* it would pollute the git history,
* etc.

and because `lint` currently lints all files except `./vendor`, this breaks Weave Net's build.
As a result, Weave Net is not using the latest commit in `build-tools`'s `master` branch, which is also source of problems:

* it limits standardisation of projects across Weaveworks,
* it limits the ability to use the latest features of `build-tools`,
* it complexifies merging changes in `build-tools` (e.g. #53 and weaveworks/weave#2694 for weaveworks/weave#2647),
* etc.

This change adds the ability to optionally process a `.lintignore` file, which can contain any number of `GLOB`s then used to filter out files one does not want to lint, a-la `.gitignore`.
This, coupled with a `.lintignore` file in Weave Net will then enable Weave Net to use the latest version of `build-tools`.

lint

@@ -17,6 +17,7 @@
 
 set -e
 
+DEBUG_DIR="${DEBUG_DIR:-/tmp}"
 IGNORE_LINT_COMMENT=
 IGNORE_SPELLINGS=
 while true; do
@@ -171,6 +172,30 @@ lint_files() {
     exit $lint_result
 }
 
+filter_out() {
+    local globs="$1"
+    if [ -n "$globs" -a -r "$globs" ]; then
+        # Files to debug the regular expressions, as they get quite complex:
+        local files_in="$(mktemp $DEBUG_DIR/lint_files_in_XXX.txt)"
+        local files_out="$(mktemp $DEBUG_DIR/lint_files_out_XXX.txt)"
+        # Globs blacklist found: filter out stdin before passing it to stdout:
+        while read -r filename; do
+            # Remove comments. Remove empty lines. Expand directories (e.g. "bin/" -> "bin/.*"). And then regexify globs.
+            # These regexes are then passed to grep to filter out matching elements from stdin:
+            echo "${filename}" | tee -a "$files_in" | grep -vEf <(sed '/^#.*$/d ; /^\s*$/d ; s/^\(.*\/\)$/\1*/g ; s/\([.|]\)/\\\1/g ; s/\?/./g ; s/\*/.*/g ; s/\(.*\)/\^\1\$/g' "$globs") | tee -a "$files_out"
+        done
+        # List all files we have just filtered out:
+        local files_filtered="$(mktemp $DEBUG_DIR/lint_files_filtered_out_XXX.txt)"
+        grep -xvFf "$files_out" "$files_in" > "$files_filtered"
+        rm -f "$files_in" "$files_out"
+        [ -n "$DEBUG" ] && >&2 echo "Files not linted:" && >&2 cat "$files_filtered"
+        rm -f "$files_filtered"
+    else
+        # No globs blacklist was provided: simply propagate stdin to stdout:
+        while read -r filename; do echo "${filename}"; done
+    fi
+}
+
 list_files() {
     if [ $# -gt 0 ]; then
         git ls-files --exclude-standard | grep -vE '(^|/)vendor/'
@@ -179,4 +204,4 @@ list_files() {
     fi
 }
 
-list_files "$@" | lint_files
+list_files "$@" | filter_out .lintignore | lint_files