Bash commons (bash_commons)

Library of bash functions organized in several thematic collections; each is one source file.

How to:

• Clone this repository
• Source the collection file which contains the desired functions

Example:

commons_path="/path/to/bash_commons"
. "$commons_path/filesystem.sh"
# from here on, all filesystem collection functions are available

It's compulsory to set up $commons_path with the absolute path of the folder with the cloned code (many collections use it to load internal dependencies; for details see the table below). $commons_path may also be set up as global variable, the easiest way to do this it to append an entry to /etc/environment, with the command

echo "commons_path=\"/path/to/bash_commons\"" >> /etc/environment

Then, sourcing a collection simply becomes:

. "$commons_path/filesystem.sh"
# from here on, all filesystem collection functions are available

The tests can be found in /tests.

Collections:

#	Name	Links				Internal dependency
1	Filesystem	Overview	Documentation	Code	Tests	#2, #7
2	String handling	Overview	Documentation	Code	Tests	#7
3	Logging	Overview	Documentation	Code	Tests	#2, #7
4	Installer tools	Overview	Documentation	Code	Tests	#1, #2, #7
5	Git handling	Overview	Documentation	Code	Tests	#1, #2, #7
6	Interaction	Overview	Documentation	Code		-
7	Helpers	Function index	Documentation	Code	Tests	-
8	Testing	Overview	Documentation	Code		#2, #7

Filesystem

The filesystem collection provides:

• wrappers for mkdir, mv/cp and rm with permission and existence checks that allow for detailed error status codes combined with a parameter which controls the functions' stdout behavior and a verbose mode with configurable message templates using runtime variable injection:
  • create_folder()
  • move_file() and move_folder()
  • copy_file() and copy_folder()
  • remove_file() and remove_folder()
• get_real_path() and the special application get_script_path() which return "clean" paths. Further path utilities include is_path_a() to check if a path fulfills certain conditions as well as the complementary get_existing_path_part() and get_new_path_part()
• is_readable() which allows to get a detailed status about a path
• is_writeable() which, on top of the classic write permission check, is able to handle permission checks for filesystem operations involving nested folders, like mkdir with the -p flag
• try_filepath_deduction() for a "if there's only one file matching, take that" logic
• load_configuration_file_value() which allows to load values from files instead of sourcing them

String handling

The string handling collections includes functions to modify strings and control certain properties as well as sed helpers:

• escape() adds backslashes to certain characters on a string provided as piped input
• trim() removes leading or trailing whitespaces
• find_substring() returns the position of the first match of a string inside of another string, if there's any
• sanitize_variable_quotes() allows to remove enclosing quotes in a string
• is_string_a() check whether a string complies to a certain type, f.ex. "absolute filepath" or "email"
• get_absolute_path() is a helper to prepend relative paths with a root directory
• get_string_bytes() and get_string_bytelength() allow to work with strings that contain non-ASCII characters
• sed helpers: get_sed_replace_expression() and get_sed_extract_expression() return "expressions" that can be directly passed to sed - the functions take care to select a collision-free separator character using find_sed_operation_separator() and escape characters with special signification using escape_sed_special_characters().
• get_random_string()

Installer tools

The installer tools module provides a helper called get_executable_status() which checks all details about an executable beyond what which does. The installer tools' handle_dependency() takes in charge the installation of packages if necessary and provides an extensive status return for a precise overview of the situation. This allows an installer to define package lists (by package manager) and every time a command is missing, this list is looked up and it attempts to install the package(s) listed. The handler's behavior may be further customized with callback functions and configurable status message templates.

Git handling

The git handling module provides execute_git_command_in_repository() to execute git commands in a repository context and get_git_repository_remote_url() to get details about a local repository. get_git_repository() is a git clone wrapper with detailed error status codes combined with a parameter which controls the functions' stdout behavior and a verbose mode with configurable message templates using runtime variable injection.

Interaction

The interaction collection provides the basic building blocks for interactive scripts, f.ex. installers: get_user_confirmation() for yes/no type questions, get_user_choice() for multiple choice questions. Both use read_and_validate() internally, which is simply the combination of a read and a regex check.

Logging

The logging collection's functions work together as one module which provides several features:

• distinct output channels for stdout and file logging, each with their own logging level and message pattern
• a log message buffer which allows to use log() before the logger is configured. Applications can start logging from the very beginning with logging "disabled" - in fact, messages go into the buffer, nothing is actually logged. Once the configuration is known (usually, when the script parameters were processed - typically to handle that -v flag that enables stdout logging), the application calls launch_logging() to "replay" the buffered messages and log them (or not) according to the configuration in force when launch_logging() is called
• a utility to shorten and hide secrets before they enter the logs: prepare_secret_for_logging()

Testing

Testing is based on "sessions" which are sequences of test operations. A session begins with initialize_test_session() and ends with conclude_test_session(). Each test follows the scheme:

1. set the expected result in terms of status code and stdout content with configure_test()
2. run the command capturing these values with test()

In some cases, f.ex. if the command uses piped input, it's not possible to use test(), the command has to be run in the testing script itself (example). In this case the results shall be evaluated using check_test_results() (test() calls it internally).

For examples of test scripts, check out bash_commons' own tests.

Helpers

Used by the other modules. Function index:

• calculate()
• capture()
• conditional_exit()
• execute_working_directory_dependant_command()
• get_array_element()
• is_array_index()
• is_globbing_enabled()
• is_variable_defined()

TODO

• improve handling & clean up if the root user executes the tests, especially for filesystem functions and installer tools
• transform verbose mode for wrapper functions in filesystem collection: merge the "stdout configuration" and "external message template definition array name" parameters. The logic is:
  • if that new merged parameter is omitted or an empty string, it corresponds to stdout silent mode
  • if it's set to stderr and status, these modes are applied as before
  • if it's set to a non-empty value which is not a variable name, it uses the default message templates (= current verbose mode without external msg def param.)
  • if it's set to an array variable name, it tries to find a custom message template (a element in the array at the index corresponding to the status, = current verbose mode with external msg def param). If no template is found, it falls back to the defaults The only restriction is that an external message definition array variable can't have the names stderr or status. It's implemented this way in the git handling get_git_repository().
• extend git handling with other git operations
• extend installer tools to detect/support other package managers