diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index f4f61b9..67fb7db 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -18,5 +18,5 @@ jobs:
run: |
curl -sL https://git.io/fisher | source
fisher install $GITHUB_WORKSPACE
- fishtape test/*.fish
+ fishtape test/*
shell: fish {0}
diff --git a/README.md b/README.md
index 5545ee4..3753514 100644
--- a/README.md
+++ b/README.md
@@ -1,8 +1,37 @@
# Fishtape
-> TAP-based testing for [Fish](https://fishshell.com).
+> 100% _pure_-[Fish](https://fishshell.com) test runner.
-Fishtape is a Test-Anything-Protocol test runner. Because each test runs concurrently in its own sub-shell, you can define functions, set variables, and modify the executing environment without hijacking the current session or other tests. What learning curve? If you know how to use [`test`](https://fishshell.com/docs/current/commands.html#test), you can start using Fishtape now.
+Fishtape is a Test Anything Protocol compliant test runner for Fish. Use it to test anything: scripts, functions, plugins without ever leaving your favorite shell. Here's the first example to get you started:
+
+```fish
+@test "the ultimate question" (math "6 * 7") -eq 42
+
+@test "got root?" $USER = root
+```
+
+Now put that in a `fish` file and run it with `fishtape` installed. Behold, the TAP stream!
+
+```console
+$ fishtape example.fish
+TAP version 13
+ok 1 the ultimate question
+not ok 2 got root?
+ ---
+ operator: =
+ expected: root
+ actual: jb
+ at: ~/example.fish:3
+ ...
+
+1..2
+# pass 1
+# fail 1
+```
+
+> See [reporting options](#reporting-options) for alternatives to TAP output.
+
+Each test file runs inside its own shell, so you can modify the global environment without cluttering your session or breaking other tests. If all the tests pass, `fishtape` exits with `0` or `1` otherwise.
## Installation
@@ -12,85 +41,62 @@ Install with [Fisher](https://github.com/jorgebucaran/fisher):
fisher install jorgebucaran/fishtape
```
-## Getting Started
+## Writing Tests
-A test file is a regular `fish` file with `@test` declarations. A test declaration (or test case) consists of a description, followed by one or more operators and their arguments. You can use any operator supported by the [`test`](https://fishshell.com/docs/current/commands.html#test) builtin except for the `-a` and `-o` conditional operators.
+Tests are defined with the `@test` function. Each test begins with a description, followed by a typical `test` expression. Refer to the `test` builtin [documentation](https://fishshell.com/docs/current/cmds/test.html) for operators and usage details.
-```fish
-@test "math is real" (math 41 + 1) -eq 42
+> Operators to combine expressions are not currently supported: `!`, `-a`, `-o`.
-@test "basename is fish" (
- string split -rm1 / /usr/local/bin/fish
-)[-1] = "fish"
-
-@test "test is a builtin" (
- contains -- test (builtin -n)
-) $status -eq 0
-
-@test "print a sequence of numbers" (seq 3) = "1 2 3"
+```fish
+@test "has a config.fish file" -e ~/.config/fish/config.fish
```
-Run `fishtape` with one or more test files to run your tests.
+Sometimes you need to test the exit status of running one or more commands and for that, you use command substitutions. Just make sure to suppress stdout to avoid cluttering your `test` expression.
-```console
-fishtape tests/*.fish
-TAP version 13
-ok 1 math is real
-ok 2 basename is fish
-ok 3 test is a builtin
-ok 4 print a sequence of numbers
-
-1..4
-# pass 4
-# ok
+```fish
+@test "repo is clean" (git diff-index --quiet @) $status -eq 0
```
-Test files run in the background in a subshell while individual test cases run sequentially. The output is buffered (delivered in batches) until all jobs are complete. If all the tests pass, `fishtape` exits with status `0`—else, it exits with status `1`.
-
-A buffered output means we can't write to stdout or stderr without running into race conditions. To print a TAP message along with a batch of test results, use the `@mesg` declaration.
+Often you have work that needs to happen before and after tests run like preparing the environment and cleaning up after you're done. The best way to do this is directly in your test file.
```fish
-@mesg "Brought to you by the friendly interactive shell."
-```
+set temp (mktemp -d)
-### Setup and Teardown
+cd $temp
-You can define special `setup` and `teardown` functions, which run before and after each test case, respectively. Use them to load fixtures, set up your environment, and clean up when you're done.
+@test "a regular file" (touch file) -f file
+@test "nothing to see here" -z (read < file)
-```fish
-function setup
- set -g tmp (command mktemp -d /tmp/foo.XXXXX)
- command mkdir -p $tmp
-end
-
-function teardown
- command rm -rf $tmp
-end
-
-@test "directory is empty" -z (
- pushd $tmp
- command ls -1
- popd
-)
+rm -rf $temp
```
-### Special Variables
+When comparing multiline output you usually have two options, collapse newlines using `echo` or collect your input into a single argument with [`string collect`](https://fishshell.com/docs/current/cmds/string-collect.html). It's your call.
-The following variables are globally available for all test files:
+```fish
+@test "first six evens" (echo (seq 2 2 12)) = "2 4 6 8 10 12"
-- `$current_dirname` is the directory where the currently running test file is located.
-- `$current_filename` is the name and extension of the currently running test file.
+@test "one two three" (seq 3 | string collect) = "1
+2
+3"
+```
-## Reporting Options
+If you want to write to stdout while tests are running, use the `@echo` function. It's equivalent to `echo "# $argv"`, which prints a TAP comment.
-TAP is a simple text-based protocol for reporting test results. It's easy to parse for machines and still readable for humans. If you are looking for reporting alternatives, see [this list of reporters](https://github.com/substack/tape#pretty-reporters) or try [tap-mocha-reporter](https://github.com/tapjs/tap-mocha-reporter) for an all-in-one solution.
+```fish
+@echo -- strings --
+```
-Once you've downloaded a TAP-compliant reporter and put it somewhere in your `$PATH`, pipe `fishtape` to it.
+## Reporting Options
-> Redirections and pipes involving blocks are run serially in fish (see [fish-shell/#1396](https://github.com/fish-shell/fish-shell/issues/1396)). This means we must run `fishtape` in a subshell to enable streaming support.
+If you're looking for something fancier than plaintext, [here's a list](https://github.com/sindresorhus/awesome-tap#reporters) of reporters that you can pipe TAP into.
-```fish
-fish -c "fishtape test/*.fish" | tap-nyan
+```console
+$ fishtape test/* | tnyan
+ 30 -_-_-_-_-_-_-_-_-_-_-_-_-_-_-__,------,
+ 0 -_-_-_-_-_-_-_-_-_-_-_-_-_-_-__| /\_/\
+ 0 -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_~|_( ^ .^)
+ -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_ "" ""
+ Pass!
```
## License
diff --git a/completions/fishtape.fish b/completions/fishtape.fish
index 2fd8c62..ad81efe 100644
--- a/completions/fishtape.fish
+++ b/completions/fishtape.fish
@@ -1,2 +1,2 @@
-complete --command fishtape --long version --description "Print version"
-complete --command fishtape --long help --description "Print help"
+complete --command fishtape --short v --long version --description "Print version"
+complete --command fishtape --short h --long help --description "Print help"
diff --git a/conf.d/fishtape.fish b/conf.d/fishtape.fish
deleted file mode 100644
index a51a5cd..0000000
--- a/conf.d/fishtape.fish
+++ /dev/null
@@ -1,7 +0,0 @@
-function _fishtape_uninstall --on-event fishtape_uninstall
- set --names |
- string replace --filter --regex "^fishtape_" -- "set --erase fishtape_" |
- source
- functions --erase (functions --all | string match --entire --regex -- "^_fishtape_")
- complete --erase --command fishtape
-end
diff --git a/functions/fishtape.fish b/functions/fishtape.fish
index 328ade3..9408945 100644
--- a/functions/fishtape.fish
+++ b/functions/fishtape.fish
@@ -1,173 +1,110 @@
-function fishtape -d "TAP-based test runner"
- if not isatty
- if not contains -- $argv @{test,mesg}
- set argv $argv -
- end
- end
- switch "$argv[1]"
+function fishtape --description "Test scripts, functions, and plugins in Fish"
+ switch "$argv"
+ case -v --version
+ echo "fishtape, version 3.0.0"
case "" -h --help
- echo "Usage: fishtape Run tests in files"
+ echo "Usage: fishtape Run test files"
echo "Options:"
- echo " fishtape --help Print this help message"
- echo " fishtape --version Show the current version"
- echo "Examples:"
- echo " fishtape 1 && !i && !(not && NR == 2) && /^(!?=|-(eq|ne|gt|ge|lt|le))$/) {
- a[i] = operator ? (a[i] ? a[i] " " : "") operator : a[i]
- operator = $0
- i++
- } else {
- a[i] = (a[i] ? a[i]" " : "") $0
- }
- }
- END {
- print not operator "\n" a[0] "\n" (i ? a[i]\
- : operator == "-n" ? "a non-zero length string"\
- : operator == "-z" ? "a zero length string"\
- : operator == "-b" ? "a block device"\
- : operator == "-c" ? "a character device"\
- : operator == "-d" ? "a directory"\
- : operator == "-e" ? "an existing file"\
- : operator == "-f" ? "a regular file"\
- : operator == "-g" ? "a file with the set-group-ID bit set"\
- : operator == "-G" ? "a file with same group ID as the current user"\
- : operator == "-L" ? "a symbolic link"\
- : operator == "-O" ? "a file owned by the current user"\
- : operator == "-p" ? "a named pipe"\
- : operator == "-r" ? "a file marked as readable"\
- : operator == "-s" ? "a file of size greater than zero"\
- : operator == "-S" ? "a socket"\
- : operator == "-t" ? "a terminal tty file descriptor"\
- : operator == "-u" ? "a file with the set-user-ID bit set"\
- : operator == "-w" ? "a file marked as writable"\
- : operator == "-x" ? "a file marked as executable"\
- : "a valid operator") "\n"\
- (not ? not "\n" : "") (i ? a[0] "\n" operator "\n" a[1] : operator "\n" a[0])
- }
- ')
- if test $rest[4..-1] 2>/dev/null
- echo -s {$argv[2],1,$argv[3]}\t
- else
- echo -s {$argv[2],0,$argv[3],$rest[1],$rest[2],$rest[3]}\t
- end
- functions -q teardown; and teardown
- else
- echo -s {$argv[2],todo,$argv[3]\ \#\ TODO}\t
- end
+ echo " -v or --version Print version"
+ echo " -h or --help Print this help message"
case \*
- set -l files (printf "%s\n" $argv | command awk -v PWD="$PWD" '
- /^-$/ {
- print; next
- }
- {
- n = split((/^\// ? "" : PWD "/") $0, tree, "/")
- for (i = k = 0; ++i <= n; ) {
- node = tree[i]
- k += (_k = node == ".." ? k ? -1 : 0 : node && node != "." ? 1 : 0)
- path[k] = _k > 0 ? node : path[k]
- }
- out = ""
- for (i = 1; i <= k || !out; i++) {
- out = out "/" path[i]
- }
- print out
- }
- ')
+ set --local files (realpath $argv)
for file in $files
- if test $file != - -a ! -f $file
- echo "fishtape: can't open file \"$file\" -- is this a valid file?"
+ if test ! -f $file
+ echo "fishtape: Invalid file or file not found: \"$file\"" >&2
return 1
end
end
- set -l tmp (random)
-
- command awk '
- FNR == 1 {
- print (NR > 1 ? end_batch() ";" : "") begin_batch()
- id++
- }
- !/^[ \t\r\n\f\v]*#/ && $0 {
- gsub(/\'/, "\\\\\'")
- gsub(/\$current_dirname/, d[split(FILENAME, d, /\/[^\/]*$/) - 1])
- gsub(/\$current_filename/, f[split(FILENAME, f, "/")])
- sub(/^[ \t\r\n\f\v]*@mesg/, "fishtape @mesg " id)
- sub(/^[ \t\r\n\f\v]*@test/, "functions -q setup; and setup; true; fishtape @test " id)
- print
- }
- END {
- print end_batch() ";while for j in $jobs;contains -- $j " jobs() ";and break;end;end"
- }
- function begin_batch() {
- return "fish -c \'"
- }
- function end_batch() {
- return "echo " id "\'&;set -l jobs $jobs " jobs(" -l")
- }
- function jobs(opt) {
- return "(jobs" opt " | command awk \'/^[0-9]+\\\t/ { print $1 }\')"
- }
- ' $files >@fishtape$tmp
-
- fish @fishtape$tmp | command awk -F\t '
- BEGIN {
- print "TAP version 13"
- }
- NF == 1 {
- for (i = 0; i < count[$1]; i++) {
- print\
- (mesg = batch[$1 i "mesg"])\
- ? mesg : sub(/ok/, "ok " (++total), batch[$1 i])\
- ? batch[$1 i] ((error = batch[$1 i "error"]) && ++failed ? "\n" error : "") : ""
- todo = (batch[$1 i "todo"]) ? todo + 1 : todo
- fflush()
- }
- }
- NF > 1 {
- id = $1 count[$1]++
- if ($2 == "mesg") {
- batch[id $2] = "# " $3
- } else if ($2) {
- batch[id] = batch[id $2] = "ok " $3
- } else {
- batch[id] = "not ok " $3
- batch[id "error"] =\
- " ---\n"\
- " operator: "$4 "\n"\
- " expected: "$6 "\n"\
- " actual: "$5 "\n"\
- " ..."
- }
- }
- END {
- print "\n1.." total
- print "# pass " (total - failed - todo)
-
- if (failed) print "# fail " failed
- if (todo) print "# todo " todo
- else if (!failed) print "# ok"
-
- exit (failed > 0)
- }
- '
-
- set -l _status $status
- command rm -f @fishtape$tmp
- return $_status
+ set --local operators -{n,z,b,c,d,e,f,g,G,k,L,O,p,r,s,S,t,u,w,x}
+ set --local expectations \
+ "a non-zero length string" \
+ "a zero length string" \
+ "a block device" \
+ "a character device" \
+ "a directory" \
+ "an existing file" \
+ "a regular file" \
+ "a file with the set-group-ID bit set" \
+ "a file with same group ID as the current user" \
+ "a file with the sticky bit set" \
+ "a symbolic link" \
+ "a file owned by the current user" \
+ "a named pipe" \
+ "a file marked as readable" \
+ "a file of size greater than zero" \
+ "a socket" \
+ "a terminal tty file descriptor" \
+ "a file with the set-user-ID bit set" \
+ "a file marked as writable" \
+ "a file marked as executable"
+
+ set --universal _fishtape_test_number 0
+ set --universal _fishtape_test_passed 0
+ set --universal _fishtape_test_failed 0
+
+ function @echo
+ echo "# $argv"
+ end
+
+ function @test --argument-names name --inherit-variable operators --inherit-variable expectations
+ set --erase argv[1]
+ set --query argv[2] || set --append argv ""
+
+ set _fishtape_test_number (math $_fishtape_test_number + 1)
+
+ if test $argv
+ set _fishtape_test_passed (math $_fishtape_test_passed + 1)
+
+ echo "ok $_fishtape_test_number $name"
+ else
+ set _fishtape_test_failed (math $_fishtape_test_failed + 1)
+
+ status print-stack-trace |
+ string replace --filter --regex -- "\s+called on line (\d+) of file (.+)" '$2:$1' |
+ read --local at
+
+ if set --query argv[3]
+ set operator $argv[2]
+ set expected (string escape -- $argv[3])
+ set actual (string escape -- $argv[1])
+ else
+ set operator $argv[1]
+ set expected $expectations[(contains --index -- $operator $operators)]
+ set actual (string escape -- $argv[2])
+ end
+
+ echo "not ok $_fishtape_test_number $name"
+ echo " ---"
+ echo " operator: $operator"
+ echo " expected: $expected"
+ echo " actual: $actual"
+ echo " at: $at"
+ echo " ..."
+ end
+ end
+
+ echo TAP version 13
+
+ for file in $files
+ fish --init-command=(functions @echo | string collect) --init-command=(functions @test | string collect) $file
+ end
+
+ echo
+ echo "1..$_fishtape_test_number"
+ echo "# pass $_fishtape_test_passed"
+ test $_fishtape_test_failed -eq 0 &&
+ echo "# ok" ||
+ echo "# fail $_fishtape_test_failed"
+
+ functions --erase @echo @test
+
+ set --local failed $_fishtape_test_failed
+ set --erase _fishtape_test_number
+ set --erase _fishtape_test_passed
+ set --erase _fishtape_test_failed
+
+ test $failed -eq 0
end
end
diff --git a/test/conditional.fish b/test/conditional.fish
deleted file mode 100644
index 72d94ec..0000000
--- a/test/conditional.fish
+++ /dev/null
@@ -1,8 +0,0 @@
-@mesg $current_filename
-
-if true
- @test "true" $status -eq 0
-end
-if false
- @test "never runs" $status -eq 1
-end
diff --git a/test/files.fish b/test/files.fish
index 49af991..f9cfd52 100644
--- a/test/files.fish
+++ b/test/files.fish
@@ -1,15 +1,11 @@
-@mesg $current_filename
+@echo === files ===
-function setup
- set -g tmp_dir (command mktemp -d /tmp/foo.XXXX)
-end
+set temp (command mktemp -d)
-function teardown
- command rm -rf $tmp_dir
-end
+builtin cd $temp
-@test "file exists" (touch $tmp_dir/file) -e $tmp_dir/file
-@test "file is regular" (touch $tmp_dir/file) -f $tmp_dir/file
-@test "file is empty" (touch $tmp_dir/file) ! -s $tmp_dir/file
-@test "file is non-empty" (echo foo > $tmp_dir/file) -s $tmp_dir/file
-@test "file is a directory" -d $tmp_dir
+@test "a directory" -d $temp
+@test "a regular file" (command touch file) -f file
+@test "nothing to see here" -z (read