-
Notifications
You must be signed in to change notification settings - Fork 1
/
dejagnu-report-card.1
146 lines (146 loc) · 5.43 KB
/
dejagnu-report-card.1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
.\" Copyright (C) 2018 Free Software Foundation, Inc.
.\" You may distribute this file under the terms of the GNU Free
.\" Documentation License.
.Dd December 31, 2018
.Os GNU
.Dt DEJAGNU-REPORT-CARD 1 URM
.Sh NAME
.Nm dejagnu\ report\ card
.Nd summarize results from testing multiple tools
.Sh SYNOPSIS
.Nm dejagnu\ report\ card
.Oo Ao Ar option Ac \*(Ba Ao Ar tool Ac \*(Ba Ao Ar file Ac Oc ...
.Sh DESCRIPTION
The
.Nm
command displays results from testing multiple tools in a tabular format.
The produced table lists, for each tool (and if multiple passes were run,
each pass) the number of tests passed, failed, unsupported, unresolved, and
untested. Tests that are expected to fail are counted in separate columns
from tests expected to pass, but "known" failures and "expected" failures
are summarized together. If a test generated warnings or errors, a tag
.Ql !W!
or
.Ql !E!
is appended at the end of the relevant line.
.Pp
Aside from options, the argument list may include tool or file names. The
.Nm
command prefers to read DejaGnu summary files and will translate names accordingly:
.Bl -tag -width ".Pa *.sum"
.It Pa *.sum
Used as-is.
.It Pa *.log
Rewritten to
.Pa *.sum
with the same stem.
.It Pa *.
The string
.Pa sum
is appended to select a summary file. This processing is done for
convenience when using Readline file name completion in a shell, which will
complete to the dot.
.It Pa *
Taken as a tool name;
.Pa .sum
is appended.
.El
.Sh OPTIONS
.Bl -tag -width ".Fl v , -verbose"
.It Fl v , -verbose
Emit additional output describing the operation of
.Nm
itself.
.El
.Sh FILES
The
.Nm
command produces its output by reading the summary files produced by
DejaGnu and counting "PASS", "FAIL", etc.
.Pp
If no names are given as arguments, all files matching
.Pa *.sum
in the current directory are read.
.Sh EXAMPLES
.Ss A simple example from DejaGnu's own testsuite
.Bd -literal
$ dejagnu report card
\ __________________________________________________
\ / PASS FAIL ?PASS ?FAIL UNSUP UNRES UNTEST
\ |--------------------------------------------------
\ launcher | 52 0 0 0 0 0 0
libdejagnu | 5 0 0 0 0 0 0
\ runtest | 135 0 0 0 0 0 0
\ |--------------------------------------------------
\ | 192 0 0 0 0 0 0
\ \\__________________________________________________
.Ed
.Pp
Three tools were tested, with a total of 192 tests, all expected to pass.
In this example, all tests did pass, so all other columns are zero. The
.Ql ?PASS
and
.Ql ?FAIL
columns count tests known or expected to fail that either unexpectedly
passed or failed as expected. The remaining three columns count the
exceptional results for unsupported tests, unresolved tests and stub tests
that simply declare themselves untested.
.Pp
.ne 16v
.Ss The same example after tests were added for dejagnu-report-card
.Bd -literal
$ dejagnu report-card
\ __________________________________________________
\ / PASS FAIL ?PASS ?FAIL UNSUP UNRES UNTEST
\ |--------------------------------------------------
\ launcher | 52 0 0 0 0 0 0
\ libdejagnu | 5 0 0 0 0 0 0
report-card / awk | 36 0 0 0 0 0 0
report-card / sh | 36 0 0 0 0 0 0
report-card / tcl | 36 0 0 0 0 0 0
\ runtest | 135 0 0 0 0 0 0
\ |--------------------------------------------------
\ awk | 36 0 0 0 0 0 0
\ sh | 36 0 0 0 0 0 0
\ tcl | 36 0 0 0 0 0 0
\ |--------------------------------------------------
\ | 300 0 0 0 0 0 0
\ \\__________________________________________________
.Ed
.Pp
The
.Ql report-card
tool has been added, with three passes, one for each implementation. (The
shell and Tcl implementations were later dropped to reduce future
maintenance burden.) As before, all tests passed as expected. The
interesting difference from the previous example is the use of DejaGnu's
multipass testing feature and the additional per-pass summary lines added.
For this example, only the
.Ql report-card
tool uses multipass testing, so each pass total is simply the count of
tests for
.Ql report-card
instead of a distinct total.
.Pp
Also note that the command used to invoke
.Nm
is slightly different here. The
.Xr dejagnu 1
launcher will also accept multiple words joined with dashes into a single
argument. This allows individual words in a command name to be separated
with either dashes or spaces on the command line interchangeably.
.Sh SEE ALSO
.Xr dejagnu 1
.Xr runtest 1
.Pp
The full documentation for DejaGnu is maintained as a Texinfo manual. If the
.Nm info
program is properly installed at your site, the command
.Li info dejagnu
should give you access to the complete manual.
.Sh AUTHORS
.An Jacob Bachmeyer
.\".Sh BUGS
.\" LocalWords: Dt dejagnu URM Nm Ao Oo Oc DejaGnu Xr runtest DejaGnu's Bd Ql
.\" LocalWords: testsuite UNSUP UNRES UNTEST libdejagnu Readline Ss tcl awk
.\" LocalWords: ne multipass