Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not write verbose logs synchronously #1167

Merged
merged 1 commit into from
Oct 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions lib/verbose/log.js
Original file line number Diff line number Diff line change
@@ -1,15 +1,23 @@
import {writeFileSync} from 'node:fs';
import {inspect} from 'node:util';
import {escapeLines} from '../arguments/escape.js';
import {defaultVerboseFunction} from './default.js';
import {applyVerboseOnLines} from './custom.js';

// Write synchronously to ensure lines are properly ordered and not interleaved with `stdout`
// This prints on stderr.
// If the subprocess prints on stdout and is using `stdout: 'inherit'`,
// there is a chance both writes will compete (introducing a race condition).
// This means their respective order is not deterministic.
// In particular, this means the verbose command lines might be after the start of the subprocess output.
// Using synchronous I/O does not solve this problem.
// However, this only seems to happen when the stdout/stderr target
// (e.g. a terminal) is being written to by many subprocesses at once, which is unlikely in real scenarios.
export const verboseLog = ({type, verboseMessage, fdNumber, verboseInfo, result}) => {
const verboseObject = getVerboseObject({type, result, verboseInfo});
const printedLines = getPrintedLines(verboseMessage, verboseObject);
const finalLines = applyVerboseOnLines(printedLines, verboseInfo, fdNumber);
writeFileSync(STDERR_FD, finalLines);
if (finalLines !== '') {
console.warn(finalLines.slice(0, -1));
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

finalLines might be an empty string, indicating nothing should be printed.
Else, it is guaranteed to end with a newline due to the logic in applyVerboseOnLines(). We strip that newline since console.warn() appends one.

}
};

const getVerboseObject = ({
Expand All @@ -35,9 +43,6 @@ const getPrintedLine = verboseObject => {
return {verboseLine, verboseObject};
};

// Unless a `verbose` function is used, print all logs on `stderr`
const STDERR_FD = 2;

// Serialize any type to a line string, for logging
export const serializeVerboseMessage = message => {
const messageString = typeof message === 'string' ? message : inspect(message);
Expand Down
14 changes: 14 additions & 0 deletions test/verbose/log.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ import test from 'ava';
import {setFixtureDirectory} from '../helpers/fixtures-directory.js';
import {foobarString} from '../helpers/input.js';
import {nestedSubprocess} from '../helpers/nested.js';
import {getNormalizedLines, getCommandLine, getCompletionLine} from '../helpers/verbose.js';
import {PARALLEL_COUNT} from '../helpers/parallel.js';

setFixtureDirectory();

Expand All @@ -25,3 +27,15 @@ const testColor = async (t, expectedResult, forceColor) => {

test('Prints with colors if supported', testColor, true, '1');
test('Prints without colors if not supported', testColor, false, '0');

test.serial('Prints lines in order when interleaved with subprocess stderr', async t => {
const results = await Promise.all(Array.from({length: PARALLEL_COUNT}, () =>
nestedSubprocess('noop-fd.js', ['2', `${foobarString}\n`], {verbose: 'full', stderr: 'inherit'}, {all: true}),
));
for (const {all} of results) {
t.deepEqual(
getNormalizedLines(all),
[getCommandLine(all), foobarString, getCompletionLine(all)],
);
}
});