AssertionError

Rough draft proposal for an AssertionError exception thrown to indicate the failure of an assertion has occurred.

Status

This proposal has not yet been introduced to TC39.

Authors

• @DerekNonGeneric (Derek Lewis, AMPHTML / OpenINF)
• Spec. Guest(s):
  • @boneskull (Chris Hiller, Mocha)

Champion Group

• Invited Expert(s):
  • TBD
• Delegate(s):
  • TBD

Motivation

Today, a very common design pattern (especially in assertion library code) is to throw an AssertionError for any failed assertions.

In particular, the following popular assertion libraries do this:

• Chai
• Expect
• Node.js core assert module
• See Related

Ideally, a standard AssertionError from any assertion library would be used by the error reporters of test frameworks and runtime error loggers to display the difference between what was expected and what the assertion saw for providing rich error reporting such as displaying a pretty printed diff.

However, due to the various assertion libraries using disparate AssertionError classes, with each providing varying degrees of contextual information, a standard API for assertion error message pretty printing does not yet exist.

This proposal introduces a 7th standard error type to the language to provide a consistent API necessary to enable assertion library-agnostic error handlers with the contextual information necessary for rich error reporting.

There are several existing modules with similar ideas of how the interface for this error is expected to look:

• npm: assertion-error
• npm: assertion-error-diff
• See Related

Proposal

Today, it is very common for unit testing frameworks to have assertion methods like assertNull write code like:

// file: check.mjs
export function check(actual) {
  return {
    is: function (expect, message) {
      if (actual !== expect)
        throw new AssertionError({
          actual: actual,
          expected: expect,
          message: message,
        });
    },
  };
}

export default check;

This would be used as follows:

import check from './check.mjs';

export function assertNull(value) {
  check(value).is(null, `${value} is not null`);
}

export default assertNull;

The AssertionError class is not only for pretty printing error messages. It plays a key role in applying “Design by Contract”, which often means performing runtime assertions.

An assertion specifies that a program satisfies certain conditions at particular points in its execution. There are three types of assertion:

• Preconditions: Specify conditions at the start of a function.

• Postconditions: Specify conditions at the end of a function.

• Invariants: Specify conditions over a defined region of a program.

—https://ptolemy.berkeley.edu/~johnr/tutorials/assertions.html

The examples below demonstrate “Postconditions”.

assert.species = (pokemon, species, message) => {
  const actual = pokemon.template.species;
  if (actual === species) return;
  throw new AssertionError({
    actual,
    expected: species,
    message:
      message || `Expected ${pokemon} species to be ${species}, not ${actual}.`,
    stackStartFunction: assert.species,
  });
};

However, none of the properties of the options object would be mandatory.

assert.fullHP = function (pokemon, message) {
  if (pokemon.hp === pokemon.maxhp) return;
  throw new AssertionError({
    message:
      message ||
      `Expected ${pokemon} to be fully healed, not at ${pokemon.hp}/${pokemon.maxhp}.`,
    stackStartFunction: assert.fullHP,
  });
};

The more contextual information, the richer the error reports. So, if we had an operator property…

assert.heals = (pokemon, fn, message) => {
  const prevHP = pokemon.hp;
  fn();
  if (pokemon.hp > prevHP) return;
  throw new AssertionError({
    actual: pokemon.hp,
    expected: `${prevHP}`,
    operator: '>',
    message: message || `Expected ${pokemon} to be healed.`,
    stackStartFunction: assert.heals,
  });
};

… it would enable us to construct highly descriptive reports that include comparison result data.

Attribution

Inspiration

Some notable prior art helping drive this proposal.

Node.js Built-in Class: assert.AssertionError

• Extends: {errors.Error}

Indicates the failure of an assertion. All errors thrown by the assert module will be instances of the AssertionError class.

new assert.AssertionError(options)

A subclass of Error that indicates the failure of an assertion.

• options {Object}
  • message {string} If provided, the error message is set to this value.
  • actual {any} The actual property on the error instance.
  • expected {any} The expected property on the error instance.
  • operator {string} The operator property on the error instance.
  • stackStartFn {Function} If provided, the generated stack trace omits frames before this function.

Ecosystem uses

Custom assert modules

To make it possible to combine assertions from different modules in one test suite, all assert methods should throw an AssertionError that has properties for actual and expected an common API for error message pretty printing.

—http://wiki.commonjs.org/wiki/Unit_Testing/1.0#Custom_Assert_Modules

Test & validation frameworks

Mocha supports the err.expected and err.actual properties of any thrown AssertionErrors from an assertion library. Mocha will attempt to display the difference between what was expected, and what the assertion actually saw.

—https://github.com/mochajs/mocha/blob/HEAD/docs/index.md#diffs

Runtime feature detection errors

An expected error is identified by the .expected property on the Error object being true. You can use the Log.prototype.expectedError method to create an error that is marked as expected.

—https://github.com/ampproject/amphtml/blob/main/docs/spec/amp-errors.md#expected-errors

Options

All instances of AssertionError would contain the built-in Error properties (message and name) and perhaps any of the new properties in common use below.

Support for new properties

	actual	expected	operator	messagePattern	generateMessage()	generatedMessage	diffable	showDiff	toJson()	stack	stackStartFn()	stackStartFunction()	code	details	truncate	previous	negate	_message	assertion	fixedSource	improperUsage	actualStack	raw	statements	savedError
AVA			×																×	×	×	×	×	×	×
Chai	×	×	×					×	×
Closure Library				×
Deno	×	×	×			×				×	×	×	×	×
Jest	×	×	×
Mocha	×	×						×					x
Mozilla Assert.jsm	×	×	×												×
Must.js	×	×	×				×	×		×
Node.js Assert	×	×	×			×				×	×	×	×	×
Should.js	×	×	×		×	×				×	×	×		×		×	×	×
WPT										×

Note: Error.prototype.stack is not supported in all browsers/versions.

Descriptions of custom properties

Property	Type	Meaning
actual	unknown	Set to the actual argument for methods such as assert.strictEqual().
callsite	Function	Location where the assertion happened.
code	string	Value is always ERR_ASSERTION to show that the error is an assertion error.
details	Object	The context data necessary in a single object.
diffable	boolean	Whether it makes sense to compare objects granularly or even show a diff view of the objects involved.
expected	unknown	Set to the expected value for methods such as assert.strictEqual().
generatedMessage	boolean	Indicates if the message was auto-generated (true) or not.
message	string?	Message describing the assertion error.
messagePattern	unknown	The message pattern used to format the error message. Error handlers can use this to uniquely identify the assertion.
operator	string	Set to the passed in operator value.
showDiff	boolean	Same as diffable. Used by mocha; whether to do string or object diffs based on actual/expected.
stack	unknown	The stack trace at the point where this error was first thrown.
stackStartFn	Function	If provided, the generated stack trace omits frames before this function.
stackStartFunction	Function	Legacy name for stackStartFn in Node.js also in Deno.
toJSON()	Function	Allow errors to be converted to JSON for static transfer.
truncate	boolean	Whether or not actual and expected should be truncated when printing.

Implementations in other languages

• assert.AssertionError has been one of Node's core modules since 2009 https://nodejs.org/dist/latest-v15.x/docs/api/assert.html#assert_class_assert_assertionerror

• AssertionError has been one of Python's Standard Exception Classes since Python 1.5. https://www.python.org/doc/essays/stdexceptions/

• AssertionError in Java also accepts a cause https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/AssertionError.html

• AssertionError in PHP simply extends Error https://www.php.net/manual/en/class.assertionerror.php

• AssertionError in Julia (since v0.5.0)

  • optionally accepts a message https://docs.julialang.org/en/v1/base/base/#Core.AssertionError
  • is thrown from @assert macros

Related

Good amount of implementations in JS existence today.

Predicates

• fn: isAssertionError in Jest
• fn: isAssertionError in assertion-error-diff

Modules

• npm: assertion-error
• npm: assertion-error-diff
• node: assert.AssertionError
• npm: better-assert
• npm: callsite

Formatters

• fn: console.assert() in the console Web API
• fn: formatNodeAssertErrors() in jest-circus

Q&A

Should this exception be thrown when an import assertion fails?

The simple answer is that it would be the most appropriate error type to throw.

Further reading…

Why not custom subclasses of Error

While there are lots of ways to achieve the behavior of the proposal, if the various AssertionError properties are explicitly defined by the language, debug tooling, error reporters, and other AssertionError consumers can reliably use this info rather than contracting with developers to construct an error properly.

Notes

• Implement ES stage-3 proposal: error-cause