block and inline are declared to TypeScript as exported, but are never exported

[darkuranium]

Marked version: 14.0.0

Describe the bug
marked.d.ts marks both block and inline as export declare const ..., implying they are available for import. However, neither symbol is actually available (in neither CJS, ESM, or UMD form).

Note marked.d.ts declares both as exported:

export declare const block: {
...
export declare const inline: {
...

But marked.esm.js (or .cjs & .umd.js files) do not in fact export it:

export { _Hooks as Hooks, _Lexer as Lexer, Marked, _Parser as Parser, _Renderer as Renderer, _TextRenderer as TextRenderer, _Tokenizer as Tokenizer, _defaults as defaults, _getDefaults as getDefaults, lexer, marked, options, parse, parseInline, parser, setOptions, use, walkTokens };

To Reproduce
Simply install [email protected], and write the following (TypeScript!) file:

// Okay as per TypeScript/tsc, but we get a runtime error: `SyntaxError: The requested module 'marked' does not provide an export named 'block'`
import { block, inline } from 'marked';
console.log(block, inline); // is *not* available

Expected behavior
The type declarations should match the actual exports --- either marked.d.ts should not declare those types as export, or (ideally) block & inline should actually be exported.

[UziTech]

Nice catch! 💯

I don't think those should be exported since they are an implementation detail and they can change without releasing a major version.

[darkuranium]

For what it's worth, the reason I need these is to be able to override every single renderer, to output a token tree instead of HTML. Can you think of a better way of doing that?

Edit: Is hooks.processAllTokens is guaranteed to receive exactly the set of the root tokens in the initial call, i.e. if I don't recurse with this.processAllTokens(token.tokens)? If so, than that would resolve my conundrum.

[UziTech]

marked.lexer(src, options) will return a token tree.

Yes, hooks.processAllTokens is the way to get access to the output of marked.lexer before it goes to walkTokens then to marked.parser to be converted to html.

marked/src/Instance.ts

Lines 307 to 321 in 0076503

	if (opt.hooks) {
	src = opt.hooks.preprocess(src) as string;
	}
	let tokens = lexer(src, opt);
	if (opt.hooks) {
	tokens = opt.hooks.processAllTokens(tokens) as Token[] | TokensList;
	}
	if (opt.walkTokens) {
	this.walkTokens(tokens, opt.walkTokens);
	}
	let html = parser(tokens, opt);
	if (opt.hooks) {
	html = opt.hooks.postprocess(html) as string;
	}
	return html;