Start writing implementation documentation

Also rename Dispatcher to CompilationDispatcher for clarity

CONTRIBUTING.md

@@ -9,6 +9,7 @@ Want to contribute? Great! First, read this page.
   * [Synchronizing](#synchronizing)
   * [File Headers](#file-headers)
 * [Release Process](#release-process)
+* [Package Structure](#package-structure)
 
 ## Before You Contribute
 
@@ -233,3 +234,13 @@ few things to do before pushing that tag:
 
 You *don't* need to create tags for packages in `pkg`; that will be handled
 automatically by GitHub actions.
+
+## Package Structure
+
+The structure of the Sass package is documented in README.md files in most
+directories under `lib/`. This documentation is intended to help contributors
+quickly build a basic understanding of the structure of the compiler and how its
+various pieces fit together. [`lib/src/README.md`] is a good starting point to get
+an overview of the compiler as a whole.
+
+[`lib/src/README.md`]: lib/src/README.md

lib/src/README.md

@@ -0,0 +1,230 @@
+# The Sass Compiler
+
+* [Life of a Compilation](#life-of-a-compilation)
+  * [Late Parsing](#late-parsing)
+  * [Early Serialization](#early-serialization)
+* [JS Support](#js-support)
+* [APIs](#apis)
+  * [Importers](#importers)
+  * [Custom Functions](#custom-functions)
+  * [Loggers](#loggers)
+* [Built-In Functions](#built-in-functions)
+* [`@extend`](#extend)
+
+This is the root directory of Dart Sass's private implementation libraries. This
+contains essentially all the business logic defining how Sass is actually
+compiled, as well as the APIs that users use to interact with Sass. There are
+two exceptions:
+
+* [`../../bin/sass.dart`] is the entrypoint for the Dart Sass CLI (on all
+  platforms). While most of the logic it runs exists in this directory, it does
+  contain some logic to drive the basic compilation logic and handle errors. All
+  the most complex parts of the CLI, such as option parsing and the `--watch`
+  command, are handled in the [`executable`] directory. Even Embedded Sass runs
+  through this entrypoint, although it gets immediately gets handed off to [the
+  embedded compiler].
+
+  [`../../bin/sass.dart`]: ../../bin/sass.dart
+  [`executable`]: executable
+  [the embedded compiler]: embedded/README.md
+
+* [`../sass.dart`] is the entrypoint for the public Dart API. This is what's
+  loaded when a Dart package imports Sass. It just contains the basic
+  compilation functions, and exports the rest of the public APIs from this
+  directory.
+
+  [`../sass.dart`]: ../sass.dart
+
+Everything else is contained here, and each file and some subdirectories have
+their own documentation. But before you dive into those, let's take a look at
+the general lifecycle of a Sass compilation.
+
+## Life of a Compilation
+
+Whether it's invoked through the Dart API, the JS API, the CLI, or the embedded
+host, the basic process of a Sass compilation is the same. Sass is implemented
+as an AST-walking [interpreter] that operates in roughly three passes:
+
+[interpreter]: https://en.wikipedia.org/wiki/Interpreter_(computing)
+
+1. **Parsing**. The first step of a Sass compilation is always to parse the
+   source file, whether it's SCSS, the indented syntax, or CSS. The parsing
+   logic lives in the [`parse`] directory, while the abstract syntax tree that
+   represents the parsed file lives in [`ast/sass`].
+
+   [`parse`]: parse/README.md
+   [`ast/sass`]: ast/sass/README.md
+
+2. **Evaluation**. Once a Sass file is parsed, it's evaluated by
+   [`visitor/async_evaluate.dart`]. (Why is there both an async and a sync
+   version of this file? See [Synchronizing] for details!) The evaluator handles
+   all the Sass-specific logic: it resolves variables, includes mixins, executes
+   control flow, and so on. As it goes, it builds up a new AST that represents
+   the plain CSS that is the compilation result, which is defined in
+   [`ast/css`].
+
+   [`visitor/async_evaluate.dart`]: visitor/async_evaluate.dart
+   [Synchronizing]: ../../CONTRIBUTING.md#synchronizing
+   [`ast/css`]: ast/css/README.md
+
+   Sass evaluation is almost entirely linear: it begins at the first statement
+   of the file, evaluates it (which may involve evaluating its nested children),
+   adds its result to the CSS AST, and then moves on to the second statement. On
+   it goes until it reaches the end of the file, at which point it's done. The
+   only exception is module resolution: every Sass module has its own compiled
+   CSS AST, and once the entrypoint file is done compiling the evaluator will go
+   back through these modules, resolve `@extend`s across them as necessary, and
+   stitch them together into the final stylesheet.
+
+   SassScript, the expression-level syntax, is handled by the same evaluator.
+   The main difference between SassScript and statement-level evaluation is that
+   the same SassScript values are used during evaluation _and_ as part of the
+   CSS AST. This means that it's possible to end up with a Sass-specific value,
+   such as a map or a first-class function, as the value of a CSS declaration.
+   If that happens, the Serialization phase will signal an error when it
+   encounters the invalid value.
+
+3. **Serialization**. Once we have the CSS AST that represents the compiled
+   stylesheet, we need to convert it into actual CSS text. This is done by
+   [`visitor/serialize.dart`], which walks the AST and builds up a big buffer of
+   the resulting CSS. It uses [a special string buffer] that tracks source and
+   destination locations in order to generate [source maps] as well.
+
+   [`visitor/serialize.dart`]: visitor/serialize.dart
+   [a special string buffer]: util/source_map_buffer.dart
+   [source maps]: https://web.dev/source-maps/
+
+There's actually one slight complication here: the first and second pass aren't
+as separate as they appear. When one Sass stylesheet loads another with `@use`,
+`@forward`, or `@import`, that rule is handled by the evaluator and _only at
+that point_ is the loaded file parsed. So in practice, compilation actually
+switches between parsing and evaluation, although each individual stylesheet
+naturally has to be parsed before it can be evaluated.
+
+### Late Parsing
+
+Some syntax within a stylesheet is only parsed _during_ evaluation. This allows
+authors to use `#{}` interpolation to inject Sass variables and other dynamic
+values into various locations, such as selectors, while still allowing Sass to
+parse them to support features like nesting and `@extend`. The following
+syntaxes are parsed during evaluation:
+
+* [Selectors](parse/selector.dart)
+* [`@keyframes` frames](parse/keyframe_selector.dart)
+* [Media queries](parse/media_query.dart) (for historical reasons, these are
+  parsed before evaluation and then _reparsed_ after they've been fully
+  evaluated)
+
+### Early Serialization
+
+There are also some cases where the evaluator can serialize values before the
+main serialization pass. For example, if you inject a variable into a selector
+using `#{}`, that variable's value has to be converted to a string during
+evaluation so that the evaluator can then parse and handle the newly-generated
+selector. The evaluator does this by invoking the serializer _just_ for that
+specific value. As a rule of thumb, this happens anywhere interpolation is used
+in the original stylesheet, although there are a few other circumstances as
+well.
+
+## JS Support
+
+One of the main benefits of Dart as an implementation language is that it allows
+us to distribute Dart Sass both as an extremely efficient stand-alone executable
+_and_ an easy-to-install pure-JavaScript package, using the dart2js compilation
+tool. However, properly supporting JS isn't seamless. There are two major places
+where we need to think about JS support:
+
+1. When interfacing with the filesystem. None of Dart's IO APIs are natively
+   supported on JS, so for anything that needs to work on both the Dart VM _and_
+   Node.js we define a shim in the [`io`] directory that will be implemented in
+   terms of `dart:io` if we're running on the Dart VM or the `fs` or `process`
+   modules if we're running on Node. (We don't support IO at all on the browser
+   except to print messages to the console.)
+
+   [`io`]: io/README.md
+
+2. When exposing an API. Dart's JS interop is geared towards _consuming_ JS
+   libraries from Dart, not producing a JS library written in Dart, so we have
+   to jump through some hoops to make it work. This is all handled in the [`js`]
+   directory.
+
+   [`js`]: js/README.md
+
+## APIs
+
+One of Sass's core features is its APIs, which not only compile stylesheets but
+also allow users to provide plugins that can be invoked from within Sass. In
+both the JS API, the Dart API, and the embedded compiler, Sass provides three
+types of plugins: importers, custom functions, and loggers.
+
+### Importers
+
+Importers control how Sass loads stylesheets through `@use`, `@forward`, and
+`@import`. Internally, _all_ stylesheet loads are modeled as importers. When a
+user passes a load path to an API or compiles a stylesheet through the CLI, we
+just use the built-in [`FilesystemImporter`] which implements the same interface
+that we make available to users.
+
+[`FilesystemImporter`]: importer/filesystem.dart
+
+In the Dart API, the importer root class is [`importer/async_importer.dart`].
+The JS API and the embedded compiler wrap the Dart importer API in
+[`importer/node_to_dart`] and [`embedded/importer`] respectively.
+
+[`importer/async_importer.dart`]: importer/async_importer.dart
+[`importer/node_to_dart`]: importer/node_to_dart
+[`embedded/importer`]: embedded/importer
+
+### Custom Functions
+
+Custom functions are defined by users of the Sass API but invoked by Sass
+stylesheets. To a Sass stylesheet, they look like any other built-in function:
+users pass SassScript values to them and get SassScript values back. In fact,
+all the core Sass functions are implemented using the Dart custom function API.
+
+Because custom functions take and return SassScript values, that means we need
+to make _all_ values available to the various APIs. For Dart, this is
+straightforward: we need to have objects to represent those values anyway, so we
+just expose those objects publicly (with a few `@internal` annotations here and
+there to hide APIs we don't want users relying on). These value types live in
+the [`value`] directory.
+
+[`value`]: value/README.md
+
+Exposing values is a bit more complex for other platforms. For the JS API, we do
+a bit of metaprogramming in [`node/value`] so that we can return the
+same Dart values we use internally while still having them expose a JS API that
+feels native to that language. For the embedded host, we convert them to and
+from a protocol buffer representation in [`embedded/value.dart`].
+
+[`node/value`]: node/value/README.md
+[`embedded/value.dart`]: embedded/value.dart
+
+### Loggers
+
+Loggers are the simplest of the plugins. They're just callbacks that are invoked
+any time Dart Sass would emit a warning (from the language or from `@warn`) or a
+debug message from `@debug`. They're defined in:
+
+* [`logger.dart`](logger.dart) for Dart
+* [`node/logger.dart`](node/logger.dart) for Node
+* [`embedded/logger.dart`](embedded/logger.dart) for the embedded compiler
+
+## Built-In Functions
+
+All of Sass's built-in functions are defined in the [`functions`] directory,
+including both global functions and functions defined in core modules like
+`sass:math`. As mentioned before, these are defined using the standard custom
+function API, although in a few cases they use additional private features like
+the ability to define multiple overloads of the same function name.
+
+[`functions`]: functions/README.md
+
+## `@extend`
+
+The logic for Sass's `@extend` rule is particularly complex, since it requires
+Sass to not only parse selectors but to understand how to combine them and when
+they can be safely optimized away. Most of the logic for this is contained
+within the [`extend`] directory.
+
+[`extend`]: extend/README.md

lib/src/ast/css/README.md

@@ -0,0 +1,50 @@
+# CSS Abstract Syntax Tree
+
+This directory contains the abstract syntax tree that represents a plain CSS
+file generated by Sass compilation. It differs from other Sass ASTs in two major
+ways:
+
+1. Instead of being created by [a parser], it's created by [the evaluator] as it
+   traverses the [Sass AST].
+
+   [a parser]: ../../parse/README.md
+   [the evaluator]: ../../visitor/async_evaluate.dart
+   [Sass AST]: ../sass/README.md
+
+2. Because of various Sass features like `@extend` and at-rule hoisting, the CSS
+   AST is mutable even though all other ASTs are immutable.
+
+**Note:** the CSS AST doesn't have its own representation of declaration values.
+Instead, declaration values are represented as [`Value`] objects. This does mean
+that a CSS AST can be in a state where some of its values aren't representable
+in plain CSS (such as maps)—in this case, [the serializer] will emit an error.
+
+[`Value`]: ../../value/README.md
+[the serializer]: ../../visitor/serialize.dart
+
+## Mutable and Immutable Views
+
+Internally, the CSS AST is mutable to allow for operations like hoisting rules
+to the root of the AST and updating existing selectors when `@extend` rules are
+encountered. However, because mutability poses a high risk for "spooky action at
+a distance", we limit access to mutating APIs exclusively to the evaluator.
+
+We do this by having an _unmodifiable_ interface (written in this directory) for
+each CSS AST node which only exposes members that don't modify the node in
+question. The implementations of those interfaces, which _do_ have modifying
+methods, live in the [`modifiable`] directory. We then universally refer to the
+immutable node interfaces except specifically in the evaluator, and the type
+system automatically ensures we don't accidentally mutate anything we don't
+intend to.
+
+[`modifiable`]: modifiable
+
+(Of course, it's always possible to cast an immutable node type to a mutable
+one, but that's a very clear code smell that a reviewer can easily identify.)
+
+## CSS Source Files
+
+A lesser-known fact about Sass is that it actually supports _three_ syntaxes for
+its source files: SCSS, the indented syntax, and plain CSS. But even when it
+parses plain CSS, it uses the Sass AST rather than the CSS AST to represent it
+so that parsing logic can easily be shared with the other stylesheet parsers.

lib/src/ast/sass/README.md

@@ -0,0 +1,34 @@
+# Sass Abstract Syntax Tree
+
+This directory contains the abstract syntax tree that represents a Sass source
+file, regardless of which syntax it was written in (SCSS, the indented syntax,
+or plain CSS). The AST is constructed recursively by [a parser] from the leaf
+nodes in towards the root, which allows it to be fully immutable.
+
+[a parser]: ../../parse/README.md
+
+The Sass AST is broken up into three categories:
+
+1. The [statement AST], which represents statement-level constructs like
+   variable assignments, style rules, and at-rules.
+
+   [statement AST]: statement
+
+2. The [expression AST], which represents SassScript expressions like function
+   calls, operations, and value literals.
+
+   [expression AST]: exprssion
+
+3. Miscellaneous AST nodes that are used by both statements and expressions or
+   don't fit cleanly into either category that live directly in this directory.
+
+The Sass AST nodes are processed (usually from the root [`Stylesheet`]) by [the
+evaluator], which runs the logic they encode and builds up a [CSS AST] that
+represents the compiled stylesheet. They can also be transformed back into Sass
+source using the `toString()` method. Since this is only ever used for debugging
+and doesn't need configuration or full-featured indentation tracking, it doesn't
+use a full visitor.
+
+[`Stylesheet`]: statement/stylesheet.dart
+[the evaluator]: ../../visitor/async_evaluate.dart
+[CSS AST]: ../css/README.md

lib/src/ast/selector/README.md

@@ -0,0 +1,24 @@
+# Selector Abstract Syntax Tree
+
+This directory contains the abstract syntax tree that represents a parsed CSS
+selector. This AST is constructed recursively by [the selector parser]. It's
+fully immutable.
+
+[the selector parser]: ../../parse/selector.dart
+
+Unlike the [Sass AST], which is parsed from a raw source string before being
+evaluated, the selector AST is parsed _during evaluation_. This is necessary to
+ensure that there's a chance to resolve interpolation before fully parsing the
+selectors in question.
+
+[Sass AST]: ../sass/README.md
+
+Although this AST doesn't include any SassScript, it _does_ include a few
+Sass-specific constructs: the [parent selector] `&` and [placeholder selectors].
+Parent selectors are resolved by [the evaluator] before it hands the AST off to
+[the serializer], while placeholders are omitted in the serializer itself.
+
+[parent selector]: parent.dart
+[placeholder selectors]: placeholder.dart
+[the evaluator]: ../../visitor/async_evaluate.dart
+[the serializer]: ../../visitor/serialize.dart

lib/src/embedded/README.md

@@ -0,0 +1,28 @@
+# Embedded Sass Compiler
+
+This directory contains the Dart Sass embedded compiler. This is a special mode
+of the Dart Sass command-line executable, only supported on the Dart VM, in
+which it uses stdin and stdout to communicate with another endpoint, the
+"embedded host", using a protocol buffer-based protocol. See [the embedded
+protocol specification] for details.
+
+[the embedded protocol specification]: https://github.com/sass/sass/blob/main/spec/embedded-protocol.md
+
+The embedded compiler has two different levels of dispatchers for handling
+incoming messages from the embedded host:
+
+1. The [`IsolateDispatcher`] is the first recipient of each packet. It decodes
+   the packets _just enough_ to determine which compilation they belong to, and
+   forwards them to the appropriate compilation dispatcher. It also parses and
+   handles messages that aren't compilation specific, such as `VersionRequest`.
+
+   [`IsolateDispatcher`]: isolate_dispatcher.dart
+
+2. The [`CompilationDispatcher`] fully parses and handles messages for a single
+   compilation. Each `CompilationDispatcher` runs in a separate isolate so that
+   the embedded compiler can run multiple compilations in parallel.
+
+   [`CompilationDispatcher`]: compilation_dispatcher.dart
+
+Otherwise, most of the code in this directory just wraps Dart APIs to
+communicate with their protocol buffer equivalents.

lib/src/embedded/dispatcher.dart → lib/src/embedded/compilation_dispatcher.dart

@@ -29,7 +29,7 @@ final _outboundRequestId = 0;
 
 /// A class that dispatches messages to and from the host for a single
 /// compilation.
-class Dispatcher {
+class CompilationDispatcher {
   /// The channel of encoded protocol buffers, connected to the host.
   final StreamChannel<Uint8List> _channel;
 
@@ -47,13 +47,13 @@ class Dispatcher {
 
   /// A completer awaiting a response to an outbound request.
   ///
-  /// Since each [Dispatcher] is only running a single-threaded compilation, it
-  /// can only ever have one request outstanding.
+  /// Since each [CompilationDispatcher] is only running a single-threaded
+  /// compilation, it can only ever have one request outstanding.
   Completer<GeneratedMessage>? _outstandingRequest;
 
-  /// Creates a [Dispatcher] that sends and receives encoded protocol buffers
-  /// over [channel].
-  Dispatcher(this._channel, this._compilationId)
+  /// Creates a [CompilatoinDispatcher] that sends and receives encoded protocol
+  /// buffers over [channel].
+  CompilationDispatcher(this._channel, this._compilationId)
       : _compilationIdVarint = serializeVarint(_compilationId);
 
   /// Listens for incoming `CompileRequests` and runs their compilations.
@@ -93,7 +93,8 @@ class Dispatcher {
             var response = await _compile(request);
             _send(OutboundMessage()..compileResponse = response);
             success = true;
-            // Each Dispatcher runs a single compilation and then closes.
+            // Each CompilationDispatcher runs a single compilation and then
+            // closes.
             _channel.sink.close();
 
           case InboundMessage_Message.canonicalizeResponse:
@@ -273,8 +274,8 @@ class Dispatcher {
 
     if (_outstandingRequest != null) {
       throw StateError(
-          "Dispatcher.sendRequest() can't be called when another request is "
-          "active.");
+          "CompilationDispatcher.sendRequest() can't be called when another "
+          "request is active.");
     }
 
     return (_outstandingRequest = Completer<T>()).future;

lib/src/embedded/host_callable.dart

@@ -7,7 +7,7 @@ import 'dart:cli';
 
 import '../callable.dart';
 import '../exception.dart';
-import 'dispatcher.dart';
+import 'compilation_dispatcher.dart';
 import 'embedded_sass.pb.dart';
 import 'function_registry.dart';
 import 'protofier.dart';
@@ -21,8 +21,8 @@ import 'utils.dart';
 /// the name defined in the [signature].
 ///
 /// Throws a [SassException] if [signature] is invalid.
-Callable hostCallable(
-    Dispatcher dispatcher, FunctionRegistry functions, String signature,
+Callable hostCallable(CompilationDispatcher dispatcher,
+    FunctionRegistry functions, String signature,
     {int? id}) {
   late Callable callable;
   callable = Callable.fromSignature(signature, (arguments) {

lib/src/embedded/importer/base.dart

@@ -5,14 +5,14 @@
 import 'package:meta/meta.dart';
 
 import '../../importer.dart';
-import '../dispatcher.dart';
+import '../compilation_dispatcher.dart';
 
 /// An abstract base class for importers that communicate with the host in some
 /// way.
 abstract class ImporterBase extends Importer {
-  /// The [CompileDispatcher] to which to send requests.
+  /// The [CompilationDispatcher] to which to send requests.
   @protected
-  final Dispatcher dispatcher;
+  final CompilationDispatcher dispatcher;
 
   ImporterBase(this.dispatcher);
 

lib/src/embedded/importer/file.dart

@@ -6,7 +6,7 @@
 import 'dart:cli';
 
 import '../../importer.dart';
-import '../dispatcher.dart';
+import '../compilation_dispatcher.dart';
 import '../embedded_sass.pb.dart' hide SourceSpan;
 import 'base.dart';
 
@@ -22,7 +22,8 @@ class FileImporter extends ImporterBase {
   /// The host-provided ID of the importer to invoke.
   final int _importerId;
 
-  FileImporter(Dispatcher dispatcher, this._importerId) : super(dispatcher);
+  FileImporter(CompilationDispatcher dispatcher, this._importerId)
+      : super(dispatcher);
 
   Uri? canonicalize(Uri url) {
     if (url.scheme == 'file') return _filesystemImporter.canonicalize(url);

lib/src/embedded/importer/host.dart

@@ -6,7 +6,7 @@
 import 'dart:cli';
 
 import '../../importer.dart';
-import '../dispatcher.dart';
+import '../compilation_dispatcher.dart';
 import '../embedded_sass.pb.dart' hide SourceSpan;
 import '../utils.dart';
 import 'base.dart';
@@ -16,7 +16,8 @@ class HostImporter extends ImporterBase {
   /// The host-provided ID of the importer to invoke.
   final int _importerId;
 
-  HostImporter(Dispatcher dispatcher, this._importerId) : super(dispatcher);
+  HostImporter(CompilationDispatcher dispatcher, this._importerId)
+      : super(dispatcher);
 
   Uri? canonicalize(Uri url) {
     // ignore: deprecated_member_use

lib/src/embedded/isolate_dispatcher.dart

@@ -12,7 +12,7 @@ import 'package:protobuf/protobuf.dart';
 import 'package:stream_channel/isolate_channel.dart';
 import 'package:stream_channel/stream_channel.dart';
 
-import 'dispatcher.dart';
+import 'compilation_dispatcher.dart';
 import 'embedded_sass.pb.dart';
 import 'util/explicit_close_transformer.dart';
 import 'util/proto_extensions.dart';
@@ -218,7 +218,8 @@ void _isolateMain(SendPort sendPort) {
     var (compilationSendPort, compilationId) = initialMessage;
     var compilationChannel =
         IsolateChannel<Uint8List>.connectSend(compilationSendPort);
-    var success = await Dispatcher(compilationChannel, compilationId).listen();
+    var success =
+        await CompilationDispatcher(compilationChannel, compilationId).listen();
     if (!success) channel.sink.close();
   });
 }

lib/src/embedded/logger.dart

@@ -8,14 +8,14 @@ import 'package:stack_trace/stack_trace.dart';
 
 import '../logger.dart';
 import '../utils.dart';
-import 'dispatcher.dart';
+import 'compilation_dispatcher.dart';
 import 'embedded_sass.pb.dart' hide SourceSpan;
 import 'utils.dart';
 
 /// A Sass logger that sends log messages as `LogEvent`s.
 class EmbeddedLogger implements Logger {
-  /// The [Dispatcher] to which to send events.
-  final Dispatcher _dispatcher;
+  /// The [CompilationDispatcher] to which to send events.
+  final CompilationDispatcher _dispatcher;
 
   /// Whether the formatted message should contain terminal colors.
   final bool _color;

lib/src/embedded/protofier.dart

@@ -3,7 +3,7 @@
 // https://opensource.org/licenses/MIT.
 
 import '../value.dart';
-import 'dispatcher.dart';
+import 'compilation_dispatcher.dart';
 import 'embedded_sass.pb.dart' as proto;
 import 'embedded_sass.pb.dart' hide Value, ListSeparator, CalculationOperator;
 import 'function_registry.dart';
@@ -16,7 +16,7 @@ import 'utils.dart';
 /// custom function call.
 class Protofier {
   /// The dispatcher, for invoking deprotofied [Value_HostFunction]s.
-  final Dispatcher _dispatcher;
+  final CompilationDispatcher _dispatcher;
 
   /// The IDs of first-class functions.
   final FunctionRegistry _functions;

lib/src/embedded/value.dart

@@ -3,7 +3,7 @@
 // https://opensource.org/licenses/MIT.
 
 import '../value.dart';
-import 'dispatcher.dart';
+import 'compilation_dispatcher.dart';
 import 'embedded_sass.pb.dart' as proto;
 import 'embedded_sass.pb.dart' hide Value, ListSeparator;
 import 'function_registry.dart';
@@ -83,8 +83,8 @@ proto.ListSeparator _protofySeparator(ListSeparator separator) {
 ///
 /// The [functions] tracks the IDs of first-class functions so that they can be
 /// deserialized to their original references.
-Value deprotofyValue(
-    Dispatcher dispatcher, FunctionRegistry functions, proto.Value value) {
+Value deprotofyValue(CompilationDispatcher dispatcher,
+    FunctionRegistry functions, proto.Value value) {
   // Curry recursive calls to this function so we don't have to keep repeating
   // ourselves.
   deprotofy(proto.Value value) => deprotofyValue(dispatcher, functions, value);

lib/src/extend/README.md

@@ -0,0 +1,35 @@
+# `@extend` Logic
+
+This directory contains most of the logic for running Sass's `@extend` rule.
+This rule is probably the most complex corner of the Sass language, since it
+involves both understanding the semantics of selectors _and_ being able to
+combine them.
+
+The high-level lifecycle of extensions is as follows:
+
+1. When [the evaluator] encounters a style rule, it registers its selector in
+   the [`ExtensionStore`] for the current module. This applies any extensions
+   that have already been registered, then returns a _mutable_
+   `Box<SelectorList>` that will get updated as extensions are applied.
+
+   [the evaluator]: ../visitor/async_evaluate.dart
+   [`ExtensionStore`]: extension_store.dart
+
+2. When the evaluator encounters an `@extend`, it registers that in the current
+   module's `ExtensionStore` as well. This updates any selectors that have
+   already been registered with that extension, _and_ updates the extension's
+   own extender (the selector that gets injected when the extension is applied,
+   which is stored along with the extension). Note that the extender has to be
+   extended separately from the selector in the style rule, because the latter
+   gets redundant selectors trimmed eagerly and the former does not.
+
+3. When the entrypoint stylesheet has been fully executed, the evaluator
+   determines which extensions are visible from which modules and adds
+   extensions from one store to one another accordingly using
+   `ExtensionStore.addExtensions()`.
+
+Otherwise, the process of [extending a selector] as described in the Sass spec
+matches the logic here fairly closely. See `ExtensionStore._extendList()` for
+the primary entrypoint for that logic.
+
+[extending a selector]: https://github.com/sass/sass/blob/main/spec/at-rules/extend.md#extending-a-selector

lib/src/functions/README.md

@@ -0,0 +1,24 @@
+# Built-In Functions
+
+This directory contains the standard functions that are built into Sass itself,
+both those that are available globally and those that are available only through
+built-in modules. Each of the files here exports a corresponding
+[`BuiltInModule`, and most define a list of global functions as well.
+
+[`BuiltInModule`]: ../module/built_in.dart
+
+There are a few functions that Sass supports that aren't defined here:
+
+* The `if()` function is defined directly in the [`functions.dart`] file,
+  although in most cases this is actually parsed as an [`IfExpression`] and
+  handled directly by [the evaluator] since it has special behavior about when
+  its arguments are evaluated. The function itself only exists for edge cases
+  like `if(...$args)` or `meta.get-function("if")`.
+
+  [`functions.dart`]: ../functions.dart
+  [`IfExpression`]: ../ast/sass/expression/if.dart
+  [the evaluator]: ../visitor/async_evaluate.dart
+
+* Certain functions in the `sass:meta` module require runtime information that's
+  only available to the evaluator. These functions are defined in the evaluator
+  itself so that they have access to its private variables.

lib/src/io/README.md

@@ -0,0 +1,17 @@
+# Input/Output Shim
+
+This directory contains an API shim for doing various forms of IO across
+different platforms. Dart chooses at compile time which of the three files to
+use:
+
+* `interface.dart` is used by the Dart Analyzer for static checking. It defines
+  the "expected" interface of the other two files, although there aren't strong
+  checks that their interfaces are exactly the same.
+
+* `vm.dart` is used by the Dart VM, and defines IO operations in terms of the
+  `dart:io` library.
+
+* `js.dart` is used by JS platforms. On Node.js, it will use Node's `fs` and
+  `process` APIs for IO operations. On other JS platforms, most IO operations
+  won't work at all, although messages will still be emitted with
+  `console.log()` and `console.error()`.

lib/src/node/README.md

@@ -0,0 +1,58 @@
+# JavaScript API
+
+This directory contains Dart Sass's implementation of the Sass JS API. Dart's JS
+interop support is primarily intended for _consuming_ JS libraries from Dart, so
+we have to jump through some hoops in order to effectively _produce_ a JS
+library with the desired API.
+
+JS support has its own dedicated entrypoint in [`../js.dart`]. The [`cli_pkg`
+package] ensures that when users load Dart Sass _as a library_, this entrypoint
+is run instead of the CLI entrypoint, but otherwise it's up to us to set up the
+library appropriately. To do so, we use JS interop to define an [`Exports`]
+class that is in practice implemented by a CommonJS-like[^1] `exports` object,
+and then assign various values to this object.
+
+[`../js.dart`]: ../js.dart
+[`cli_pkg` package]: https://google.com/google/dart_cli_pkg
+[`Exports`]: exports.dart
+
+[^1]: It's not _literally_ CommonJS because it needs to run directly on browsers
+      as well, but it's still an object named `exports` that we can hang names
+      off of.
+
+## Value Types
+
+The JS API value types pose a particular challenge from Dart. Although every
+Dart class is represented by a JavaScript class when compiled to JS, Dart has no
+way of specifying what the JS API of those classes should be. What's more, in
+order to make the JS API as efficient as possible, we want to be able to pass
+the existing Dart [`Value`] objects as-is to custom functions rather than
+wrapping them with JS-only wrappers.
+
+[`Value`]: ../value.dart
+
+To solve the first problem problems, in [`reflection.dart`] we use JS interop to
+wrap the manual method of defining a JavaScript class. We use this to create a
+JS-specific class for each value type, with all the JS-specific methods and
+properties defined by Sass's JS API spec. However, while normal JS constructors
+just sets some properties on `this`, our constructors for these classes return
+Dart `Value` objects instead.
+
+[`reflection.dart`]: reflection.dart
+
+"But wait," I hear you say, "those `Value` objects aren't instances of the new
+JS class you've created!" This is where the deep magic comes in. Once we've
+defined our class with its phony constructor, we create a single Dart object of
+the given `Value` subclass and _edit its JavaScript prototype chain_ to include
+the new class we just created. Once that's done, all the Dart value types will
+have exactly the right JS API (including responding correctly to `instanceof`!)
+and the constructor will now correctly return an instance of the JS class.
+
+## Legacy API
+
+Dart Sass also supports the legacy JS API in the [`legacy`] directory. This hews
+as close as possible to the API of the old `node-sass` package which wrapped the
+old LibSass implementation. It's no longer being actively updated, but we still
+need to support it at least until the next major version release of Dart Sass.
+
+[`legacy`]: legacy

lib/src/parse/README.md

@@ -0,0 +1,32 @@
+# Sass Parser
+
+This directory contains various parsers used by Sass. The two most relevant
+classes are:
+
+* [`Parser`]: The base class of all other parsers, which includes basic
+  infrastructure, utilities, and methods for parsing common CSS constructs that
+  appear across multiple different specific parsers.
+
+  [`Parser`]: parser.dart
+
+* [`StylesheetParser`]: The base class specifically for the initial stylesheet
+  parse. Almost all of the logic for parsing Sass files, both statement- and
+  expression-level, lives here. Only places where individual syntaxes differ
+  from one another are left abstract or overridden by subclasses.
+
+  [`StylesheetParser`]: stylesheet.dart
+
+All Sass parsing is done by hand using the [`string_scanner`] package, which we
+use to read the source byte-by-byte while also tracking source span information
+which we can then use to report errors and generate source maps. We don't use
+any kind of parser generator, partly because Sass's grammar requires arbitrary
+backtracking in various places and partly because handwritten code is often
+easier to read and debug.
+
+[`string_scanner`]: https://pub.dev/packages/string_scanner
+
+The parser is simple recursive descent. There's usually a method for each
+logical production that either consumes text and returns its corresponding AST
+node or throws an exception; in some cases, a method (conventionally beginning
+with `try`) will instead consume text and return a node if it matches and return
+null without consuming anything if it doesn't.

lib/src/value/README.md

@@ -0,0 +1,13 @@
+# Value Types
+
+This directory contains definitions for all the SassScript value types. These
+definitions are used both to represent SassScript values internally and in the
+public Dart API. They are usually produced by [the evaluator] as it evaluates
+the expression-level [Sass AST].
+
+[the evaluator]: ../visitor/async_evaluate.dart
+[Sass AST]: ../ast/sass/README.md
+
+Sass values are always immutable, even internally. Any changes to them must be
+done by creating a new value. In some cases, it's easiest to make a mutable
+copy, edit it, and then create a new immutable value from the result.

lib/src/visitor/README.md

@@ -0,0 +1,15 @@
+# Visitors
+
+This directory contains various types that implement the [visitor pattern] for
+[various ASTs]. A few of these, such as [the evaluator] and [the serializer],
+implement critical business logic for the Sass compiler. Most of the rest are
+either small utilities or base classes for small utilities that need to run over
+an AST to determine some kind of information about it. Some are even entirely
+unused within Sass itself, and exist only to support users of the [`sass_api`]
+package.
+
+[visitor pattern]: https://en.wikipedia.org/wiki/Visitor_pattern
+[various ASTs]: ../ast
+[the evaluator]: async_evaluate.dart
+[the serializer]: serialize.dart
+[`sass_api`]: https://pub.dev/packages/sass_api