Containers

Containers are an organization unit used by Inkle's compiled JSON format. You should read their documentation on containers first before tackling this document. InkCPP preserves the concept of containers from InkJSON but without their hierarchical structure.

Motivation

TODO: Simplify this. We're just re-explaining what we explain below.

InkCPP preserves the concept of containers from InkJSON because the ink runtime expects to be able to ask, for a given container, how many times it has been visited via the visits command (turns is similar).

One way to preserve this would be to simply insert a new, InkCPP exclusive START_CONTAINER_MARKER command at the beginning of each container which incremented its visit count. Unfortunately, the Ink runtime counts not just entering a container as a visit, but also jumping anywhere inside a container. Such a jump would skip right over the START_CONTAINER_MARKER command.

We could compensate for that by scanning all commands from the start of the jump to the jump's destination and test for START_CONTAINER commands, but since each InkCPP command has a different size (depending on its arguments), this would involve quite a bit of logic that would need to be updated anytime the command sizes update, and it's also just a lot of iteration in general for what should be a fast command.

InkCPP compromises with two approaches. One is with new START_CONTAINER_MARKER and END_CONTAINER_MARKER commands. The former increments the visit count for the container and the last visited turn count, the latter does some bookkeeping to keep track of which containers we're currently in (and executes a DONE command if we are actually no longer in a container).

For jumps, the InkCPP consults the "Container Map" stored in the compiled InkCPP binary file. This data structure maps each container to a start and end point in the instruction block. A jump can thus iterate this map in order, starting from the current instruction pointer all the way to the destination, and quickly discover which containers it is leaving and entering and update their visit counts accordingly.

Compiling a JSON Container to Binary

To compile a JSON container into binary...

1. Write a START_CONTAINER_MARKER command to signal the beginning of a container
2. For each child element...
   1. If it's a command, compile it into its binary form (see InkCPP Commands).
   2. If it's an inline container, recurse to compile it in-place.
3. Write out a DIVERT command with the DIVERT_IS_FALLTHROUGH flag which jumps to the final END_CONTAINER_MARKER written below.
4. For each named child container in this container's metadata (again, see the InkJSON documentation)...
   1. Recurse to compile it in-place
   2. Write out a DIVERT command with the DIVERT_IS_FALLTHROUGH flag which jumps to the final END_CONTAINER_MARKER written below.
5. Write out an END_CONTAINER_MARKER command. This instruction is where the above.

Fallthrough Diverts

The special fall-through diverts written out above are meant to mimic InkJSON container behavior even though the entire hierarchy has been flattened. For example, if you get to the end of a container, you don't want to start executing its first child. Similarly, if you get to the end of a container's child container, you don't want the interpreter to begin executing its sibling. The diverts are sandwiched between each child and just before the child list to move the instruction pointer to the end of the container.

Imagine an InkJSON container hierarchy like this

1. Container Beginning
   1. Container Beginning.AteApple
   2. Container Beginning.RefusedApple
   3. Container Beginning.PrayedToApple
2. Container NextStoryBeat

When we flatten this hierarchy, we get this

1. Container Beginning
2. Container Beginning.AteApple
3. Container Beginning.RefusedApple
4. Container Beginning.PrayedToApple
5. Container NextStoryBeat

If we were to enter the Beginning.RefusedApple container and reached its end, the next instruction would naturally be the first instruction of the Beginning.PrayedApple container. Looking back to the hierarchy, we'd actually want to fall into NextStoryBeat. Adding the diverts using the compiling container algorithm above, we get

1. Container Beginning
2. DIVERT TO NextStoryBeat
3. Container Beginning.AteApple
4. DIVERT TO NextStoryBeat
5. Container Beginning.RefusedApple
6. DIVERT TO NextStoryBeat
7. Container Beginning.PrayedToApple
8. DIVERT TO NextStoryBeat
9. Container NextStoryBeat

They're called "fallthrough" diverts in InkCPP because they represent the interpreter "falling out" of the end of a container and dropping down to the next container.

Special Fallthrough Flag

There's a special quirk concerning choices and falling through containers as above. In Ink, choices are added by CHOICE commands until a DONE is reached and the choices are displayed to the player. However, there is a special case: no DONE is required if, after the last choice, we simply run out of content. The DONE in this case is implied. However, in order to keep accurate count of which container we're in and which we've visited, we need to execute the JUMP triggered by selecting a choice from the point where the choices were done being collected. If there was an explicit DONE, that's just the position of that DONE command. In the implicit case, it's a little more tricky. We use the first point at which we started falling out of containers before running out of content.

The special DIVERT_IS_FALLTHROUGH flag just records the position of the current instruction in case its required for this purpose. However, the moment we hit any instruction that isn't a DIVERT_IS_FALLTHROUGH DIVERT, this value is cleared since are clearly not out of content.

OPTIMIZATION: This fallthrough flag is also used to optimize the Container Map algorithm detailed below.

WARNING: There may be another reason I added this that I can't figure out because I have honestly forgotten and it's a bit complicated. But those seem to be the only two places the _is_falling boolean in runner_impl is referenced in code.

The Container Map

In order to implement the VISITS command, we not only need to record how many times each container is visited, but also the current container. This is partly accomplished using the START_CONTAINER_MARKER and END_CONTAINER_MARKER commands, but Ink considers any entry into a container, even if it's jumping into one of its children from a completely different container, to be a visit. We need to be able to figure out which containers we are leaving and entering anytime we make a jump.

This is accomplished with the container map, a data block in the InkCPP Binary File. For each container, it stores the offset of its first instruction and last instruction. These markers are sorted by instruction offset. To find out what containers you are entering and exiting as a result of a jump, you simply iterate this map starting from the current instruction pointer to the destination and mark visits appropriately.

This algorithm is implemented in runner_impl::jump in runner_impl.cpp.

For more details on how the container map is stored, see the relevant section in the Ink CPP Binary File structure document.