Add support for inline/stack allocated types #778

yorickpeterse · 2024-11-06T23:58:19Z

This adds support for inline types. Such types are immutable value types that are copied upon being moved, and are allocated on the stack without an object header.

TODO

Initial implementation:

Replace ClassKind::Inline with just a flag on Class, making it easier to combine with other class kinds (e.g. Tuple) without the need for a ton of constructors
Disallow recursive inline types
- This can be determined by looking at the definition only. For generic types one has to be able to construct the type in the first place, which isn't possible (code wise) if it's infinite.
Disallow Drop implementations for inline types
- If we allow this, we'd end up calling destructors for every copy and that gets expensive fast. It's also redundant since inline types are restricted to types that we can trivially copy around
Don't call droppers for inline types since this is redundant
Don't generate droppers for inline types since they'll never be used
Merge TypeRef::is_permanent into TypeRef::is_stack_allocated, as the former is now redundant
Ensure that type parameter bounds inherit the mut and inline properties of the original parameter
Ensure that all field types of an inline type are also inline (String won't be allowed, at least for now I think)
Ensure that the struct size logic in compiler::llvm::layouts::Layouts handles inline types correctly.
- This depends on the order in which types are defined, likely requiring a different strategy than the one we use at the moment.
- We might need some sort of work list approach?
Add the TypeParameter::inline field and set this to true for any type parameter defined on an inline type
When passing types to an inline type parameter, only allow this if the passed type is also an inline type
Support class inline enum? This might not be useful if we infer enum types as inline
Add support for specializing inline (dedicated) types
- Add Shape::Custom(ClassInstance)
- Make sure we generate a stable/deterministic name for these shapes, regardless of ordering
- Make sure static dispatch is used for these types when used in generics
- This means we can also allow extern types in generics
- We should probably just hash the shape identifiers using BLAKE3, ensuring we don't end up with gigantic symbol names (this data is stripped from stack traces anyway)
- Shape implements Eq and Hash, but that isn't reliable for ClassInstance since different occurrences of e.g. Option[Foo] use a different type_arguments ID
Make inline types immutable and disallow fn mut methods
- Allowing mutation introduces a lot of compiler complexity I'm really not happy with, and requires rewriting part of the generated code post specialization due to how closures capture data
- fn and fn move methods expose self as T (owned)
The changes in this PR somehow result in values being dropped twice, fix that
Ensure that ref expr and mut expr just return the target register when the value is an inline type
Ensure closures correctly capture inline values by value
Include an extra identifier in symbol names for inline classes, such that if the inline state changes (e.g. through inference in a future setup), we automatically take this into account for object file caching
Ensure that flagging inline types as moved is a noop (i.e. move R₁ does nothing if R₁ contains stack data)
Ensure that inline types are sendable
Add tests to assert that casting inline types to traits fails
Figure out why marking std.net.socket.SocketAddress as inline causes an invalid free()
- Add support for inline/stack allocated types #778 (comment): it's likely due to type sizes not being calculated in the right order
- This is indeed the issue. Instead of first processing regular classes and then inline classes, we need a work list approach for all
Fix macOS ARM64 CI failures
- This is because we don't handle the ARM64 ABI correctly: structs larger than 16 bytes are passed as a pointer argument (see B.4)
- For x86-64, it seems clang uses a pointer combined with byval
- When loading constructor fields into registers, we use the size of the largest value and not of the one that we just loaded. This results in nonsensical data being written to the stack.
  - extractvalue just uses the type of the field its loading, rather than load which takes a type, so we need to bitcast things
Add support for explicit T: inline, otherwise you can't define e.g. a map that takes a T from self that happens to be T: inline and map that to R: inline
Disallow casting inline types to anything (e.g. UInt64) because that doesn't make sense for data on the stack

Verification:

yorickpeterse · 2024-11-07T00:26:22Z

Test code I'm using:

dt.inko

import std.fmt (fmt)
import std.stdio (Stdout)
import std.string (Bytes)
import std.time (DateTime, Duration, Instant)

let COLON = 58
let LOWER_D = 100
let LOWER_M = 109
let LOWER_Z = 122
let MINUS = 45
let NINE = 57
let PERCENT = 37
let PLUS = 43
let UPPER_H = 72
let UPPER_M = 77
let UPPER_S = 83
let UPPER_Y = 89
let ZERO = 48

fn inline digits[T: Bytes](input: ref T, start: Int, size: Int) -> Option[Int] {
  let max = start + size

  if max > input.size { return Option.None }

  let mut idx = start
  let mut num = 0

  while idx < max {
    let byte = input.byte(idx := idx + 1)

    if byte >= ZERO and byte <= NINE {
      num = num.wrapping_mul(10).wrapping_add(byte.wrapping_sub(ZERO))
    } else {
      return Option.None
    }
  }

  Option.Some(num)
}

impl DateTime {
  fn pub static parse[T: Bytes](
    input: ref T,
    format: String,
  ) -> Option[DateTime] {
    if input.size == 0 or format.size == 0 { return Option.None }

    let mut inp_idx = 0
    let mut fmt_idx = 0
    let date = DateTime(
      year: 0,
      month: 0,
      day: 0,
      hour: 0,
      minute: 0,
      second: 0,
      sub_second: 0.0,
      utc_offset: 0,
    )

    loop {
      match format.opt(fmt_idx) {
        case Some(PERCENT) -> {
          fmt_idx += 1

          match format.opt(fmt_idx) {
            case Some(PERCENT) if input.opt(inp_idx).or(-1) == PERCENT -> {
              fmt_idx += 1
              inp_idx += 1
            }
            case Some(PERCENT) -> return Option.None
            case Some(what) -> {
              fmt_idx += 1
              inp_idx += match what {
                case UPPER_Y -> {
                  date.year = try digits(input, inp_idx, size: 4)
                  4
                }
                case LOWER_M -> {
                  date.month = try digits(input, inp_idx, size: 2)
                  2
                }
                case LOWER_D -> {
                  date.day = try digits(input, inp_idx, size: 2)
                  2
                }
                case UPPER_H -> {
                  date.hour = try digits(input, inp_idx, size: 2)
                  2
                }
                case UPPER_M -> {
                  date.minute = try digits(input, inp_idx, size: 2)
                  2
                }
                case UPPER_S -> {
                  date.second = try digits(input, inp_idx, size: 2)
                  2
                }
                case LOWER_Z -> {
                  let pos = match input.opt(inp_idx := inp_idx + 1) {
                    case Some(PLUS) -> true
                    case Some(MINUS) -> false
                    case _ -> return Option.None
                  }

                  let hour = try digits(input, inp_idx, size: 2)

                  inp_idx += 2

                  if input.opt(inp_idx).or(-1) == COLON { inp_idx += 1 }

                  let min = match input.opt(inp_idx) {
                    case Some(byte) if byte >= ZERO and byte <= NINE -> {
                      try digits(input, inp_idx, size: 2)
                    }
                    case _ -> 0
                  }

                  let secs = (hour * 3600) + (min * 60)

                  date.utc_offset = if pos { secs } else { 0 - secs }
                  2
                }
                case _ -> return Option.None
              }
            }
            case _ -> return Option.None
          }
        }
        case Some(f) -> {
          match input.opt(inp_idx) {
            case Some(i) if f == i -> {
              fmt_idx += 1
              inp_idx += 1
            }
            case _ -> return Option.None
          }
        }
        case _ -> break
      }
    }

    Option.Some(date)
  }
}

class async Main {
  fn async main {
    let out = Stdout.new
    let inp = '2024-11-04 12:45:12 -02:45'
    let fmt = '%Y-%m-%d %H:%M:%S %z'
    let mut fastest = Duration.from_secs(1)
    let mut i = 0

    while i < 100_000 {
      let start = Instant.new

      DateTime.parse(inp, fmt)

      let dur = start.elapsed

      if dur < fastest { fastest = dur }

      i += 1
    }

    let res = DateTime.parse(inp, fmt)

    out.print(fmt(res))
    out.print(fmt(fastest))
  }
}

dt_fast.inko

import std.fmt (fmt)
import std.stdio (Stdout)
import std.string (Bytes, OptionalByte)
import std.time (DateTime, Duration, Instant)

let COLON = 58
let LOWER_D = 100
let LOWER_M = 109
let LOWER_Z = 122
let MINUS = 45
let NINE = 57
let PERCENT = 37
let PLUS = 43
let UPPER_H = 72
let UPPER_M = 77
let UPPER_S = 83
let UPPER_Y = 89
let ZERO = 48

fn inline digits[T: Bytes](
  input: ref T,
  start: Int,
  size: Int,
) -> OptionalByte {
  let max = start + size

  if max > input.size { return OptionalByte(-1) }

  let mut idx = start
  let mut num = 0

  while idx < max {
    let byte = input.byte(idx := idx + 1)

    if byte >= ZERO and byte <= NINE {
      num = num.wrapping_mul(10).wrapping_add(byte.wrapping_sub(ZERO))
    } else {
      return OptionalByte(-1)
    }
  }

  OptionalByte(num)
}

impl DateTime {
  fn pub static parse(input: String, format: String) -> Option[DateTime] {
    if input.size == 0 or format.size == 0 { return Option.None }

    let mut inp_idx = 0
    let mut fmt_idx = 0
    let date = DateTime(
      year: 0,
      month: 0,
      day: 0,
      hour: 0,
      minute: 0,
      second: 0,
      sub_second: 0.0,
      utc_offset: 0,
    )

    loop {
      match format.opt_fast(fmt_idx) {
        case { @value = PERCENT } -> {
          fmt_idx += 1

          match format.opt_fast(fmt_idx) {
            case { @value = -1 } -> return Option.None
            case
              { @value = PERCENT } if input.opt_fast(inp_idx).value == PERCENT
            -> {
              fmt_idx += 1
              inp_idx += 1
            }
            case { @value = PERCENT } -> return Option.None
            case { @value = what } -> {
              fmt_idx += 1
              inp_idx += match what {
                case UPPER_Y -> {
                  date.year = match digits(input, inp_idx, size: 4) {
                    case { @value = -1 } -> return Option.None
                    case { @value = v } -> v
                  }
                  4
                }
                case LOWER_M -> {
                  date.month = match digits(input, inp_idx, size: 2) {
                    case { @value = -1 } -> return Option.None
                    case { @value = v } -> v
                  }
                  2
                }
                case LOWER_D -> {
                  date.day = match digits(input, inp_idx, size: 2) {
                    case { @value = -1 } -> return Option.None
                    case { @value = v } -> v
                  }
                  2
                }
                case UPPER_H -> {
                  date.hour = match digits(input, inp_idx, size: 2) {
                    case { @value = -1 } -> return Option.None
                    case { @value = v } -> v
                  }
                  2
                }
                case UPPER_M -> {
                  date.minute = match digits(input, inp_idx, size: 2) {
                    case { @value = -1 } -> return Option.None
                    case { @value = v } -> v
                  }
                  2
                }
                case UPPER_S -> {
                  date.second = match digits(input, inp_idx, size: 2) {
                    case { @value = -1 } -> return Option.None
                    case { @value = v } -> v
                  }
                  2
                }
                case LOWER_Z -> {
                  let pos = match input.opt_fast(inp_idx := inp_idx + 1) {
                    case { @value = PLUS } -> true
                    case { @value = MINUS } -> false
                    case _ -> return Option.None
                  }

                  let hour = match digits(input, inp_idx, size: 2) {
                    case { @value = -1 } -> return Option.None
                    case { @value = v } -> v
                  }

                  inp_idx += 2

                  if input.opt_fast(inp_idx).value == COLON { inp_idx += 1 }

                  let min = match input.opt_fast(inp_idx) {
                    case { @value = byte } if byte >= ZERO and byte <= NINE -> {
                      match digits(input, inp_idx, size: 2) {
                        case { @value = -1 } -> return Option.None
                        case { @value = v } -> v
                      }
                    }
                    case _ -> 0
                  }

                  let secs = (hour * 3600) + (min * 60)

                  date.utc_offset = if pos { secs } else { 0 - secs }
                  2
                }
                case _ -> return Option.None
              }
            }
          }
        }
        case { @value = -1 } -> break
        case { @value = f } -> {
          match input.opt_fast(inp_idx) {
            case { @value = i } if f == i -> {
              fmt_idx += 1
              inp_idx += 1
            }
            case _ -> return Option.None
          }
        }
      }
    }

    Option.Some(date)
  }
}

class async Main {
  fn async main {
    let out = Stdout.new
    let inp = '2024-11-04 12:45:12 -02:45'
    let fmt = '%Y-%m-%d %H:%M:%S %z'
    let mut fastest = Duration.from_secs(1)
    let mut i = 0

    while i < 100_000 {
      let start = Instant.new

      DateTime.parse(inp, fmt)

      let dur = start.elapsed

      if dur < fastest { fastest = dur }

      i += 1
    }

    let res = DateTime.parse(inp, fmt)

    out.print(fmt(res))
    out.print(fmt(fastest))
  }
}

The dt_fast.inko program requires this diff:

diff --git a/std/src/std/string.inko b/std/src/std/string.inko
index 861f4f0d..3d0a6d54 100644
--- a/std/src/std/string.inko
+++ b/std/src/std/string.inko
@@ -150,6 +150,10 @@ trait pub Bytes {
   fn pub to_pointer -> Pointer[UInt8]
 }
 
+class pub inline OptionalByte {
+  let pub @value: Int
+}
+
 # An UTF-8 encoded and immutable string type.
 class builtin String {
   # The size of the string in bytes, _excluding_ the trailing NULL byte.
@@ -678,6 +682,12 @@ class builtin String {
   fn inline byte_unchecked(index: Int) -> Int {
     (@ptr as Int + index as Pointer[UInt8]).0 as Int
   }
+
+  fn pub opt_fast(index: Int) -> OptionalByte {
+    if index < 0 or index >= size { return OptionalByte(-1) }
+
+    OptionalByte(byte_unchecked(index))
+  }
 }
 
 impl Bytes for String {

yorickpeterse · 2024-11-21T20:08:25Z

With these changes in place, the following randomly (though not always) produces allocation errors:

class inline Example {
  let @a: Int
  let @b: Bar
}

class enum Foo {
  case A(Example)
  case B
}

class inline enum Bar {
  case A(Int)
}

class async Main {
  fn async main {
    Foo.A(Example(a: 1, b: Bar.A(42)))
  }
}

My best guess is that the type sizes aren't calculated correctly, resulting in stack data being overwritten.

The panic and no_panic tests run in a separate process, and processes don't link their stack traces. The "panic" helper already took care of setting the correct location, but the "no_panic" helper didn't. Changelog: fixed

@A

An inline type is an immutable value type that's allocated inline/on the stack. Both regular and enum types can be defined as "inline". Inline types don't have object headers, and combined with them being stored inline means they don't incur extra indirection/overhead. For example, consider this type: class inline A { let @A: Int let @b: Int } If this were a regular type, its size would be 32 bytes: 16 bytes for the header, and 16 bytes for the two fields. Because it's an inline type though, it only needs 16 bytes. Inline types are restricted to types that can be trivially copied, such as Int, Float, and other inline types. String isn't allowed in inline types at this point as this could result in an unexpected copy cost due to String using atomic reference counting. Inline types are immutable because supporting mutations introduces significant compiler complexity, especially when dealing with closures that capture generic type parameters, as support for mutations would require rewriting part of the generated code as part of type specialization. This fixes #750. Changelog: added

The Instant and Duration types are now "inline" types, removing the need for heap allocations. In addition, various methods are flagged as "inline" to guarantee they're always inlined into their callers. Changelog: performance

This removes the need for heap allocations when creating ranges. In addition, various methods are flagged as "inline" to guarantee they're inlined into their callers. Changelog: performance.

This makes std.io.Error an "inline" type, removing the need for allocations when creating such instances. Changelog: performance

Changelog: performance

This turns Ipv4Address, Ipv6Address and IpAddress into "inline" types, removing the need for heap allocations. Changelog: performance

This removes the need for a heap allocation when getting IP socket addresses. Changelog: performance

yorickpeterse added feature New things to add to Inko, such as a new standard library module performance Changes related to improving performance compiler Changes related to the compiler std Changes related to the standard library labels Nov 6, 2024

yorickpeterse added this to the 0.18.0 milestone Nov 6, 2024

yorickpeterse self-assigned this Nov 6, 2024

yorickpeterse force-pushed the inline-types branch 2 times, most recently from 393fe24 to 7b068b3 Compare November 7, 2024 00:21

yorickpeterse mentioned this pull request Nov 7, 2024

Implement some form of escape analysis to allocate values on the stack #776

Open

yorickpeterse force-pushed the inline-types branch 5 times, most recently from b7778a5 to b87494f Compare November 9, 2024 01:33

This was referenced Nov 9, 2024

Introduce support for user-defined, stack allocated value types #750

Closed

Recursive C structures blow the stack when compiling #760

Closed

yorickpeterse force-pushed the inline-types branch 13 times, most recently from 3febb00 to 13b5e73 Compare November 14, 2024 19:27

yorickpeterse force-pushed the inline-types branch 3 times, most recently from 1e55e71 to b4da7e5 Compare November 21, 2024 14:54

yorickpeterse mentioned this pull request Nov 21, 2024

Replace the class keyword with the type keyword #779

Closed

6 tasks

yorickpeterse force-pushed the inline-types branch 9 times, most recently from 7441cb8 to 90227b6 Compare November 27, 2024 01:03

Fix failure locations for no_panic tests

db72d59

The panic and no_panic tests run in a separate process, and processes don't link their stack traces. The "panic" helper already took care of setting the correct location, but the "no_panic" helper didn't. Changelog: fixed

yorickpeterse force-pushed the inline-types branch from 90227b6 to 2334e5d Compare November 27, 2024 01:32

yorickpeterse mentioned this pull request Nov 27, 2024

Consider implementing multiple ownership and scoped borrows, replacing single ownership with runtime panics #596

Open

yorickpeterse force-pushed the inline-types branch from 34ce4ce to 3476d06 Compare November 27, 2024 17:03

yorickpeterse mentioned this pull request Nov 27, 2024

Make enums and tuples stack allocated types #780

Closed

yorickpeterse added 7 commits November 28, 2024 17:39

Improve performance for Instant and Duration

2ec8ed8

The Instant and Duration types are now "inline" types, removing the need for heap allocations. In addition, various methods are flagged as "inline" to guarantee they're always inlined into their callers. Changelog: performance

Make InclusiveRange and ExclusiveRange inline

c598221

This removes the need for heap allocations when creating ranges. In addition, various methods are flagged as "inline" to guarantee they're inlined into their callers. Changelog: performance.

Make the IO Error type an inline type

a10c7b8

This makes std.io.Error an "inline" type, removing the need for allocations when creating such instances. Changelog: performance

Make std.csv.Separator an inline type

19997a7

Changelog: performance

Make IP address types inline types

9878d58

This turns Ipv4Address, Ipv6Address and IpAddress into "inline" types, removing the need for heap allocations. Changelog: performance

Make std.net.socket.SocketAddress an inline type

46e3ae7

This removes the need for a heap allocation when getting IP socket addresses. Changelog: performance

yorickpeterse force-pushed the inline-types branch from dfb529c to 46e3ae7 Compare November 28, 2024 16:39

yorickpeterse marked this pull request as ready for review November 28, 2024 16:43

yorickpeterse merged commit 46e3ae7 into main Nov 28, 2024
26 checks passed

yorickpeterse deleted the inline-types branch November 28, 2024 16:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for inline/stack allocated types #778

Add support for inline/stack allocated types #778

yorickpeterse commented Nov 6, 2024 •

edited

Loading

yorickpeterse commented Nov 7, 2024 •

edited

Loading

yorickpeterse commented Nov 21, 2024

Add support for inline/stack allocated types #778

Add support for inline/stack allocated types #778

Conversation

yorickpeterse commented Nov 6, 2024 • edited Loading

TODO

Verification:

yorickpeterse commented Nov 7, 2024 • edited Loading

yorickpeterse commented Nov 21, 2024

yorickpeterse commented Nov 6, 2024 •

edited

Loading

yorickpeterse commented Nov 7, 2024 •

edited

Loading