Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add flattened device tree/devicetree blobs specification #467

Merged
merged 24 commits into from
Aug 18, 2021
Merged

add flattened device tree/devicetree blobs specification #467

Changes from 14 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
453d819
add flattened device tree/devicetree blobs specification
armijnhemel Apr 3, 2021
e5596d5
dtb: add doc-ref to text versions of the specifications, add extra
armijnhemel Apr 3, 2021
efc6f28
dtb: add orig-ids, rename some ids, add xref, add more docs
armijnhemel Apr 3, 2021
41345e4
dtb: add links to example files, describe where on Linux systems files
armijnhemel Apr 3, 2021
f68d07c
dtb: add some applications that use DTB
armijnhemel Apr 7, 2021
5f5ba2f
dtb: clarify which strings are encoded in a subset of ASCII
armijnhemel Apr 7, 2021
ce10d84
dtb: fixes according to KSY style guide
armijnhemel Apr 8, 2021
0ea5632
dtb: use short(er) Git hash in doc-ref
armijnhemel Apr 8, 2021
7fc9eff
dtb: remove superfluous encoding lines
armijnhemel Apr 8, 2021
c4551e5
dtb: rewrite offsets to comply with style guide
armijnhemel Apr 8, 2021
1215a2b
dtb: style fixes, add instance to unlock some more data
armijnhemel Apr 8, 2021
5b83d43
dtb: process memory block
armijnhemel Apr 12, 2021
b3b6979
dtb: add `-webide-representation` to `fdt_begin_node`
generalmimon Jul 18, 2021
41155f6
dtb: remove `.rst` link from `doc` and `doc-ref`
generalmimon Jul 19, 2021
4d87410
dtb: use `*` instead of `-` for list bullets in `/doc`
generalmimon Aug 4, 2021
a8331ca
dtb: move from `database/` to `hardware/`
generalmimon Aug 18, 2021
4e11912
dtb: add `serialization` to `/meta/tags`
generalmimon Aug 18, 2021
ca73d6c
dtb: reorder `/types` so it makes more sense
generalmimon Aug 18, 2021
87ab85b
dtb: rename `{boundary_ => }padding`
generalmimon Aug 18, 2021
856a5c4
dtb: rename `{last => min}_compatible_version`
generalmimon Aug 18, 2021
c048a09
dtb: link to the HTML version of the specification instead of PDF
generalmimon Aug 18, 2021
8037492
dtb: drop field name prefixes
generalmimon Aug 18, 2021
65f9e70
dtb: minor `/doc` fixes
generalmimon Aug 18, 2021
45d9065
dtb: add more info to `/doc`
generalmimon Aug 18, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
152 changes: 152 additions & 0 deletions database/dtb.ksy
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
meta:
id: dtb
title: Flattened Devicetree Format
file-extension: dtb
armijnhemel marked this conversation as resolved.
Show resolved Hide resolved
application:
- Linux
- Das U-Boot
xref:
wikidata: Q16960371
armijnhemel marked this conversation as resolved.
Show resolved Hide resolved
tags:
- linux
license: CC0-1.0
armijnhemel marked this conversation as resolved.
Show resolved Hide resolved
ks-version: 0.9
encoding: ASCII
endian: be
doc: |
Also referred to as Devicetree Blob (DTB). It is a flat
binary encoding of data (primarily devicetree data, although
other data is possible as well).

On Linux systems that support this the blobs can be accessed in
`/sys/firmware/fdt`:

- <https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-firmware-ofw>

The encoding of strings used in the strings block and struct block is
actually a subset of ASCII (section "2.2.1 Node Names"):

<https://github.com/devicetree-org/devicetree-specification/releases/download/v0.3/devicetree-specification-v0.3.pdf>

Example files:

- <https://github.com/qemu/qemu/tree/master/pc-bios>
doc-ref:
- https://github.com/devicetree-org/devicetree-specification/releases/download/v0.3/devicetree-specification-v0.3.pdf
armijnhemel marked this conversation as resolved.
Show resolved Hide resolved
Flattened Devicetree (DTB) Format
- https://elinux.org/images/f/f4/Elc2013_Fernandes.pdf
seq:
- id: magic
armijnhemel marked this conversation as resolved.
Show resolved Hide resolved
-orig-id: magic
contents: [0xd0, 0x0d, 0xfe, 0xed]
- id: total_size
-orig-id: totalsize
type: u4
- id: ofs_structure_block
-orig-id: off_dt_struct
type: u4
- id: ofs_strings_block
-orig-id: off_dt_strings
type: u4
- id: ofs_memory_reservation_block
-orig-id: off_mem_rsvmap
type: u4
- id: version
type: u4
- id: last_compatible_version
Copy link
Member

@generalmimon generalmimon Aug 4, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This name is misleading, at least to me - if I see last compatible version I imagine the latest one (i.e. the one that came last), not the lowest or earliest. However, it's clear that the spec intended the exact opposite:

last_comp_version
This field shall contain the lowest version of the devicetree data structure with which the version used is backwards compatible. So, for the structure as defined in this document (version 17), this field shall contain 16 because version 17 is backwards compatible with version 16, but not earlier versions. As per section 5.1, a DTSpec boot program should provide a devicetree in a format which is backwards compatible with version 16, and thus this field shall always contain 16.

I'm not sure why did the spec authors choose last_ adjective in the name. Probably when you imagine the latest version as the "first", then you go further into the version history and declare the oldest compatible version as the "last" that would the current version be compatible with (if we went beyond the "last", any older versions would no longer work). However, this concept seems to be upside down - I guess it's natural that "last" should mean latest (the last one that has been released so far) and the "first" would mean the initial version, i.e. the oldest version. In this justification for the spec's naming that I came up with, I needed to reverse the timeline and the role of "first" and "last" pointers (we go back in time).

Does first_compatible_version, oldest_compatible_version, lowest_compatible_version or min_compatible_version make more sense? Which one would you choose?

-orig-id: last_comp_version
type: u4
valid:
max: version
- id: boot_cpuid_phys
-orig-id: boot_cpuid_phys
type: u4
- id: len_strings_block
-orig-id: size_dt_strings
type: u4
- id: len_structure_block
-orig-id: size_dt_struct
type: u4
instances:
memory_reservation_block:
pos: ofs_memory_reservation_block
size: ofs_structure_block - ofs_memory_reservation_block
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to https://github.com/devicetree-org/devicetree-specification/releases/download/v0.3/devicetree-specification-v0.3.pdf (section 5.3 Memory Reservation Block > 5.3.2 Format, page 47/56), this lacks inner structure:

5.3.2 Format

The memory reservation block consists of a list of pairs of 64-bit big-endian integers, each pair being represented by the following C structure.

struct fdt_reserve_entry {
    uint64_t address;
    uint64_t size;
};

Each pair gives the physical address and size in bytes of a reserved memory region. These given regions shall not overlap each other. The list of reserved blocks shall be terminated with an entry where both address and size are equal to 0. Note that the address and size values are always 64-bit. On 32-bit CPUs the upper 32-bits of the value are ignored.

Each uint64_t in the memory reservation block, and thus the memory reservation block as a whole, shall be located at an 8-byte aligned offset from the beginning of the devicetree blob (see section 5.6).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is correct. I skipped this part, as for my particular use case it is not interesting. Also, none of the test files I looked at actually had anything useful in there, except the terminator.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added this now as well.

type: memory_block
structure_block:
pos: ofs_structure_block
size: len_structure_block
type: fdt_block
strings_block:
pos: ofs_strings_block
size: len_strings_block
type: strings
types:
memory_block:
seq:
- id: memory_block_entries
type: memory_block_entry
repeat: eos
memory_block_entry:
seq:
- id: address
type: u8
doc: physical address of a reserved memory region
- id: size
type: u8
doc: size of a reserved memory region
strings:
seq:
- id: strings
type: strz
repeat: eos
fdt_node:
armijnhemel marked this conversation as resolved.
Show resolved Hide resolved
-webide-representation: '{type} {body}'
seq:
- id: type
type: u4
enum: fdt
- id: body
type:
switch-on: type
cases:
fdt::begin_node: fdt_begin_node
fdt::prop: fdt_prop
fdt_block:
seq:
- id: fdt_nodes
type: fdt_node
repeat: until
repeat-until: _.type == fdt::end
fdt_begin_node:
-webide-representation: '{name}'
seq:
- id: name
type: strz
- id: boundary_padding
size: (- _io.pos) % 4
fdt_prop:
-webide-representation: '{name}'
seq:
- id: len_property
-orig-id: len
type: u4
- id: ofs_name
-orig-id: nameoff
type: u4
armijnhemel marked this conversation as resolved.
Show resolved Hide resolved
- id: property
size: len_property
- id: boundary_padding
size: (- _io.pos) % 4
instances:
name:
io: _root.strings_block._io
pos: ofs_name
type: strz
-webide-parse-mode: eager
enums:
fdt:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't there a more meaningful enum name? I guess that fdt is an abbreviation of Flattened Devicetree Format, not something like token type.

Please don't forget about

kaitai_struct_formats/database/dtb.ksy

Lines 88 to 90 in c4551e5

- id: token_type
type: u4
enum: fdt

if you rename it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like the word token here, maybe just type?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason I called it token is because they are called that in the documentation:

https://github.com/devicetree-org/devicetree-specification/blob/v0.3/source/flattened-format.rst#lexical-structure

The five token types are as follows:

Perhaps because they are in an element fdt_note it can indeed be just token. @generalmimon any thoughts?

0x00000001: begin_node
0x00000002: end_node
0x00000003: prop
0x00000004: nop
0x00000009: end