add flattened device tree/devicetree blobs specification

[armijnhemel]

This is a specification for DTB (devicetree blobs/flattened device tree), the compiled result of .dts files as found in the Linux kernel. Test files can be found here:

https://github.com/qemu/qemu/tree/master/pc-bios

Note: this is NOT the same as the dtb/dtbo image format that was pulled here: #415

[armijnhemel]

Note: this specification checks the structure of the dtb, but does not process the individual properties (as there are lots of them).

[armijnhemel]

Also, I have added this to the directory "database", but that doesn't seem to be the right place.

[KOLANICH]

but that doesn't seem to be the right place.

We have a directory firmware, but we should really to invent a better structure for the repo (with more formal criteria what goes where which can be used to predict the location of a spec with onpy basic knowledge agout the format it implements). I guess serialization (but not firmware) will be better for this format.

[armijnhemel]

Note: this specification checks the structure of the dtb, but does not process the individual properties (as there are lots of them).

This specification also treats the tokens in the structure block as a list of tokens. It would be more accurate to have it as a tree.

[generalmimon]

Hopefully it will be easier for you to address everything at once...

[generalmimon]

According to https://github.com/devicetree-org/devicetree-specification/releases/download/v0.3/devicetree-specification-v0.3.pdf (section 5.3 Memory Reservation Block > 5.3.2 Format, page 47/56), this lacks inner structure:

5.3.2 Format

The memory reservation block consists of a list of pairs of 64-bit big-endian integers, each pair being represented by the following C structure.

struct fdt_reserve_entry {
    uint64_t address;
    uint64_t size;
};

Each pair gives the physical address and size in bytes of a reserved memory region. These given regions shall not overlap each other. The list of reserved blocks shall be terminated with an entry where both address and size are equal to 0. Note that the address and size values are always 64-bit. On 32-bit CPUs the upper 32-bits of the value are ignored.

Each uint64_t in the memory reservation block, and thus the memory reservation block as a whole, shall be located at an 8-byte aligned offset from the beginning of the devicetree blob (see section 5.6).

[armijnhemel]

That is correct. I skipped this part, as for my particular use case it is not interesting. Also, none of the test files I looked at actually had anything useful in there, except the terminator.

[armijnhemel]

I have added this now as well.

[generalmimon]

Isn't there a more meaningful enum name? I guess that fdt is an abbreviation of Flattened Devicetree Format, not something like token type.

Please don't forget about

kaitai_struct_formats/database/dtb.ksy

Lines 88 to 90 in c4551e5

	- id: token_type
	type: u4
	enum: fdt

if you rename it.

[KOLANICH]

I don't like the word token here, maybe just type?

[armijnhemel]

The reason I called it token is because they are called that in the documentation:

https://github.com/devicetree-org/devicetree-specification/blob/v0.3/source/flattened-format.rst#lexical-structure

The five token types are as follows:

Perhaps because they are in an element fdt_note it can indeed be just token. @generalmimon any thoughts?

[generalmimon]

@armijnhemel

Also, I have added this to the directory "database", but that doesn't seem to be the right place.

@KOLANICH

We have a directory firmware, but we should really to invent a better structure for the repo (with more formal criteria what goes where which can be used to predict the location of a spec with onpy basic knowledge agout the format it implements). I guess serialization (but not firmware) will be better for this format.

I also think serialization is quite reasonable - it's definitely better than database. However, as for #404 (comment), it doesn't really matter what the format looks from inside but rather what it's mainly used for and where people would expect it. From this perspective, I'm thinking about hardware - a devicetree is used to describe system hardware. However, the hardware folder is classified as Hardware Protocols at https://formats.kaitai.io, which is not really what we mean. I don't see any connection with firmware.

[generalmimon]

This name is misleading, at least to me - if I see last compatible version I imagine the latest one (i.e. the one that came last), not the lowest or earliest. However, it's clear that the spec intended the exact opposite:

last_comp_version

This field shall contain the lowest version of the devicetree data structure with which the version used is backwards compatible. So, for the structure as defined in this document (version 17), this field shall contain 16 because version 17 is backwards compatible with version 16, but not earlier versions. As per section 5.1, a DTSpec boot program should provide a devicetree in a format which is backwards compatible with version 16, and thus this field shall always contain 16.

I'm not sure why did the spec authors choose last_ adjective in the name. Probably when you imagine the latest version as the "first", then you go further into the version history and declare the oldest compatible version as the "last" that would the current version be compatible with (if we went beyond the "last", any older versions would no longer work). However, this concept seems to be upside down - I guess it's natural that "last" should mean latest (the last one that has been released so far) and the "first" would mean the initial version, i.e. the oldest version. In this justification for the spec's naming that I came up with, I needed to reverse the timeline and the role of "first" and "last" pointers (we go back in time).

Does first_compatible_version, oldest_compatible_version, lowest_compatible_version or min_compatible_version make more sense? Which one would you choose?

[generalmimon]

As I wrote before in #467 (comment), I moved the spec to the hardware/ directory (a8331ca) because I think it's the first place where people would look for it.

[armijnhemel]

As I wrote before in #467 (comment), I moved the spec to the hardware/ directory (a8331ca) because I think it's the first place where people would look for it.

For pure dtb: yes, but there are, as always, exceptions, namely FIT images where entire Linux kernel images and file systems are included and dtb purely acts as a container.

[generalmimon]

Thanks for contributing! Merging in.