AArch64 build attributes specification

[smithp35]

Introduce a new specification document for AArch64 build attributes.

The 32-bit ABI makes extensive use of the build attributes defined in addenda32. These attributes defined for relocatable objects are used for a number of purposes including:

• Checking ABI compatibility.

• Checking required processor feature compatibility.

• Automatic static library selection.

With AArch64's focus on software platforms, there has not been a strong requirement to introduce extra metadata to denote the requirements and optional processor features used by a relocatable object. Programs are expected to do runtime testing to check if processor features are available.

With the advent of security features like PAuthABI, BTI and GCS we need a way to record additional metadata that a static linker can use to set metadata like GNU properties in a loadable-unit (executable or shared-library).

A brief summary:

• Made a generic "aeabi" prefix for Arm subsections. Including a header that states optionality and encoding. All ULEB or all NTBS.

• Describes a single subsection that maps directly to the existing .note.gnu.properties and uses language similar to AAELF64. This includes the existing BTI and PAC properties.

• Is accompanied by a design rationale with a suggestion for tool syntax.

[smithp35]

I don't think the PDF creation is to do with this pull-request as I didn't add the directory to the generate-pdfs script and the same attribute error happens in other documents.

Looks like we may need to upgrade rst2pdf to 0.101 for python 3.12 compatibility.

[MaskRay]

The design doc mentions "optimized for size". If we optimize for size, "optional" and "parameter type" can be merged into one byte. In contrast, the current design using two bytes is cleaner. I wonder whether there are additional thoughts into the current choice.

[smithp35]

That is true. In this particular case I'm not expecting too many different subsections, currently we only define one so my instinct is to keep it simple for now.

[MaskRay]

Looks good. This can be extended in the future if addenda32-style Tag_compatibility is needed.

[MaskRay]

Use lowercase "syntax"?

[smithp35]

Yes, will fix.

[MaskRay]

I assume that this AArch32-specific example can be replaced when an AArch64 tag with the right semantics arises.

[smithp35]

Yes. I'm somewhat limited with the examples right now.

[MaskRay]

Should .note.gnu.property be placed in backquotes?

[smithp35]

Yes, will fix.

[MaskRay]

perhaps semilattice. Since we support just the "join" operation, not "meet".

[smithp35]

Looking it up, it seems like join-semilattice is the formal term. What I'll do for now is explain that it is a join-semilattice but we'll use lattice for brevity.

[smithp35]

I ended up using partial order to remove the use of lattice. While build attributes are modelled on a lattice, the addenda32 only mentions it once.

[MaskRay]

The existing docs don't use the term "loadable-unit". "link unit" has been used in several docs. Is there a rationale to introduce this alias?

[smithp35]

In some internal feedback some of my colleagues didn't like link-unit :-) . Loadable unit was felt to be more inituitively understandable than link-unit.

I'm confortable with link-unit, but that's many years of familiarity.

[MaskRay]

This sentence appears to contradict with "Encoding of meta data in a loadable-unit" in buildattr64.rst. If the recommended practice is for loaders not to inspect the platform-specific build attributes, mention it in buildattr64.rst?

[smithp35]

In buildattr64.rst there is:

Encoding of meta data in a loadable-unit
----------------------------------------

Build attributes are only defined for relocatable objects.  The
encoding of metadata in a loadable-unit (executable or shared-library)
is platform specific.  For example a platform may choose to use GNU
program properties as defined in (LINUX_ABI_). This document specifies
the meaning of build attributes for relocatable objects only.  The
presence of ``.ARM.attributes`` sections in a loadable-unit is
permitted, but the interpretation of the contents of the section is
outside the scope of this document.

I think that is consistent with Build Attributes are relocatable only.

Arm has traditionally taken the approach that the executable and shared-library are mostly "owned" by the platform holder, be that Android or the Linux community and the AArch64 ABI. I've written #229 which is a rationale for using GNU properties in sysvabi64.rst files which we equate to Linux/BSD.

Hope that makes sense. Any suggestions for improvement are welcome though.

[MaskRay]

I am puzzled by this sentence. Perhaps clarify it?

[smithp35]

OK, I'll try and rewrite it.

[MaskRay]

Add a colon after "Alternative" to clarify that "alternative one" is not a phrase?

[smithp35]

OK, will fix.

[MaskRay]

Thanks for starting https://discourse.llvm.org/t/rfc-aarch64-build-attributes-for-elf-relocatable-objects/75161 and I came here while reading that post. I have done a lot of object file format/assembler/linker work in llvm-project and I am in favor of AArch64 Build Attributes. I haven't studied much about PAuth, but if loaders don't need to know the relevant build attributes, using build attributes should be good.

LLD processing of Build Attributes sections similar to the ARM target.

In lld/ELF/Driver.cpp, I added mergeRISCVAttributesSections. For AArch64 we will need something similar.

[MaskRay]

Forgot to push some "pending comments"...

[MaskRay]

missing words after "the"?

[smithp35]

ACK

[MaskRay]

delete trailing blank line

[smithp35]

ACK

[MaskRay]

typo: "then"

[smithp35]

ACK

[MaskRay]

delete trailing blank lines?

[smithp35]

ACK

[smithp35]

Uploaded new version, thanks for the additional comments.

[sivan-shani]

The first byte will be followed by 0 or more subsections.

[ <uint32: subsection-length> NTBS: vendor-name
  <bytes: vendor-data>
]

Is the equivalent of a single .aeabi_subsection in the assembly.
for each .aeabi_subsection there must be single

[ <uint32: subsection-length> NTBS: vendor-name
  <bytes: vendor-data>
]

In the ELF.

Each unique .aeabi_subsection vendor-name will result in a single subsection. That permits an assembler to allow .aeabi_subsection to appear multiple times in the same file with the same vendor name, like a .section directive. Therefore it is possible to change subsections, but be able to return to the previous one. I don't think this is likely in practice, but it is often how "stateful" directives are implemented in assembler.

For example:

// switch to subsection aeabi-feature-and-bits
.aeabi_subsection aeabi-feature-and-bits , 0, 0
.aeabi_attribute Tag_Feature_BTI, 1

// switch to subsection aeabi-pauthabi
.aeabi_subsection aeabi-pauthabi, 1, 0
.aeabi_attribute Tag_PAuth_Platform, 1
.aeabi_attribute Tag_PAuth_Schema, 1

// switch back to subsection aeabi-feature-and-bits
// Note this doesn't create a new subsection.
.aeabi_subsection aeabi-feature-and-bits , 0, 0
.aeabi_attribute Tag_Feature_GCS, 1

There will be two subsections in the ELF file aeabi-feature-and-bits and aeabi-pauthabi.

[smithp35]

The buildattr64.rst doesn't mention assembly syntax as these are not part of the "binary" standard. They are covered for GNU compatible assembley in the rationale document, I'll find a way of covering this there.

Yes the intention is that each uniquely named .aeabi_subsection will be a subsection in the ELF file, with the vendor-name set to the name given in the directive.

[sivan-shani]

The <format-version: > (double '<, >' also around 'version-letter') and <uint32: subsection-length> (single '<, >', not around 'subsection-length') does not indicated a different between the cases.

[smithp35]

The original in addenda32 hard coded A, but mentioned that this was just the current version. I think I'll go back to that form.

[sivan-shani]

A tag with the same name in a different .aeabi_subsections is considered a different tag.

[smithp35]

This part of the document is specifically about encoding and subsections haven't been described yet. I'll see if I can find a place to express this later on in either subsections or tags.

[sivan-shani]

The syntax is not final. There are consideration around whether the values should be optional and with what exact syntax.
Options:

• as it is now, and to denote optional arguments:
  .aeabi_subsection MySectionA 0 -> 'optional'
  .aeabi_subsection MySectionA, 0 -> 'parameter type'
• The syntax might be .aeabi_subsection name [, optional] [, parameter type]
  and to denote optional arguments:
  .aeabi_subsection MySectionA, 0, or:
  .aeabi_subsection MySectionA, ,0
• Having both the arguments being mandatory
• Make encoding and parameter type either both present or absent

[smithp35]

Internally with a conversation with the GNU team we decided that it should be [, optional] I think that while possible to disambiguate between

.aeabi-subsection vendor-name optional

and

.aeabi-subsection vendor-name ,ntbs

Is possible, given that optional and ,ntbs can be digits then it will be difficult for a human to spot.

[sivan-shani]

Edit: reading it again I realize that this part describe the ELF while I thought that it describes the assembly. Therefore some of the comments are irrelevant, but some still makes sense so I figure I better leave them.
If details like vendor name has to be be encoded in the ELF, does it mean that they must also appear in the assembly?
Might it be a good idea to have a clear section regarding assembly, or some related document? even though strictly speaking the ABI is only the binary specification for the ELF file.

Here is a suggestion for this paragraph that reflect my understanding of public subsections: (Edit: in the assembly, maybe this can serve for another document that specify the assembly)

Suggested change

	About public tags
	Public subsections
	Subsections are denoted with the tag: `.aeabi_subsection <name>`
	Public subsections are subsections with <name> starting with a prefix of "aeabi".
	Public subsections have a header that indicates whether the subsection is mandatory or not, and what is the type of the value for all the subsection's attributes.
	A consumer must be able to parse the header of all the mandatory public subsections. A consumer may skip an optional subsection.

Several points:

• The header does not indicates only optionality, but also the attribute's values type.
  Edit: irrelevant to ELF, only to assembly.
• The part about the subsection that do not need to be read is confusing. My understanding of the essence is: If subsection is mandatory, it has to be read, not being able to read it is an error. If it is optional, it can be skipped, from any reason (e.g. not being able to recognize the full vendor-name or any other reason, it is not clear what is special about the vendor-name)
  Edit: This part seems relevant also for ELF.
• Remove the mention of "vendor name" since in actuality sections names does not contained a vendor name.
  Edit: not true to ELF, still, since this data is not in the assembly, how do we know what to put it in the ELF?

[smithp35]

Thanks for the suggestion, I'll have a think about what I can incorporate.

I'm thinking that I could move the assembly support into the main specification as an appendix.

[sivan-shani]

It is unclear to me whether this is true only for optional subsections, only for mandatory subsections, or for both, optional and mandatory and subsections.

[smithp35]

I'll see if I can make the mandatory part clearer. I think an examples section in the rationale would help too. All tags get an implicit default of 0 whether in a mandatory or optional section.

A subsection should be mandatory if a static linker must be able to understand at least one of the tags defined in it to produce correct output. By making the subsection mandatory a static linker is expected to give an error message if it can't process the subsection.

If the subsection is optional and a static linker doesn't understand either the subsection or a tag within it then it can just skip over it. The output will be correct, if not optimal.

[sivan-shani]

Might it be a good idea to call 'tags' -> subsections and therefore 'public tags' -> public subsections?
"Tags" might mean quite a few things.

[smithp35]

I'll see if I can define these terms a bit better. A public tag is just a tag defined in a public (name starting with aeabi) subsection.

I agree that tag is a general term, I've taken a lot of the language from the 32-bit ABI.

[sivan-shani]

Suggested change

	Encoding build attributes are encoded in a vendor-specific section of
	Build attributes are encoded ...

[smithp35]

ACK, I'll adopt in the next patch.

It looks like this partially came from addenda32.rst

Encoding build attributes needs more than a few bits, so we encode them in an vendor-specific section of type SHT_ARM_ATTRIBUTES and name .ARM.attributes (for further details see 

[smithp35]

Thanks for all the comments. I'll aim to get a response tomorrow and I'll post an update to the spec and rationale this week.

[smithp35]

I'll start work on an update tomorrow.

[smithp35]

ACK, I'll adopt in the next patch.

It looks like this partially came from addenda32.rst

Encoding build attributes needs more than a few bits, so we encode them in an vendor-specific section of type SHT_ARM_ATTRIBUTES and name .ARM.attributes (for further details see 

[smithp35]

This part of the document is specifically about encoding and subsections haven't been described yet. I'll see if I can find a place to express this later on in either subsections or tags.

[smithp35]

The original in addenda32 hard coded A, but mentioned that this was just the current version. I think I'll go back to that form.

[smithp35]

The buildattr64.rst doesn't mention assembly syntax as these are not part of the "binary" standard. They are covered for GNU compatible assembley in the rationale document, I'll find a way of covering this there.

Yes the intention is that each uniquely named .aeabi_subsection will be a subsection in the ELF file, with the vendor-name set to the name given in the directive.

[smithp35]

Thanks for the suggestion, I'll have a think about what I can incorporate.

I'm thinking that I could move the assembly support into the main specification as an appendix.

[smithp35]

I'll see if I can make the mandatory part clearer. I think an examples section in the rationale would help too. All tags get an implicit default of 0 whether in a mandatory or optional section.

A subsection should be mandatory if a static linker must be able to understand at least one of the tags defined in it to produce correct output. By making the subsection mandatory a static linker is expected to give an error message if it can't process the subsection.

If the subsection is optional and a static linker doesn't understand either the subsection or a tag within it then it can just skip over it. The output will be correct, if not optimal.

[smithp35]

I'll see if I can define these terms a bit better. A public tag is just a tag defined in a public (name starting with aeabi) subsection.

I agree that tag is a general term, I've taken a lot of the language from the 32-bit ABI.

[smithp35]

Internally with a conversation with the GNU team we decided that it should be [, optional] I think that while possible to disambiguate between

.aeabi-subsection vendor-name optional

and

.aeabi-subsection vendor-name ,ntbs

Is possible, given that optional and ,ntbs can be digits then it will be difficult for a human to spot.

[sivan-shani]

Starting to doubt whether it is a good idea to have optional parameters with default values.
It creates the possibility for user mistakes when writing build attributes as assembly and implementation mistakes.
Are there any real benefits for allowing the omission of parameters?

[smithp35]

Default values are necessary for reasoning about the properties of old object files in new toolchains and vice-versa. If we introduce a new attribute in ABI release N, then toolchains that implement some version prior to N won't have that attribute, toolchains need to be able to reason about that case as we can't guarantee that everyone recompiles all their objects, including all their dependencies, all of the time. An explicit default value of 0 forces the attribute definition to think about backwards compatibility. With the most common kinds of attribute combination being "all objects must have property X to set X in output", "If at least one object has property X to set X in output" or "all objects must have the same property for the combination to be valid" then 0 works well.

[sivan-shani]

Thank you for the reply, it is a bit difficult to understand.

Given old object file O1 which does not have attrX,
Being handled with new object files O2, O3 that has attrX.
The way to handle O1, O2 and O3 together will be implemented in the logic of the handling tool (e.g. linker)
Something like "if no attrx, then..."

It is not so clear how having default values for attrX helps this process.

1. Having default value for attrX can not imply that old object has to be treated like it has the default value, because there is no way to know if it will make sense for future attributes.
2. "An explicit default value of 0 forces the attribute definition to think about backwards compatibility" - Why? when there is an old object involved, it is not the same as having new object with no explicit attribute (see (1)) It is a completely different case that has to be handled anyhow. e.g. even in a world without old object files, there could be a case of a missing attribute, either by design (if the attribute is optional) or mistake (if it is mandatory)

It seems to be completely unrelated?

[sivan-shani]

There is some logic to the idea that the default is '0', and it means that no attribute -> value is '0', and then it imply that all other objects can not have '1'.
However, if this is the intention, do we really want to force all future developers to comply with this logic? It is a bit micromanagy on our part, and it might be better to leave those decisions to the future implementers.
Edit: especially given that the partial order on the attributes could be defined ad-hoc if I am not mistaken.

[sivan-shani]

Sorry, I am gathering my thoughts slowly. Just want to add this:
Handling old object files has to be done anyhow, and I believe that having default values will hardly effect it or will hint anything to those who will implement it, while it will add the need to remember that 1) there are default values and 2) what they are to everyone who read/write assembly, or handle the tools that parse/write assembly/elf. We adding complexity and making things more error prone for benefits that does not seems quite there.

[sivan-shani]

(sorry, a very last comment ) It will be clear from the partial order defined on the attribute, what should be the way to handle the case of missing attribute. (or "what should be the default value for missing attribute") and in the process of implementation and review, it should be done correctly, is there a need to hint it by setting a default value?

[smithp35]

Hope this goes through all the questions:

A given tool will understand a non-strict subset of the documented attributes and documented subsections of the current specification.

A build attributes producer like an assembler or compiler can only set the attributes and subsections it knows about. A build attributes consumer like a linker can only take action (the encoding permits it to read the attributes even if it doesn't understand them) on attributes and subsections it knows about.

On the producer side, the attributes and subsections that the tool doesn't know about will not be present. When the intended value of the attribute is the same as the default then it is up to the producer whether to emit the attribute or leave it out. Personally I would expect tools like compilers to just set the attribute explicitly to 0, however in hand-written assembly I would expect humans to only set the ones they care about. It isn't realistic to expect a human to look up all the attributes in the spec.

On the consumer side it is clear what happens if an attribute is encountered that the consumer doesn't understand. If it is in an optional subsection, it ignores it, if it is an non-optional subsection it gives a diagnostic.

If the consumer encounters an object file that is missing an attribute that it understands it uses the default 0 value for the object. As you suggest the partial order and combination rule will tell the consumer how to update the output value of the attribute.

[sivan-shani]

Thanks, I was confusing Assembly default values, such as subsection header values such as optional/value and ELF default value such as an entire tag.
ELF does not have default values like Assembly, therefore my point was not relevant.