Devise an explicit convention to document which parts of pna.p4 are intended to be vendor-customizable vs. not

[jafingerhut]

psa.p4 and pna.p4 both have a section in them today that the PSA authors have always considered to be a portion that vendors SHOULD modify and customize for their target device. That is the part that defines the data plane bit widths of a small collection of types such as PortId_t and a few others.

Here are the relevant lines of the psa.p4 include file: https://github.com/p4lang/p4-spec/blob/main/p4-16/psa/psa.p4#L40-L60

There are similar sections in the pna.p4 include file that were originally copies from psa.p4, then modified to add a few types and maybe to remove a few types.

Why? Because P4 source code is quite explicit when it comes to things like bit widths of types, and it seems about a 0% chance that every target device will have the precise same bit width for these types of things. Even two target devices made by the same vendor in different years seem likely to change in at least some of those bit widths.

There are certainly other ways to address this without having a section of the pna.p4 include file that target device vendors are expected to customize, but this is at least one way.

Another possible thing that seems likely that a vendor might want to customize would be to add vendor- or target-specific annotations to some of the definitions in pna.p4. We could perhaps try to combine the union of what all vendors want in a standard public pna.p4 include file, but then merging updates into that file becomes a hindrance to vendors releasing their software development tools.

The purpose of this issue is (a) to verify that vendor-specific and/or target-specific portions of the pna.p4 include file seem technically reasonable, and (b) if yes, create some documentation that makes it clear to vendors and users of PNA devices exactly what parts of the pna.p4 include file are expected to differ from target to target, vs. which parts we expect to remain the same.