[mlir] Add property combinators, initial ODS support (#94732)

While we have had a Properties.td that allowed for defining
non-attribute-backed properties, such properties were not plumbed
through the basic autogeneration facilities available to attributes,
forcing those who want to migrate to the new system to write such code
by hand.

## Potentially breaking changes

- The `setFoo()` methods on `Properties` struct no longer take their
inputs by const reference. Those wishing to pass non-owned values of a
property by reference to constructors and setters should set the
interface type to `const [storageType]&`
- Adapters and operations now define getters and setters for properties
listed in ODS, which may conflict with custom getters.
- Builders now include properties listed in ODS specifications,
potentially conflicting with custom builders with the same type
signature.

## Extensions to the `Property` class

This commit  adds several fields to the `Property` class, including:
- `parser`, `optionalParser`, and `printer` (for parsing/printing
properties of a given type in ODS syntax)
- `storageTypeValueOverride`, an extension of `defaultValue` to allow
the storage and interface type defaults to differ
- `baseProperty` (allowing for classes like `DefaultValuedProperty`)

Existing fields have also had their documentation comments updated.

This commit does not add a `PropertyConstraint` analogous to
`AttrConstraint`, but this is a natural evolution of the work here.

This commit also adds the concrete property kinds `I32Property`,
`I64Property`, `UnitProperty` (and special handling for it like for
UnitAttr), and `BoolProperty`.

## Property combinators

`Properties.td` also now includes several ways to combine properties.

One is `ArrayProperty<Property elem>`, which now stores a
variable-length array of some property as
`SmallVector<elem.storageType>` and uses `ArrayRef<elem.storageType>` as
its interface type. It has `IntArrayProperty` subclasses that change its
conversion to attributes to use `DenseI[N]Attr`s instead of an
`ArrayAttr`.

Similarly, `OptionalProperty<Property p>` wraps a property's storage in
`std::optional<>` and adds a `std::nullopt` default value. In the case
where the underlying property can be parsed optionally but doesn't have
its own default value, `OptionalProperty` can piggyback off the optional
parser to produce a cleaner syntax, as opposed to its general form,
which is either `none` or `some<[value]>`.

(Note that `OptionalProperty` can be nested if desired).

  ## Autogeneration changes

Operations and adaptors now support getters and setters for properties
like those for attributes. Unlike for attributes, there aren't separate
value and attribute forms, since there is no `FooAttr()` available for a
`getFooAttr()` to return.

The largest change is to operation formats. Previously, properties could
only be used in custom directives. Now, they can be used anywhere an
attribute could be used, and have parsers and printers defined in their
tablegen records.

These updates include special `UnitProperty` logic like that used for
`UnitAttr`.

## Misc.

Some attempt has been made to test the new functionality.

This commit takes tentative steps towards updating the documentation to
account for properties. A full update will be in order once any followup
work has been completed and the interfaces have stabilized.

---------

Co-authored-by: Mehdi Amini <[email protected]>
Co-authored-by: Christian Ulmann <[email protected]>

mlir/docs/DefiningDialects/Operations.md

@@ -101,6 +101,9 @@ their semantics via a special [TableGen backend][TableGenBackend]:
 *   The `AttrConstraint` class hierarchy: They are used to specify the
     constraints over attributes. A notable subclass hierarchy is `Attr`, which
     stands for constraints for attributes whose values are of common types.
+*   The `Property` class hierarchy: They are used to specify non-attribute-backed
+    properties that are inherent to operations. This will be expanded to a
+    `PropertyConstraint` class or something similar in the future.
 
 An operation is defined by specializing the `Op` class with concrete contents
 for all the fields it requires. For example, `tf.AvgPool` is defined as
@@ -172,9 +175,9 @@ understanding the operation.
 
 ### Operation arguments
 
-There are two kinds of arguments: operands and attributes. Operands are runtime
-values produced by other ops; while attributes are compile-time known constant
-values, including two categories:
+There are three kinds of arguments: operands, attributes, and properties.
+Operands are runtime values produced by other ops; while attributes and properties
+are compile-time known constant values, including two categories:
 
 1.  Natural attributes: these attributes affect the behavior of the operations
     (e.g., padding for convolution);
@@ -187,22 +190,27 @@ values, including two categories:
     even though they are not materialized, it should be possible to store as an
     attribute.
 
-Both operands and attributes are specified inside the `dag`-typed `arguments`,
-led by `ins`:
+Properties are similar to attributes, except that they are not stored within
+the MLIR context but are stored inline with the operation.
+
+Operands, attributes, and properties are specified inside the `dag`-typed
+`arguments`, led by `ins`:
 
 ```tablegen
 let arguments = (ins
   <type-constraint>:$<operand-name>,
   ...
   <attr-constraint>:$<attr-name>,
   ...
+  <property-constraint>:$<property-name>,
 );
 ```
 
 Here `<type-constraint>` is a TableGen `def` from the `TypeConstraint` class
 hierarchy. Similarly, `<attr-constraint>` is a TableGen `def` from the
-`AttrConstraint` class hierarchy. See [Constraints](#constraints) for more
-information.
+`AttrConstraint` class hierarchy and `<property-constraint>` is a subclass
+of `Property` (though a `PropertyConstraint` hierarchy is planned).
+See [Constraints](#constraints) for more information.
 
 There is no requirements on the relative order of operands and attributes; they
 can mix freely. The relative order of operands themselves matters. From each
@@ -324,6 +332,18 @@ Right now, the following primitive constraints are supported:
 
 TODO: Design and implement more primitive constraints
 
+#### Optional and default-valued properties
+
+To declare a property with a default value, use `DefaultValuedProperty<..., "...">`.
+If the property's storage data type is different from its interface type,
+for example, in the case of array properties (which are stored as `SmallVector`s
+but use `ArrayRef` as an interface type), add the storage-type equivalent
+of the default value as the third argument.
+
+To declare an optional property, use `OptionalProperty<...>`.
+This wraps the underlying property in an `std::optional` and gives it a
+default value of `std::nullopt`.
+
 #### Combining constraints
 
 `AllAttrOf` is provided to allow combination of multiple constraints which
@@ -429,6 +449,8 @@ def MyOp : ... {
     I32Attr:$i32_attr,
     F32Attr:$f32_attr,
     ...
+    I32Property:$i32_prop,
+    ...
   );
 
   let results = (outs
@@ -453,7 +475,8 @@ static void build(OpBuilder &odsBuilder, OperationState &odsState,
 static void build(OpBuilder &odsBuilder, OperationState &odsState,
                   Type i32_result, Type f32_result, ...,
                   Value i32_operand, Value f32_operand, ...,
-                  IntegerAttr i32_attr, FloatAttr f32_attr, ...);
+                  IntegerAttr i32_attr, FloatAttr f32_attr, ...,
+                  int32_t i32_prop);
 
 // Each result-type/operand/attribute has a separate parameter. The parameters
 // for attributes are raw values unwrapped with mlir::Attribute instances.
@@ -462,13 +485,15 @@ static void build(OpBuilder &odsBuilder, OperationState &odsState,
 static void build(OpBuilder &odsBuilder, OperationState &odsState,
                   Type i32_result, Type f32_result, ...,
                   Value i32_operand, Value f32_operand, ...,
-                  APInt i32_attr, StringRef f32_attr, ...);
+                  APInt i32_attr, StringRef f32_attr, ...,
+                  int32_t i32_prop, ...);
 
 // Each operand/attribute has a separate parameter but result type is aggregate.
 static void build(OpBuilder &odsBuilder, OperationState &odsState,
                   TypeRange resultTypes,
                   Value i32_operand, Value f32_operand, ...,
-                  IntegerAttr i32_attr, FloatAttr f32_attr, ...);
+                  IntegerAttr i32_attr, FloatAttr f32_attr, ...,
+                  int32_t i32_prop, ...);
 
 // All operands/attributes have aggregate parameters.
 // Generated if return type can be inferred.
@@ -921,8 +946,10 @@ optional-group: `(` then-elements `)` (`:` `(` else-elements `)`)? `?`
 The elements of an optional group have the following requirements:
 
 *   The first element of `then-elements` must either be a attribute, literal,
-    operand, or region.
+    operand, property, or region.
     -   This is because the first element must be optionally parsable.
+    -   If a property is used, it must have an `optionalParser` defined and have a
+        default value.
 *   Exactly one argument variable or type directive within either
     `then-elements` or `else-elements` must be marked as the anchor of the
     group.
@@ -984,6 +1011,8 @@ foo.op is_read_only
 foo.op
 ```
 
+The same logic applies to a `UnitProperty`.
+
 ##### Optional "else" Group
 
 Optional groups also have support for an "else" group of elements. These are
@@ -1026,6 +1055,8 @@ to:
 1.  All operand and result types must appear within the format using the various
     `type` directives, either individually or with the `operands` or `results`
     directives.
+1.  Unless all non-attribute properties appear in the format, the `prop-dict`
+    directive must be present.
 1.  The `attr-dict` directive must always be present.
 1.  Must not contain overlapping information; e.g. multiple instances of
     'attr-dict', types, operands, etc.

mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td

@@ -59,22 +59,9 @@ class LLVM_IntArithmeticOpWithOverflowFlag<string mnemonic, string instName,
                                    list<Trait> traits = []> :
     LLVM_ArithmeticOpBase<AnySignlessInteger, mnemonic, instName,
     !listconcat([DeclareOpInterfaceMethods<IntegerOverflowFlagsInterface>], traits)> {
-  dag iofArg = (ins EnumProperty<"IntegerOverflowFlags">:$overflowFlags);
+  dag iofArg = (ins EnumProperty<"IntegerOverflowFlags", "", "IntegerOverflowFlags::none">:$overflowFlags);
   let arguments = !con(commonArgs, iofArg);
 
-  let builders = [
-    OpBuilder<(ins "Type":$type, "Value":$lhs, "Value":$rhs,
-                   "IntegerOverflowFlags":$overflowFlags), [{
-      $_state.getOrAddProperties<Properties>().overflowFlags = overflowFlags;
-      build($_builder, $_state, type, lhs, rhs);
-    }]>,
-    OpBuilder<(ins "Value":$lhs, "Value":$rhs,
-                   "IntegerOverflowFlags":$overflowFlags), [{
-      $_state.getOrAddProperties<Properties>().overflowFlags = overflowFlags;
-      build($_builder, $_state, lhs, rhs);
-    }]>
-  ];
-
   string mlirBuilder = [{
     auto op = $_builder.create<$_qualCppClassName>($_location, $lhs, $rhs);
     moduleImport.setIntegerOverflowFlags(inst, op);

mlir/include/mlir/IR/ODSSupport.h

@@ -33,6 +33,37 @@ convertFromAttribute(int64_t &storage, Attribute attr,
 /// Convert the provided int64_t to an IntegerAttr attribute.
 Attribute convertToAttribute(MLIRContext *ctx, int64_t storage);
 
+/// Convert an IntegerAttr attribute to an int32_t, or return an error if the
+/// attribute isn't an IntegerAttr. If the optional diagnostic is provided an
+/// error message is also emitted.
+LogicalResult
+convertFromAttribute(int32_t &storage, Attribute attr,
+                     function_ref<InFlightDiagnostic()> emitError);
+
+/// Convert the provided int32_t to an IntegerAttr attribute.
+Attribute convertToAttribute(MLIRContext *ctx, int32_t storage);
+
+/// Extract the string from `attr` into `storage`. If `attr` is not a
+/// `StringAttr`, return failure and emit an error into the diagnostic from
+/// `emitError`.
+LogicalResult
+convertFromAttribute(std::string &storage, Attribute attr,
+                     function_ref<InFlightDiagnostic()> emitError);
+
+/// Convert the given string into a StringAttr. Note that this takes a reference
+/// to the storage of a string property, which is an std::string.
+Attribute convertToAttribute(MLIRContext *ctx, const std::string &storage);
+
+/// Extract the boolean from `attr` into `storage`. If `attr` is not a
+/// `BoolAttr`, return failure and emit an error into the diagnostic from
+/// `emitError`.
+LogicalResult
+convertFromAttribute(bool &storage, Attribute attr,
+                     function_ref<InFlightDiagnostic()> emitError);
+
+/// Convert the given string into a BooleanAttr.
+Attribute convertToAttribute(MLIRContext *ctx, bool storage);
+
 /// Convert a DenseI64ArrayAttr to the provided storage. It is expected that the
 /// storage has the same size as the array. An error is returned if the
 /// attribute isn't a DenseI64ArrayAttr or it does not have the same size. If
@@ -49,9 +80,24 @@ LogicalResult
 convertFromAttribute(MutableArrayRef<int32_t> storage, Attribute attr,
                      function_ref<InFlightDiagnostic()> emitError);
 
+/// Convert a DenseI64ArrayAttr to the provided storage, which will be
+/// cleared before writing. An error is returned and emitted to the optional
+/// `emitError` function if the attribute isn't a DenseI64ArrayAttr.
+LogicalResult
+convertFromAttribute(SmallVectorImpl<int64_t> &storage, Attribute attr,
+                     function_ref<InFlightDiagnostic()> emitError);
+
+/// Convert a DenseI32ArrayAttr to the provided storage, which will be
+/// cleared before writing. It is expected that the storage has the same size as
+/// the array. An error is returned and emitted to the optional `emitError`
+/// function if the attribute isn't a DenseI32ArrayAttr.
+LogicalResult
+convertFromAttribute(SmallVectorImpl<int32_t> &storage, Attribute attr,
+                     function_ref<InFlightDiagnostic()> emitError);
+
 /// Convert the provided ArrayRef<int64_t> to a DenseI64ArrayAttr attribute.
 Attribute convertToAttribute(MLIRContext *ctx, ArrayRef<int64_t> storage);
 
 } // namespace mlir
 
-#endif // MLIR_IR_ODSSUPPORT_H
+#endif // MLIR_IR_ODSSUPPORT_H