RFC: React/GraphQL GridField

[unclecheese]

Overview

This RFC proposes a series of changes to the API and developer experience to support GridFields rendered with React and composed by GraphQL/Apollo data fetching.

Proof of concept

https://github.com/open-sausages/react-gridfield-poc

Key Challenges

• GraphQL queries are presumed static, bound to components at compile time.
• Mandating that developers to hook into compile time operations is a non-starter (i.e. requiring node)
• While developers will have to write Javascript, GridField remains primarily a PHP API.
• Extensibility to both the UI and data layer are fully supported to the extent that they are in the current implementation
• getCMSFields() is assumed non-deterministic, and therefore a non-qualifier for static analysis or evaluation (i.e. through singletons).

Proposal

Most of this can be done without major changes to the API, but the changes to the developer experience, particularly around getCMSFields will be non-trivial. The fundamental idea behind this proposal is that, while we can't inject at compile time, we can do quite a bit at boot time. This requires that every GridField declares itself on boot and this information is provided to the client to process the necessary transformations in Injector. Because every GridField needs to declare itself, all GridFields must be uniquely identified.

Create a GridField registry

A singleton instance of a GridFieldRegistry is used to store all GridFields in the system by their identifiers. This registry is populated using a method on a common interface, e.g. provideGridFields(GridFieldRegistry $registry), much like providePermissions.

class GridFieldRegistry
{
    public function add(string $identifier, GridField $gridField): void;

    public function getAll(): array;

    public function get(string $id): ?GridFieldRegistration;

}

class GridFieldRegistration
{
    public function __construct(string $identifier, GridField $gridField);
}

Classes that use GridField become providers

class MyDataObject extends DataObject implements GridFieldProvider
{
    public function provideGridFields(GridFieldRegistry $registry)
    {
        $registry->add(
            'MyGridIdentifier',
            GridField::create(
                'Products',
                'View / edit products ',
                $this->Products()
            )
        );
    }
}

The grid field is then fetched from the registry instead of instantiated in getCMSFields().

class MyDataObject extends DataObject implements GridFieldProvider
{
    public function provideGridFields(GridFieldRegistry $registry)
    {
        //...
    }

    public function getCMSFields()
    {
        return FieldList::create(
            TextField::create('Title'),
            Injector::inst()->get(GridFieldRegistry::class)->get('AllNotes')
        );
    } 
}

Ideally, the explicit call to Injector could be made more user-friendly with a trait (e.g. getGridField(string $id). Or even GridField::getByIdentifier($id)

Expose all dataobjects to graphql

A simple ScaffoldingProvider implementation will suffice.

class UniversalScaffolder implements ScaffoldingProvider
{
    public function provideGraphQLScaffolding(SchemaScaffolder $scaffolder)
    {
        foreach(ClassInfo::subclassesFor(DataObject::class) as $dataObjectClass) {
            if ($dataObjectClass === DataObject::class) continue;
            $dataObjectScaffold = $scaffolder
                ->type($dataObjectClass)
                ->addAllFields();
            foreach(OperationScaffolder::getOperations() as $identifier => $class) {
                $dataObjectScaffold->operation($identifier);
            }
        }

        return $scaffolder;
    }
}

A GridField ScaffoldingProvider runs through the registry and adds the requirements of each GridField

class GridFieldScaffolder implements ScaffoldingProvider
{
    public function __construct(GridFieldRegistry $registry);

    public function provideGraphQLScaffolding(SchemaScaffolder $scaffolder)
    {
        foreach ($this->registry->getAll() as $gridFieldRegistration) {
            $gridField = $gridFieldRegistration->getGridField();
            $identifier = $gridFieldRegistration->getIdentifier();
            $scaffolder->query(
                'readGridField' . $identifier,
                $gridField->getList()->dataClass(),
                function ($object, array $args, $context, ResolveInfo $info) use ($gridField) {
                    return $gridField->getList();
                }
            )
        }

        return $scaffolder;
    }
}

This presumes that GridField lists are DataList instances. If they're arbitrary ArrayList / ArrayData compositions, the developer would need to create his or her own custom GraphQL types and use a custom resolver.

Ideally this would be extensible through all the current channels, so that the SortComponent could jump in here and add ->addSortableColumns(array $fields) or something like that.

LeftAndMain provides client configuration about the registered gridfields.

        $combinedClientConfig['gridFieldQueries'] = [];
        $registry = Injector::inst()->get(GridFieldRegistry::class);
        foreach($registry->getAll() as $gridFieldRegistration) {
            $combinedClientConfig['gridFieldQueries'][] = [
                'name' => $gridFieldRegistration->getIdentifier(),
                'fields' => $gridFieldRegistration->getFields(),
                'components' => $gridFieldRegistration->getGridField()->getComponents(),
            ];
        }

getFields() is effectively GridField::getColumns(). getComponents() would be a serialised representation of the component configuration:

[
  { type: 'AddNewButton', position: 'before', title: 'Add a new record' }
]

A boot function transforms all of the GridField registrations with the injectGraphql HOC.

  Config.get('gridFieldQueries').forEach(gridFieldQuery => {
    const { name, fields, components } = gridFieldQuery;
    const query = {
      apolloConfig: {
         // map query data to props
      },
      templateName: READ,
      pluralName: `GridField${name}`,
      // ...
      fields,
    };

    const queryName = `${name}Query`;
    Injector.query.register(queryName, query);
    Injector.transform(
      `gridfield-graphql-${name}`,
      (updater) => {
        updater.component(context, injectGraphql(queryName));
      }
    );
});

This results in the creation of a query readGridField<MyGridFieldIdentifier>. Ideally this would use a custom template, other than read so it could be disambiguated as a GridField-specific operation in the schema, and we could name it something like get<GridFieldIdentifier>Data

GridField declares "slots" for component injection

      <div>
        <BeforeComponents />
        <table>
          <HeaderComponents />
          <tbody />
          <FooterComponents />
        </table>
        <AfterComponents />
      </div>

These slots are registered with Injector. (Injector.component.register('GridFieldBeforeComponentSlot', GridFieldComponentSlot))

The assigned components for each GridField are in the config and injected at boot time

    Injector.transform(
      `gridfield-${name}-components`,
      (updater) => {
        ['Before', 'Header', 'After'].forEach(key => {
          updater.component(
            `GridField${key}Components.${name}`,
            inject(
              appliedComponents,
              (...injectedComponents) => ({
                children: injectedComponents,
              })
            )
          );
        });
      }
    );

GridField components can transform the query via injector.

When the query is injected, the list of components assigned to the GridField is available in the Injector context. This allows components to hook into only those GridField queries that are using the component.

  Config.get('gridFieldQueries').forEach(gridFieldQuery => {
    const {name} = gridFieldQuery;
    Injector.transform(
      `some-graphql-component-${name}`,
      (updater) => {
        updater.query(`${name}Query.SomeComponent`, (manager) => {
          manager.addParam('someArg', 'String');
        });
      }
    );
  });

GridField exposes part of the Apollo API to components via context.

class MyGridFieldComponent extends React.Component {
  handleChange(e) {
    this.context.graphql.refetch({
      someArg: this.state.someValue
    });
  }
}

Considerations

1. Because GridFields are declared at boot time, for the most part they become immutable. Something like this would break the entire mechanism:

Injector::inst()->get(GridFieldRegistry::class)->get('MyGrid')
  ->setList(SomeOtherObject::get());

For this reason, it might make sense to split the GridField API into two sub APIs -- UI and Data. The latter would be frozen after registration. But because UI composition can be conveyed via FormSchema, mutability to UI only (e.g. ->setTitle()) should be supported as it is now.

2. Much of the above proposal relies on naming conventions, which are quite fragile. We should back these up with selectors and generators to keep things consistent, or add some other layer of abstraction so it doesn't feel so flimsy.

3. Many of the existing GridField components can keep much of what they have and remain relevant. Things like DataManipulator and ColumnProvider really have no concerns about how the data is fetched.

4. Special attention should be given to support the 80% use case, which is CRUD operations for $this->SomeHasManyRelationship(). If this gets clunky, we'll have a lot of unhappy developers.

5. Enforcing that GridField is not instantiated in getCMSFields could be tricky. Heavy-handed solution would be to make Field() throw. I don't believe there is any case where server-side rendering of a GridField would be appropriate or useful.

[chillu]

Discussed with Aaron this morning, he'll try another approach which doesn't come with so much long-term pain (diverging GraphQL APIs between CMS usage and a general-purpose "Content API"). The key point here is figuring out how to avoid Apollo knowing about all GraphQL queries at boot time (through it's HoC). Looks like Apollo v2 gives us more options around this.

[patricknelson]

To start out, I'd like to preface this by letting you know ahead of time that I'm not deeply familiar with React (and thus HoC's) nor Apollo and only have a basic understanding of GraphQL. So this will be from the perspective of a heavy and user of fairly advanced functionality the existing GridField's in SS v3 (which are very similar in the current v4 IIRC). That should help you understand my perspective.

Regarding this comment by @chillu on another thread #463 (comment) about the move of GridField to React, I wasn't sure if it was appropriate to comment there, so I'm responding here to close the loop a bit (since it seems more relevant here).

I think solving the DataList vs. GraphQL vs. GridField architecture is actually more important than how we represent components in the form schema, and what APIs we provide to build and customise these (lots of this has already been solved by the likes of Griddle).

Can you elaborate on what is meant by "DataList vs. GraphQL vs. GridField" here? Or are you simply calling out that the new GridField architecture may impact how DataList's are used given the leveraging of a newly introduced GraphQL API? At first glance to me (subjective), it implies that this is potentially an "either-or" situation.

Also:

getCMSFields() is assumed non-deterministic, and therefore a non-qualifier for static analysis or evaluation (i.e. through singletons).

But so is ->provideGraphQLScaffolding() since it's also an instance-level method. Do you mean to say that this makes it easier to resolve GridField's by specializing on a new method or is this something much deeper (i.e. implying it should be static and dependent purely on arguments and not object state, etc). Obviously a static declaration isn't possible, since the GridField returned/registered by this method is already subject to change after registration due to the dependent DataList populating it, which can change as execution progresses.

Internally, I already have static API's that return pre-configured GridField's because they are necessary to reuse throughout the system (e.g. MyObj::getGrid(...)) where either the class called, via late static binding, or the parameters passed will impact the construction of the returned GridField in some way. So, it'd actually be pretty useful to expose a more standardized API for returning GridField's. Due to the dynamic nature of the system and all the potential dependencies, I don't see why this needs to be static/deterministic (it already requires an identifier and must include a dynamic DataList. For example, GridField's produced by the 10 or so internal static methods in my current system can vary on:

• DataList populating it (important, see below)
• The parent object (i.e. the one that "has many" of the items populating the GridField itself)
• Grid field title
• Sort column (if not sortable via clickable header fields)
• Other input necessary for initializing values for new items added on that grid field (such as the parent ID, a category and etc) prior to first save.
  • Currently this requires a customized version of GridFieldDetailForm since the request used to create the form that then exposes the item to be edited must can initialized (in my implementation) with a callback that is then executed to initialize the DataObject's fields (via GridFieldDetailForm_ItemRequest)
• And other factors that can only be known at run time

So, with that said: Can you please explain precisely why (at least in the current proposal) the API must be architected such that all GridField's (which come bundled with both their data and configuration) and their variations must be defined ahead of time instead of fetched when needed (i.e. when defined via ID in the ->getCMSField() list to notify the framework that a particular grid with a particular configuration is now needed)? Is this a constraint somehow imposed due to the use of GraphQL? Is it because GraphQL (via Apollo, which I'm not familiar with) requires so much defined ahead of time, or the use of the HoC pattern in React or something similar?

That leads me to my next question/suggest, I'd like to underscore this point:

For this reason, it might make sense to split the GridField API into two sub APIs -- UI and Data.

We already have two API's at the PHP level, ->getConfig() and ->getList(). I'd like to clarify that architecturally maybe this should be exposed, maybe at the framework/admin level, two API's (fetched via HTTP, be it over GraphQL, REST or otherwise) which return these two things. And, architecturally, instead of requiring a sort of "registration" based PHP API where users are expected to register idempotent GridField's (not just config, but also underlying data) they instead simply expose their GridField's which are potentially fetched ahead of time for React/HoC registration purposes but are also fetched again for data purposes.

So, to wrap up (and maybe over-simplify a tiny bit) I really like the ideas of:

• Using React to define and fully render the GridField's client-side instead of server-side
• Using GraphQL and/or Apollo to fetch the configuration and the data for those GridFields
• Being able to still configure the GridField such that I can intercept how items are added (see current convoluted method above)

I just think that we should carefully consider the separation of concerns in config vs. data, maybe config is more static, immutable and etc, but the data underpinning is a strictly separate concern and probably handled with subsequent requests. Also, this implies potentially adding validation to ensure GridFieldConfig hasn't changed on the subsequent data HTTP requests, but that's why there's some room maybe for better defining a more explicit config (statically, maybe even declarative) vs. a dynamic data API (called dynamically) a bit lower down.

Some complexities: Configuration may need to allow for parameters, such as in the point above about being able to define a parent or category ID. Here's an extremely simple example (->whenEditing() is a method exposed via Injector):

// Setup association ahead of time while editing.
$tabsGridConfig->whenEditing(function(TabListItem $item) {
	$item->ContentSectionID = $this->ID;
});

[unclecheese]

Thanks for taking the time to read through all this and provide your feedback, @patricknelson. Lots to respond to, but I'll just hit you with the high level.

• In Ingo's last comment, you'll see we discussed a lot of this offline and determined that we can inject queries more dynamically than we thought due to the changes in Apollo 2 (now quite old) that offer Query components that can accept graphql queries as props, rather than forcing the developer to inject them with composition (e.g. bound to compile time). This means we can pass the query via the FormSchema API. It probably means creating some abstraction layer for building graphql queries in Silverstripe CMS, though. Not necessarily a terrible thing, and necessary for extensibility.

• provideGraphqlScaffolding() is non-deterministic: You can probably overlook this. provideGraphqlScaffolding will not play into the interaction of admin components with the graphql API. The basic idea here is that we expose everything to the admin/graphql API, so we have one deterministic function in some central place, say LeftAndMain, that just exposes all fields and all operations for every dataobject. But for instance-level provideGraphqlScaffolding definitions in your dataobjects, that only plays into your custom graphql schemas. I suppose you could still use them to affect admin, but there wouldn't be much point, since the admin just comes in with a sledgehammer and exposes everything.

• UI/Data split: I don't think getConfig() and getList() is an aspect-oriented split, per se. getConfig() can be loaded with concerns about data (e.g. any data manipulator). What this is really getting at is what we can deliver over FormSchema calls and what will be constrained to the graphql calls.