feature(core): Import / Export capabilities between CTO files and metamodel

[jeromesimeon]

Description

This PR includes new code to export and export CTO files (in concerto syntax) into Accord Project's concerto metamodel JSON objects.

Changes

• Add utility to import CTO file into Concerto MetaModel v0.2.0
• Add utility to export Concerto MetaModel v0.2.0 back into CTO syntax
• Add concerto transform CLI command to convert between .json (metamodel) and .cto files

Flags

• Opening this for early review
• Some things that should be done:
  • Remove spurious import to concerto.* base classes in the generated metamodel
  • Add validation against the metamodel itself (needs to be published)
  • Should be tested against the new Model Builder (which will need a few changes due to meta model changes)
  • Add missing support for default values
  • Need to update the typescript definitions
  • Add support for name resolution
  • Add missing support for decorators
  • Add missing support for field validators / regex
  • Improve the CLI for metadata import/export
  • Add a bit more testing, notably for error cases

[jeromesimeon]

Note. Here is a mini demo:

From CTO to metamodel:

zsh-5.8$ ../concerto-cli/index.js transform --input ./test/data/model/person.cto
{
  "$class": "concerto.metamodel.ModelFile",
  "namespace": "test.person",
  "imports": [
    {
      "namespace": "org.accordproject.time",
      "$class": "concerto.metamodel.ImportAll",
      "uri": "https://models.accordproject.org/[email protected]"
    }, 
    {
      "namespace": "org.accordproject.time",
      "$class": "concerto.metamodel.ImportType",
      "identifier": {
        "$class": "concerto.metamodel.TypeIdentifier",
        "name": "TemporalUnit"
      },
      "uri": "https://models.accordproject.org/[email protected]"
    }
  ],
  "declarations": [
    {
      "$class": "concerto.metamodel.EnumDeclaration",
      "name": "Gender",
      "isAbstract": false,
      "fields": [
        {
          "$class": "concerto.metamodel.EnumFieldDeclaration",
          "name": "MALE",
          "isArray": false,
          "isOptional": false
        }, 
        {
          "$class": "concerto.metamodel.EnumFieldDeclaration",
          "name": "FEMALE",
          "isArray": false,
          "isOptional": false
        }, 
        {
          "$class": "concerto.metamodel.EnumFieldDeclaration",
          "name": "OTHER",
          "isArray": false,
          "isOptional": false
        }
      ]
    }, 
    {
      "$class": "concerto.metamodel.ParticipantDeclaration",
      "name": "Individual",
      "isAbstract": true,
      "fields": []
    }, 
    {
      "$class": "concerto.metamodel.ParticipantDeclaration",
      "name": "Person",
      "isAbstract": false,
      "superType": {
        "$class": "concerto.metamodel.TypeIdentifier",
        "name": "Individual"
      },
      "fields": [
        {
          "name": "firstName",
          "isArray": false,
          "isOptional": false,
          "$class": "concerto.metamodel.StringFieldDeclaration"
        }, 
        {
          "name": "lastName",
          "isArray": false,
          "isOptional": false,
          "$class": "concerto.metamodel.StringFieldDeclaration"
        }, 
        {
          "name": "address",
          "isArray": false,
          "isOptional": false,
          "$class": "concerto.metamodel.ObjectFieldDeclaration",
          "type": {
            "$class": "concerto.metamodel.TypeIdentifier",
            "name": "Address"
          }
        }, 
        {
          "$class": "concerto.metamodel.RelationshipDeclaration",
          "type": {
            "$class": "concerto.metamodel.TypeIdentifier",
            "name": "Address"
          },
          "name": "address2",
          "isArray": false,
          "isOptional": false
        }, 
        {
          "$class": "concerto.metamodel.RelationshipDeclaration",
          "type": {
            "$class": "concerto.metamodel.TypeIdentifier",
            "name": "Address"
          },
          "name": "address3",
          "isArray": true,
          "isOptional": false
        }, 
        {
          "name": "gender",
          "isArray": false,
          "isOptional": false,
          "$class": "concerto.metamodel.ObjectFieldDeclaration",
          "type": {
            "$class": "concerto.metamodel.TypeIdentifier",
            "name": "Gender"
          }
        }, 
        {
          "name": "dob",
          "isArray": false,
          "isOptional": true,
          "$class": "concerto.metamodel.DateTimeFieldDeclaration"
        }
      ]
    }, 
    {
      "$class": "concerto.metamodel.ParticipantDeclaration",
      "name": "Employee",
      "isAbstract": false,
      "superType": {
        "$class": "concerto.metamodel.TypeIdentifier",
        "name": "Person"
      },
      "fields": [
        {
          "name": "company",
          "isArray": false,
          "isOptional": false,
          "$class": "concerto.metamodel.StringFieldDeclaration"
        }, 
        {
          "name": "onLeave",
          "isArray": false,
          "isOptional": false,
          "$class": "concerto.metamodel.BooleanFieldDeclaration"
        }
      ]
    }, 
    {
      "$class": "concerto.metamodel.ConceptDeclaration",
      "name": "Address",
      "isAbstract": false,
      "identified": {
        "$class": "concerto.metamodel.Identified"
      },
      "fields": [
        {
          "name": "zip",
          "isArray": false,
          "isOptional": false,
          "$class": "concerto.metamodel.IntegerFieldDeclaration"
        }, 
        {
          "name": "zip2",
          "isArray": false,
          "isOptional": false,
          "$class": "concerto.metamodel.LongFieldDeclaration"
        }, 
        {
          "name": "zip3",
          "isArray": false,
          "isOptional": false,
          "$class": "concerto.metamodel.DoubleFieldDeclaration"
        }, 
        {
          "name": "city",
          "isArray": false,
          "isOptional": false,
          "$class": "concerto.metamodel.StringFieldDeclaration"
        }, 
        {
          "name": "country",
          "isArray": false,
          "isOptional": false,
          "$class": "concerto.metamodel.StringFieldDeclaration"
        }, 
        {
          "name": "street",
          "isArray": true,
          "isOptional": false,
          "$class": "concerto.metamodel.StringFieldDeclaration"
        }
      ]
    }, 
    {
      "$class": "concerto.metamodel.AssetDeclaration",
      "name": "A",
      "isAbstract": false,
      "fields": []
    }, 
    {
      "$class": "concerto.metamodel.EventDeclaration",
      "name": "E",
      "isAbstract": false,
      "fields": []
    }, 
    {
      "$class": "concerto.metamodel.TransactionDeclaration",
      "name": "T",
      "isAbstract": false,
      "identified": {
        "$class": "concerto.metamodel.IdentifiedBy",
        "name": "id"
      },
      "fields": [
        {
          "name": "id",
          "isArray": false,
          "isOptional": false,
          "$class": "concerto.metamodel.StringFieldDeclaration"
        }
      ]
    }
  ]
}

From MetaModel back to CTO:

zsh-5.8$ ../concerto-cli/index.js transform --input ./test/data/model/person.json 
1:13:12 PM - INFO: namespace test.person

import org.accordproject.time.* from https://models.accordproject.org/[email protected]
import org.accordproject.time.TemporalUnit from https://models.accordproject.org/[email protected]

enum Gender {
  o MALE
  o FEMALE
  o OTHER
}

abstract participant Individual {
}

participant Person extends Individual {
  o String firstName
  o String lastName
  o Address address
  --> Address address2
  --> Address[] address3
  o Gender gender
  o DateTime dob optional
}

participant Employee extends Person {
  o String company
  o Boolean onLeave
}

concept Address identified {
  o Integer zip
  o Long zip2
  o Double zip3
  o String city
  o String country
  o String[] street
}

asset A {
}

event E {
}

transaction T identified by id {
  o String id
}

[dselman]

LKTM. I don't have anything to add to your existing TODOs and task list. This is really exciting for model editors!

[jeromesimeon]

@dselman @mttrbrts Now with validation:

zsh-5.8$ ../concerto-cli/index.js transform --input test/data/model/person.json
11:52:30 AM - ERROR: Unexpected properties for type concerto.metamodel.ParticipantDeclaration: supertype

and with default values:

zsh-5.8$ ../concerto-cli/index.js transform --input test/data/model/person.json
11:52:48 AM - INFO: namespace test.person

import org.accordproject.time.* from https://models.accordproject.org/[email protected]
import org.accordproject.time.TemporalUnit from https://models.accordproject.org/[email protected]

enum Gender {
  o MALE
  o FEMALE
  o OTHER
}

abstract participant Individual {
}

participant Person extends Individual {
  o String firstName
  o String lastName
  o Address address
  o Address address2 default="USAddress"
  --> Address address3
  --> Address[] address4
  --> Address address5 optional
  o Gender gender
  o DateTime dob optional
}

participant Employee extends Person {
  o String company
  o Boolean onLeave
}

concept Address identified {
  o Integer zip default=10001
  o Long zip2 default=10001
  o Double zip3 default=10001.0
  o String city default="NYC"
  o String country default="USA"
  o String[] street
  o Boolean isResidential default=true
  o Boolean isPrivate default=false
}

concept USAddress extends Address {
  o Integer zip4
  o Long zip5
  o Double zip6
}

asset A {
}

event E {
}

transaction T identified by id {
  o String id
}