Please review the Background to better understand the concepts behind this repo.
The component templates directory may also be a simple way to get started making different types of Meta Nodes.
Anyone who wants to! Please create a pull request to add new content.
Below is a guide to add a component to the playbook. You can additionally reuse the existing content in the registry and augment it for your own purposes.
- Follow the Installation Guide to Install the system dependencies required if they are not already installed -- these are:
- Visual Studio Code
- NodeJS
- Git
- Python 3 (optional)
- Clone this repository and checkout a branch for your work.
git clone https://github.com/MaayanLab/Playbook-Workflow-Builder/ cd Playbook-Workflow-Builder git checkout -b my-new-component - Open up the repository directory with your editor.
- Install dependencies and start the development webui, this provides tools testing and debugging metanodes, the webserver will "hot-reload" when files are modified.
# install dependencies npm i # start dev server npm run dev
- (OPTIONAL) Install existing python dependencies to execute some of the existing components.
# collect requirements.txt from components into one npm run codegen:requirements # install them pip install -r requirements.txt
- Add new components in directories under
components/, potentially copying from an existing component. After adding a new component directory, be sure to executenpm run codegen:componentswhich adds it to the full graph. Avoid adding your components to existing files created by someone else, opt instead for a new directory. - Develop, test, and document your component,
index.tsxshould ultimately export your component'smetanodes, see below for information describing how different types of Meta Nodes should be implemented. - Submit a pull request against the main branch.
Components should have their own directory and minimally contain an index.ts (typescript) or index.tsx (typescript+react) file and a package.json. See subsequent sections about index.tsx depending on your MetaNode type.
The package.json is a standard by the javascript ecosystem and is used to capture the name (which should be the same as the directory), version, license, author, contributors, and npm dependencies.
components/{mycomponent}/package.json
{
"name": "mycomponent",
"version": "1.0.0",
"license": "CC-BY-NC-SA-4.0",
"author": "Daniel J. B. Clarke <[email protected]>",
"contributors": [],
"private": true,
"dependencies": {},
"devDependencies": {}
}Each data type has a view, the view is a react component capable of meaningfully visualizing a given data type. Some views simply show the data, such as Gene, while others are more elaborate, such as PlotlyPlot which render a plot.
Its possible to write one from scratch, especially with the help of the typescript tab completion, but a view's basic structure is as follows:
components/{DataType}/index.tsx
import React from 'react'
import { MetaNode } from '@/spec/metanode'
import { z } from 'zod'
// a zod type contract describing your data type
// this is the "shape" of your data type
export const DataType = z.object({
mydatatype: z.array(z.string())
})
export const Data = MetaNode('Data')
// This extra metadata will be used by the ultimate website, types should not have spaces or special symbols
// but labels can contain whatever. We may have additional attributes here in the future including
// icons, version, authorship information and more.
.meta({
label: 'My Data Type Human Label',
description: 'A short description for my data type',
})
// The codec is responsible for the conversion of datatype => string and string => datatype
// with validation. zod-described types should be json serializable
.codec(DataType)
// The view function is a react component for visualizing your data, the type will be the same
// as the type your provided to the codec, i.e. data has the properties defined in DataType
.view(data => (
<div>{JSON.stringify(data)}</div>
))
// Finalize the metanode (currently does nothing, in the future might perform some additional validation)
.build()Each process has input types, an output type and an implementation. The two main process types are prompts and resolvers.
A prompt allows the user to have control of the resulting output, and relies on the user to make some kind of decision or input some kind of data. If you only intend to visualize information without an output, you're looking for a DataType. You may need to create a resolver to resolve an existing datatype into a new datatype for which your specific visualization is attached.
components/{PromptName}/index.tsx
import React from 'react'
import { MetaNode } from '@/spec/metanode'
import { GeneTerm } from '@/components/core/term'
export const PromptName = MetaNode('PromptName')
// As with data types, we have metadata for the process
.meta({
label: 'Do something',
description: 'A useful description',
})
// prompts *can* also take inputs like resolvers
.inputs()
.output(GeneTerm)
// the prompt function is a react component responsible for constructing the output based on
// user interaction. This output should be provided to the `submit` function passed as an argument
.prompt(props => {
// in practice, prompts may be modified after having already been submitted, thus
// the component should restore state if output is not undefined
const [gene, setGene] = React.useState(props.output || '')
return (
<div>
<input value={gene} onChange={evt => setGene(evt.target.value)} />
<button onClick={evt => {
// the submit callback produces the output of this node
props.submit(gene)
}}>Submit</button>
</div>
)
})
.story(props => ({
abstract: `The start with a gene${props.output ? ` ${props.output}` : ''} from the user.`,
}))
.build()A resolver is a data augmentation step which does not require direct user input. They can be made for APIs or data transformations.
components/{ResolverName}/index.ts
import { MetaNode } from '@/spec/metanode'
import { Gene } from '@/components/Gene'
export const ResolverName = MetaNode('ResolverName')
.meta({
label: 'ResolverName',
description: 'My resolver description',
})
// the map here uses the key which will be used in the resolve props, so
// because a Gene metanode is referenced with the key "gene",
// props.inputs.gene will contain data which conforms to the Gene codec specification
.inputs({ gene: Gene })
.output(Gene)
// resolvers return a promise, they should resolve to data in a form that is compatible with
// the output codec specification. Typescript will enforce this and will have a red underline
// if things are wrong.
.resolve(async (props) => {
// typically you'd do your data augmentation here, for example: fetching from an API
return props.inputs.gene
})
// here you describe the step as a sentence in a methods section, ideally with an applicable citation
.story(props => ({
// each of these are optional but recommended:
// this is a sentence placed in the abstract
abstract: `We applied the identity function to ${props.input ? props.input.gene : 'the gene'}\\ref{doi:somedoi}.`,
// this should be a paragraph for the introduction section
introduction: `A common function used is the Identity function.`,
// this should be a paragraph for the methods section
methods: `The identity function does absolutely nothing when applied to genes, as shown in\\ref{doi:somedoi}.`,
// this will appear as the figure legend of the resulting figure
legend: `This gene shows exactly what we started with.`,
}))
.build()As resolvers run in the server process rather than the client, it is possible to call python code for these. As this is more convenient in many cases when data manipulation is required, helpers exist specifically to do this, and the components directory is set up in a way that makes cross-component importing simpler by convension.
components/{PromptName}/index.tsx
import { MetaNode } from '@/spec/metanode'
import { Gene } from '@/components/Gene'
import python from '@/utils/python'
export const ResolverName = MetaNode('ResolverName')
.meta({
label: 'ResolverName',
description: 'A useful description',
})
.inputs({ input: Gene })
.output(Gene)
.resolve(async (props) => {
// here we use the helper to call the python module
return await python(
// the absolute python import from the root of the repo
// with the last part referring to the actual function to run
'components.identity.myfunc',
// here we provide kargs and kwargs for the python function
// (i.e. func(*kargs, **kwargs))
{ kargs: [props.inputs.input], kwargs: {} },
message => props.notify({ type: 'info', message }),
)
})
.story(props => ({ abstract: `We applied the identity function to ${props.input ? props.input.gene : 'the gene'}\\ref{doi:somedoi}.` }))
.build()components/{PromptName}/__init__.py
# **important** arguments must be json serializable with json.dumps/json.loads
# **important** logging messages should be sent to sys.stderr, information on sys.stdout (or i.e. print)
# may break this
def identity(x):
# do whatever
return xAbsolute imports are used throughout this project and are encouraged.
If you run into python import errors trying to run some of the components, all pertinent python dependencies can be installed with:
# install python requirements
npm run codegen:requirements
pip install -r requirements.txt
# Some systems which do not have python3 installed as a default may require the use of `python3 -m pip`Additional javascript dependencies for your component should be installed with npm using the workspaces feature.
# install the dependency for the component
npm i my-dependency --workspace=mycomponentAdd a requirements.txt file under your component with any necessary python dependencies (what one would need to pip install prior to being able to run the script).
components/{componentname}/requirements.txt
numpyLet us know by submitting an issue!
Find other topics in the Playbook Workflow Builder Developer Guide.