Skip to content
This repository has been archived by the owner on Feb 28, 2022. It is now read-only.

Provide tap function in the pipeline builder to enable in-progress inspection #14

Closed
trieloff opened this issue Jul 7, 2018 · 5 comments · Fixed by #61
Closed

Provide tap function in the pipeline builder to enable in-progress inspection #14

trieloff opened this issue Jul 7, 2018 · 5 comments · Fixed by #61
Labels
enhancement New feature or request

Comments

@trieloff
Copy link
Contributor

trieloff commented Jul 7, 2018

Calling pipe.pre(mystep).tap(myfn)

should ensure that myfn will be called before mystep is executed by the pipeline, but that the return value of myfn will not enter the pipeline.

This allows setting myfn as an observer to mystep, which can ease debugging and inspection.

@trieloff trieloff added the enhancement New feature or request label Jul 7, 2018
@tripodsan
Copy link
Contributor

why before? the taps should be sequetially in the way they are defined.

pipe
.tap(t1)
.pre(p1)
.tap(t2)
.once(o)
....

will call t1 -> p1 -> t2 -> o;

OTOH, it is not clear to me what the benefit of tap is. if for debugging with a debugger, then it is easier to set the breakpoint accordingly. if debugging with loggers, then it should be possible to define the loglevels more cleverly.

i.e. I don't think that someone writes a special tap function, just to debug his pre function.

@trieloff
Copy link
Contributor Author

trieloff commented Jul 8, 2018

The use case I was thinking of was writing an integration test for the pipeline (see adobe/helix-cli#18) – you'd want to be able to get a full dump of the current payload, so your tap would just write the current payload to disk, allowing you to copy it into your test directory.

@tripodsan
Copy link
Contributor

but wouldn't this rather by a generic testing/tapping capability of the pipeline itself?
but it might be a good modular approach to this.

@trieloff trieloff added the wontfix This will not be worked on label Sep 13, 2018
@trieloff
Copy link
Contributor Author

Not needed, we have better debugging options now.

@trieloff trieloff reopened this Oct 2, 2018
@trieloff trieloff removed the wontfix This will not be worked on label Oct 2, 2018
@trieloff
Copy link
Contributor Author

trieloff commented Oct 2, 2018

We need taps (among others) for type validation.

trieloff added a commit that referenced this issue Oct 2, 2018
trieloff added a commit that referenced this issue Oct 8, 2018
* Added simple tap function #14

* quick production check utility

* Fixed an issue where when would not work with additional arguments

* pipeline leaked stuff

* dump the entire context for each step of the pipeline when in non-production #14

* documentation

* lint

* don't forget to make the dir

* typo

* more compact timestamps

* use logs/debug for debug output

* rename tap to every

* use dirname to find dir name

* write output to logs/debug

* fix lint

* Using async for @tripodsan
trieloff added a commit that referenced this issue Oct 11, 2018
* generate typescript files from JSON Schema

* Example schema files

* Adding root type definition

* More detailed schemas

* Added simple tap function #14

* quick production check utility

* Fixed an issue where when would not work with additional arguments

* pipeline leaked stuff

* dump the entire context for each step of the pipeline when in non-production #14

* documentation

* lint

* don't forget to make the dir

* typo

* fixed merge gone wrong

* more merge fixes

* Add license header to generated file

* Validate in the HTML pipeline

* declare __ow_headers and body

* initial definition of a markdown ast

* added sections

* make simplest test case pass

* Extracted MDAST schema into separate file

* Updated context schema

* Complete MDAST Schema

* Re-generated context types

* Use proper logger for warnings

* Removed dead logger code

* Adapted schemas to pipeline changes

* re-enable cleanup

* fully test the validator function

* validate JSON pipeline, too

* Enable generation of docs using jsonschema2md

* run docs in CI

* Refactoring: split schemas into smaller files (breaks stuff)

* refactored schemas to make them validate

* Example documentation

* cleanup

* added license headers

* rewrite schemas to avoid infinite recursion in schema2ts

* Updated title and status for schemas

* More elaborate description of content

* Added basic MDAST description

* nicer align description

* link to Winston

* automatically generate documentation

* lint

* Refactoring: make validate async, let the pipeline handle async taps
aggregate errors in context instead of throwing them
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants