Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Record path concatenator activity in state #2501

Open
paul-butcher opened this issue Dec 18, 2023 · 0 comments
Open

Record path concatenator activity in state #2501

paul-butcher opened this issue Dec 18, 2023 · 0 comments

Comments

@paul-butcher
Copy link
Contributor

paul-butcher commented Dec 18, 2023

When faced with a path branch/leaf, the path concatenator looks up root/branch and returns root/branch/leaf. It modifies the collectionPath of the documents in situ (rather than by creating a whole new index to put it in, like other stages do).
This is partly because it is only expected to change a small number of records and it would be wasteful and awkward to make it into a whole new step. It is also because of the transitive nature of the operation.

However, this does mean that we lose some observability, compared to the other "real" stages. We do not know the state before the change, so if a bug occurs in the concatenator, and it changes something in a way that it should not, then we can't see that on the record.

It would be good to record this in the state section of the record. Information could include:

  • path concatenation time
  • original collectionPath
  • any other records involved (this may be inappropriate, could be unwieldy)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant