Skip to content

Commit

Permalink
Add some development notes.
Browse files Browse the repository at this point in the history
  • Loading branch information
chrisberkhout committed Sep 25, 2024
1 parent 85408d9 commit 9bc3f95
Showing 1 changed file with 27 additions and 0 deletions.
27 changes: 27 additions & 0 deletions packages/tychon/DEVELOPMENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Development notes

All copies of some files are identical. This should be maintained.

It is especially important for the `common_init.yml` and `common_host.yml`
pipeline definitions that are shared across many data streams.

To check whether there are multiple versions of each of these files, use the
following commands to list them with their MD5 hashes.

```
echo; find -name 'common_host.yml' | xargs md5sum | sort
echo; find -name 'common_init.yml' | xargs md5sum | sort
echo; find -name 'default_policy.json' | xargs md5sum | sort
echo; find -name 'docker-compose.yml' | xargs md5sum | sort
echo; find -name 'test-default-config.yml' | xargs md5sum | sort
echo; find -wholename './elasticsearch/transform/*/manifest.yml' | xargs md5sum | sort
```

It is also important to keep the field definitions for each transform
destination index aligned with the field definitions for its source data
stream. (Although they will differ in `is-transform-source-*.yml` files, and
the source data streams do not need to explicitly define ECS fields.)

There are other files that have only minor differences between data streams.
When making a change to one data stream, consider whether there is a
corresponding change to make in other data streams.

0 comments on commit 9bc3f95

Please sign in to comment.