Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add custom_name config for filename #3

Merged
merged 1 commit into from
Feb 11, 2022

Conversation

ron-damon
Copy link
Contributor

Description of change

This PR adds an optional option custom_name to specify a custom name instead of the stream name in the resulting filename.
E.g: Stream name is entities and timestamp is 20210218T000000

  1. custom_name="" and do_timestamp_file=true
{
    "destination_path": "/tmp",
    "custom_name": "",
    "do_timestamp_file": true
}

Results in: /tmp/entities-20210218T000000.jsonl

  1. custom_name="entity" and do_timestamp_file=true
{
    "destination_path": "/tmp",
    "custom_name": "entity",
    "do_timestamp_file": true
}

Results in: /tmp/entity-20210218T000000.jsonl

  1. custom_name="entity" and do_timestamp_file=false
{
    "destination_path": "/tmp",
    "custom_name": "entity",
    "do_timestamp_file": false
}

Results in: /tmp/entity.jsonl

  1. custom_name="" and do_timestamp_file=false
{
    "destination_path": "/tmp",
    "custom_name": "",
    "do_timestamp_file": false
}

Results in: /tmp/entities.jsonl

Manual QA steps

  • All combinations of destination_path, custom_name and `do_timestamp_file tested and worked as expected

Risks

  • None; this is an optional configuration, new implementations will not break

Rollback steps

  • Revert this branch

@andyhuynh3
Copy link
Owner

@ron-damon LGTM (sorry I'm a year late -- just realized there was a PR created). I'll go ahead and merge this in.

@andyhuynh3 andyhuynh3 merged commit fe0ce97 into andyhuynh3:master Feb 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants