Skip to content

Commit

Permalink
fix: Secor output file pattern issue fixes for Azure
Browse files Browse the repository at this point in the history
  • Loading branch information
sowmya-dixit committed May 29, 2024
1 parent 009da70 commit 5031fef
Showing 1 changed file with 14 additions and 0 deletions.
14 changes: 14 additions & 0 deletions kubernetes/helm_charts/secor/config/secor.azure.properties
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,20 @@ secor.azure.container.name={{ $.Values.azure_container_name }}
# Microsoft Azure blob storage path where files are stored within the container.
secor.azure.path={{- get (get $.Values.secor_jobs $.Release.Name) "base_path" }}

# Output file pattern excluding prefix. Defaults to topic/partition/generation_kafkaPartition_fmOffset.gz.
# Available placeholders are
# topic - The topic name the data is being fetched
# partition - The partition name
# generation - Generation
# kafkaPartition - The kafka partition
# fmOffset - First Message offset in the file.
# randomHex - A 4 character random hex to append to the file name
# currentTimestamp - Time of upload in epoch format
# currentTime - Time of upload in HH-mm format
# currentDate - Time of upload in YYYYMMDD format
# folder - Folder to use based on message id map lookup
secor.s3.output_file_pattern={{ get (get $.Values.secor_jobs $.Release.Name) "output_file_pattern" }}

################
# END MUST SET #
################

0 comments on commit 5031fef

Please sign in to comment.