Skip to content

Commit

Permalink
feat(partition): update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
adrienaury committed Jan 15, 2025
1 parent 3c3f6cf commit 5eb751e
Show file tree
Hide file tree
Showing 4 changed files with 41 additions and 6 deletions.
30 changes: 30 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,7 @@ The following types of masks can be used :
* [`replacement`](#replacement) is to mask a data with another data from the jsonline.
* [`pipe`](#pipe) is a mask to handle complex nested array structures, it can read an array as an object stream and process it with a sub-pipeline.
* [`apply`](#apply) process selected data with a sub-pipeline.
* [`partitions`](#partitions) will rely on conditions to identify specific cases.
* [`luhn`](#luhn) can generate valid numbers using the Luhn algorithm (e.g. french SIRET or SIREN).
* [`markov`](#markov) can generate pseudo text based on a sample text.
* [`findInCSV`](#findincsv) get one or multiple csv lines which matched with Json entry value from CSV files.
Expand Down Expand Up @@ -1069,6 +1070,35 @@ By default, if not specified otherwise, these classes will be used (input -> out

[Return to list of masks](#possible-masks)

### Partitions

[![Try it](https://img.shields.io/badge/-Try%20it%20in%20PIMO%20Play-brightgreen)](https://cgi-fr.github.io/pimo-play/#c=G4UwTgzglg9gdgLgAQCICMKBQBbAhhAayjgHMFNMkkBaJCEAGxAGMAXGMcq7pAKwngAHXKwAWyFLgCuYjlh55CXHkmFhWUDfAjKVNJHFzYQEkHigN5e7gHdRIREgDkAbxcA6abLBIAPkgATEAAzaQZWVBQ-JGwpCFYAJRASEAAPAFkRZlFUAD0AbVxqAC8AXQBqAAFCkoqAHTr3GrLygBIogF8Op0prKjEHXT79Zm1WXDhWCQn4AE9sSoCYczh3UewrPVpDYwkoAM3rO0HnN08ZUQ5ooNCpcMjo2PiklIysnJQCgAZqAE4K9pILo9YZIAaIXqg2ijODxCZTVBfJHIlGHHjbIwmVAwAZgNEqcFDPrQsbw6ZwOYbTBAA&i=N4KABGBECGCuAuALA9gJ0gLjAbXBKApgLbQCWANgAIAmyJpAdgHQDGdkANHhJAIwBMAZgAsAVgBsnblABSyRAzAARZAUh4AuiAC+QA)

The partition mask will rely on conditions to identify specific cases and apply a defined list of masks for each case. Example configuration:

```yaml
- selector:
jsonpath: "ID"
mask:
partitions: # only the fist active condition will execute
- name: case1
when: '{{ regexMatch "P[A-Z]{3}[0-9]{3}" .ID }}'
then:
# List of masks for case 1
- constant: "this is case 1"
- name: case2
when: '{{ regexMatch "G[0-9]{11}" .ID }}'
then:
# List of masks for case 2
- constant: "this is case 2"
- name: default # case with no "when" condition will always execute
then:
# List of masks for unrecognized cases
- constant: "this is another case"
```

[Return to list of masks](#possible-masks)

### FindInCSV

[![Try it](https://img.shields.io/badge/-Try%20it%20in%20PIMO%20Play-brightgreen)](https://cgi-fr.github.io/pimo-play/#c=G4UwTgzglg9gdgLgAQCICMKBQBbAhhAayjgHMFMkkBaJCEAGxAGMAXGMcyrpAKwngAOuFgAtkKYgDMYWbnkIRO3aklwATNUnGzlNScTUBJOAGEAygDUlyrgFcwUcSJYsBigPTuSUCCwB03qK2AEa2dGBM8CwgcP6R2O64YNje9IwQ7mgAnAAswUySkgDMAKwADGVoIADsIMElRbgATLgAHME5OVXtTUwAbO5guADu7llNTRX5Zbh91UVqJUwgTWhoM7jqOSWdIGp9TDmVZZJ9rdXuAjAEINjwfkwQwDo2lCAAHrisALLCTGIUV7cR7AZAAcgA3hCABQGD5IPyoAAqAE8BCAkBgAJRIAA+SHoMGG4CQAF9SWDAUC3rEwCjxFC-Cw0SAAPpockvV48L5MJJqazUkHgxkAOVw2Ax+MJxLAZIpVOpMRYdIZEL8cAlUpl4E51K4ipsH3RrD24mEVEY+BYVHgIC5NhEIHU4GQKtsIENyhVUGwbrAHswQA&i=N4KABGBEAuCeAOBTA+gRkgLigMwJYCdFIAacKAOwEMBbIrSAY0v1vIBNF9IQBfIA)
Expand Down
2 changes: 1 addition & 1 deletion pkg/model/model.go
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,7 @@ type MaskType struct {
Sequence SequenceType `yaml:"sequence,omitempty" json:"sequence,omitempty" jsonschema:"oneof_required=Sequence,title=Sequence Mask" jsonschema_description:"Generate a sequenced ID that follows specified format"`
Sha3 Sha3Type `yaml:"sha3,omitempty" json:"sha3,omitempty" jsonschema:"oneof_required=Sha3,title=Sha3 Mask" jsonschema_description:"Generate a variable-length crytographic hash (collision resistant)"`
Apply ApplyType `yaml:"apply,omitempty" json:"apply,omitempty" jsonschema:"oneof_required=Apply,title=Apply Mask" jsonschema_description:"Call external masking file"`
Partition []PartitionType `yaml:"partition,omitempty" json:"partition,omitempty" jsonschema:"oneof_required=Partition,title=Partition Mask" jsonschema_description:"Identify specific cases and apply a defined list of masks for each case"`
Partition []PartitionType `yaml:"partitions,omitempty" json:"partitions,omitempty" jsonschema:"oneof_required=Partition,title=Partition Mask" jsonschema_description:"Identify specific cases and apply a defined list of masks for each case"`
}

type Masking struct {
Expand Down
13 changes: 9 additions & 4 deletions schema/v1/pimo.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -587,7 +587,7 @@
},
{
"required": [
"partition"
"partitions"
],
"title": "Partition"
}
Expand Down Expand Up @@ -785,11 +785,11 @@
"title": "Apply Mask",
"description": "Call external masking file"
},
"partition": {
"additionalProperties": {
"partitions": {
"items": {
"$ref": "#/$defs/PartitionType"
},
"type": "object",
"type": "array",
"title": "Partition Mask",
"description": "Identify specific cases and apply a defined list of masks for each case"
}
Expand Down Expand Up @@ -893,6 +893,10 @@
},
"PartitionType": {
"properties": {
"name": {
"type": "string",
"description": "name of the partition"
},
"when": {
"type": "string",
"description": "template to execute, if true the condition is active"
Expand All @@ -908,6 +912,7 @@
"additionalProperties": false,
"type": "object",
"required": [
"name",
"then"
]
},
Expand Down
2 changes: 1 addition & 1 deletion test/suites/masking_partition.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ testcases:
- selector:
jsonpath: "id"
mask:
partition:
partitions:
- name: idrh
when: '[[ .id | default "" | mustRegexMatch "^P[A-Z]{3}[0-9]{3}$" ]]'
then:
Expand Down

0 comments on commit 5eb751e

Please sign in to comment.