Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PPL-Lang]PPL support json_delete,json_append functions #970

Closed
YANG-DB opened this issue Dec 4, 2024 · 1 comment
Closed

[PPL-Lang]PPL support json_delete,json_append functions #970

YANG-DB opened this issue Dec 4, 2024 · 1 comment
Assignees
Labels
0.7 enhancement New feature or request Lang:PPL Pipe Processing Language support

Comments

@YANG-DB
Copy link
Member

YANG-DB commented Dec 4, 2024

Is your feature request related to a problem?
Continue Supporting the PPL Json grammar with adding the next functions:

  • json_delete(object,[keys list])

    • Removes one or more keys and their corresponding values from the specified JSON object.
      • objects: the Json object is the field being evaluated
      • [keys list]: keys is the list of keys to remove from the object
  • json_append(object, [(path key, value) list])

    • Adds value to an array, json_append tails the value to the arrays into their corespondent path.
      • objects: the Json object is the field being evaluated
      • [(path key, values) list]): path key is the json path name of the array and the value is the object be added

Each designates an array or value within the object json document. The json_extend function adds the values of the corresponding after the last value of the array designated by the .

What solution would you like?

support the following json_delete

Example:

    os> source=people | eval deleted = json_delete({"a":"valueA", "b":"valueB"}, ["a"]) 
    fetched rows / total rows = 1/1
    +----------------------------------+
    | deleted                          |
    +----------------------------------+
    | {"b": "valueB" }                 |
    +----------------------------------+

    os> source=people | eval  eval deleted = json_delete({"a":[{"b":1},{"b":2},{"c":3}]}, ["a.b"])
    fetched rows / total rows = 1/1
    +-----------------------------------------------------------+
    | deleted                                                   |
    +-----------------------------------------------------------+
    | {"a":[{"c":3}] }                                          |
    +-----------------------------------------------------------+

    os> source=people | eval `no_action` =  json_delete({"a":[{"b":1},{"b":2},{"c":3}]}, ["b.b"])
    fetched rows / total rows = 1/1
    +-----------------------------------+
    | no_action                         |
    +-----------------------------------+
    | {"a":[{"b":1},{"b":2},{"c":3}]}   |
    +-----------------------------------+

support the following json_append

    os> source=people | eval append = json_append(`{"a":["valueA", "valueB"]}`, ["a","valueC"]) 
    fetched rows / total rows = 1/1
    +-------------------------------------------------+
    | append                                          |
    +-------------------------------------------------+
    | {"a":["valueA", "valueB", "valueC"]}            |
    +-------------------------------------------------+

    os> source=people | eval append = json_append(`{"a":["valueA", "valueB"]}`, {"a":["valueC"]}) 
    fetched rows / total rows = 1/1
    +-----------------------------------------------+
    | append                                        |
    +-----------------------------------------------+
    | {"a":["valueA", "valueB", ["valueC"]]}        |
    +-----------------------------------------------+

    os> source=people | eval append = json_append(`{"root":{ "a":["valueA", "valueB"]}}`, {"root.a":"valueC"}) 
    fetched rows / total rows = 1/1
    +-----------------------------------------------+
    | append                                        |
    +-----------------------------------------------+
    |{"root": {"a":["valueA", "valueB", "valueC"]}} |
    +-----------------------------------------------+

Do you have any additional context?

@YANG-DB YANG-DB added enhancement New feature or request untriaged Lang:PPL Pipe Processing Language support 0.7 labels Dec 4, 2024
@YANG-DB YANG-DB moved this to Todo in PPL Commands Dec 4, 2024
@YANG-DB YANG-DB self-assigned this Dec 4, 2024
@YANG-DB YANG-DB removed the untriaged label Dec 4, 2024
@YANG-DB YANG-DB changed the title [PPL-Lang]PPL support json_delete & json_extend functions [PPL-Lang]PPL support json_delete,json_append &json_extend functions Dec 4, 2024
@YANG-DB YANG-DB moved this from Todo to Design in PPL Commands Dec 4, 2024
@LantaoJin
Copy link
Member

 os> source=people | eval deleted = json_delete({"a":"valueA", "b":"valueB"}, ["a"]) 
    fetched rows / total rows = 1/1
    +----------------------------------+
    | deleted                          |
    +----------------------------------+
    | {"a": "valueA" }                 |
    +----------------------------------+

Should be

 os> source=people | eval deleted = json_delete({"a":"valueA", "b":"valueB"}, ["a"]) 
    fetched rows / total rows = 1/1
    +----------------------------------+
    | deleted                          |
    +----------------------------------+
    | {"b": "valueB" }                 |
    +----------------------------------+
os> source=people | eval append = json_append(`{"a":["valueA", "valueB"]}`, {"a":["valueC"]}) 
    fetched rows / total rows = 1/1
    +-----------------------------------------------+
    | append                                        |
    +-----------------------------------------------+
    | {"a":["valueA", "valueB", ["valueC"]]}        |
    +-----------------------------------------------+

We might be hard to support this case since the JSON ARRAY was implemented by Spark ARRAY which all elements in it requires the same type.
So does case

    os> source=people | eval extend = json_extend(`{"a":["valueA", "valueB"]}`, {"b":["valueC","valueD"]}) 
    fetched rows / total rows = 1/1
    +-------------------------------------------------------------+
    | extend                                                      |
    +-------------------------------------------------------------+
    | {"a":["valueA", "valueB", {"b":"valueC"}, {"b":"valueD"}]}  |
    +-------------------------------------------------------------+

@YANG-DB YANG-DB changed the title [PPL-Lang]PPL support json_delete,json_append &json_extend functions [PPL-Lang]PPL support json_delete,json_append functions Dec 11, 2024
@YANG-DB YANG-DB moved this from Design to InReview in PPL Commands Dec 12, 2024
@YANG-DB YANG-DB moved this from InReview to Done in PPL Commands Dec 16, 2024
@YANG-DB YANG-DB closed this as completed by moving to Done in PPL Commands Dec 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.7 enhancement New feature or request Lang:PPL Pipe Processing Language support
Projects
Status: Done
Development

No branches or pull requests

2 participants