docs(jmespath_util): snippets split, improved, and lint

docs/utilities/jmespath_functions.md

@@ -3,6 +3,8 @@ title: JMESPath Functions
 description: Utility
 ---
 
+<!-- markdownlint-disable MD043 -->
+
 ???+ tip
     JMESPath is a query language for JSON used by AWS CLI, AWS Python SDK, and AWS Lambda Powertools for Python.
 
@@ -15,6 +17,9 @@ Built-in [JMESPath](https://jmespath.org/){target="_blank"} Functions to easily
 
 ## Getting started
 
+???+ tip
+    All examples shared in this documentation are available within the [project repository](https://github.com/awslabs/aws-lambda-powertools-python/tree/develop/examples){target="_blank"}.
+
 You might have events that contains encoded JSON payloads as string, base64, or even in compressed format. It is a common use case to decode and extract them partially or fully as part of your Lambda function invocation.
 
 Lambda Powertools also have utilities like [validation](validation.md), [idempotency](idempotency.md), or [feature flags](feature_flags.md) where you might need to extract a portion of your data before using them.
@@ -26,69 +31,34 @@ Lambda Powertools also have utilities like [validation](validation.md), [idempot
 
 You can use the `extract_data_from_envelope` function along with any [JMESPath expression](https://jmespath.org/tutorial.html){target="_blank"}.
 
-=== "app.py"
-
-	```python hl_lines="1 7"
-    from aws_lambda_powertools.utilities.jmespath_utils import extract_data_from_envelope
-
-    from aws_lambda_powertools.utilities.typing import LambdaContext
-
+???+ tip
+	Another common use case is to fetch deeply nested data, filter, flatten, and more.
 
-    def handler(event: dict, context: LambdaContext):
-        payload = extract_data_from_envelope(data=event, envelope="powertools_json(body)")
-        customer = payload.get("customerId")  # now deserialized
-        ...
-	```
+=== "extract_data_from_envelope.py"
+    ```python hl_lines="1 6 10"
+    --8<-- "examples/jmespath_functions/src/extract_data_from_envelope.py"
+    ```
 
-=== "event.json"
+=== "extract_data_from_envelope.json"
 
     ```json
-    {
-        "body": "{\"customerId\":\"dd4649e6-2484-4993-acb8-0f9123103394\"}"
-    }
+    --8<-- "examples/jmespath_functions/src/extract_data_from_envelope.json"
     ```
 
 ### Built-in envelopes
 
 We provide built-in envelopes for popular JMESPath expressions used when looking to decode/deserialize JSON objects within AWS Lambda Event Sources.
 
-=== "app.py"
+=== "extract_data_from_builtin_envelope.py"
 
-	```python hl_lines="1 7"
-    from aws_lambda_powertools.utilities.jmespath_utils import extract_data_from_envelope, envelopes
-
-    from aws_lambda_powertools.utilities.typing import LambdaContext
-
-
-    def handler(event: dict, context: LambdaContext):
-        payload = extract_data_from_envelope(data=event, envelope=envelopes.SNS)
-        customer = payload.get("customerId")  # now deserialized
-        ...
+	```python hl_lines="1 6"
+    --8<-- "examples/jmespath_functions/src/extract_data_from_builtin_envelope.py"
 	```
 
-=== "event.json"
-
-    ```json hl_lines="6"
-    {
-        "Records": [
-            {
-                "messageId": "19dd0b57-b21e-4ac1-bd88-01bbb068cb78",
-                "receiptHandle": "MessageReceiptHandle",
-                "body": "{\"customerId\":\"dd4649e6-2484-4993-acb8-0f9123103394\",\"booking\":{\"id\":\"5b2c4803-330b-42b7-811a-c68689425de1\",\"reference\":\"ySz7oA\",\"outboundFlightId\":\"20c0d2f2-56a3-4068-bf20-ff7703db552d\"},\"payment\":{\"receipt\":\"https:\/\/pay.stripe.com\/receipts\/acct_1Dvn7pF4aIiftV70\/ch_3JTC14F4aIiftV700iFq2CHB\/rcpt_K7QsrFln9FgFnzUuBIiNdkkRYGxUL0X\",\"amount\":100}}",
-                "attributes": {
-                    "ApproximateReceiveCount": "1",
-                    "SentTimestamp": "1523232000000",
-                    "SenderId": "123456789012",
-                    "ApproximateFirstReceiveTimestamp": "1523232000001"
-                },
-                "messageAttributes": {},
-                "md5OfBody": "7b270e59b47ff90a553787216d55d91d",
-                "eventSource": "aws:sqs",
-                "eventSourceARN": "arn:aws:sqs:us-east-1:123456789012:MyQueue",
-                "awsRegion": "us-east-1"
-            }
-        ]
-    }
+=== "extract_data_from_builtin_envelope.json"
+
+    ```json hl_lines="6 15"
+    --8<-- "examples/jmespath_functions/src/extract_data_from_builtin_envelope.json"
     ```
 
 These are all built-in envelopes you can use along with their expression as a reference:
@@ -123,80 +93,48 @@ This sample will decode the value within the `data` key into a valid JSON before
 
 === "powertools_json_jmespath_function.py"
 
-    ```python hl_lines="9"
-    from aws_lambda_powertools.utilities.validation import validate
-
-    import schemas
-
-    sample_event = {
-        'data': '{"payload": {"message": "hello hello", "username": "blah blah"}}'
-    }
-
-    validate(event=sample_event, schema=schemas.INPUT, envelope="powertools_json(data)")
+    ```python hl_lines="7"
+    --8<-- "examples/jmespath_functions/src/powertools_json_jmespath_function.py"
     ```
 
-=== "schemas.py"
+=== "powertools_json_jmespath_schema.py"
 
     ```python hl_lines="7 14 16 23 39 45 47 52"
-    --8<-- "docs/shared/validation_basic_jsonschema.py"
+    --8<-- "examples/jmespath_functions/src/powertools_json_jmespath_schema.py"
     ```
 
 > **Idempotency scenario**
 
 This sample will decode the value within the `body` key of an API Gateway event into a valid JSON object to ensure the Idempotency utility processes a JSON object instead of a string.
 
-```python hl_lines="7" title="Deserializing JSON before using as idempotency key"
-import json
-from aws_lambda_powertools.utilities.idempotency import (
-	IdempotencyConfig, DynamoDBPersistenceLayer, idempotent
-)
-
-persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
-config = IdempotencyConfig(event_key_jmespath="powertools_json(body)")
-
-@idempotent(config=config, persistence_store=persistence_layer)
-def handler(event:APIGatewayProxyEvent, context):
-	body = json.loads(event['body'])
-	payment = create_subscription_payment(
-		user=body['user'],
-		product=body['product_id']
-	)
-	...
-	return {
-		"payment_id": payment.id,
-		"message": "success",
-		"statusCode": 200
-	}
-```
+=== "powertools_json_idempotency_jmespath.py"
 
-#### powertools_base64 function
+    ```python hl_lines="9"
+    --8<-- "examples/jmespath_functions/src/powertools_json_idempotency_jmespath.py"
+    ```
 
-Use `powertools_base64` function to decode any base64 data.
+=== "powertools_json_idempotency_jmespath.json"
 
-This sample will decode the base64 value within the `data` key, and decode the JSON string into a valid JSON before we can validate it.
+    ```json hl_lines="28"
+    --8<-- "examples/jmespath_functions/src/powertools_json_idempotency_jmespath.json"
+    ```
 
-=== "powertools_json_jmespath_function.py"
+#### powertools_base64 function
 
-    ```python hl_lines="12"
-    from aws_lambda_powertools.utilities.validation import validate
+Use `powertools_base64` function to decode any base64 data.
 
-    import schemas
+This sample will decode the base64 value within the `data` key, and decode the JSON string into a valid JSON before we can validate it.
 
-    sample_event = {
-        "data": "eyJtZXNzYWdlIjogImhlbGxvIGhlbGxvIiwgInVzZXJuYW1lIjogImJsYWggYmxhaCJ9="
-    }
+=== "powertools_base64_jmespath_function.py"
 
-    validate(
-          event=sample_event,
-          schema=schemas.INPUT,
-          envelope="powertools_json(powertools_base64(data))"
-    )
+    ```python hl_lines="7"
+    --8<-- "examples/jmespath_functions/src/powertools_base64_jmespath_function.py"
     ```
 
-=== "schemas.py"
+=== "powertools_base64_jmespath_schema.py"
 
     ```python hl_lines="7 14 16 23 39 45 47 52"
-    --8<-- "docs/shared/validation_basic_jsonschema.py"
+    --8<-- "examples/jmespath_functions/src/powertools_base64_jmespath_schema.py"
     ```
 
 #### powertools_base64_gzip function
@@ -205,28 +143,16 @@ Use `powertools_base64_gzip` function to decompress and decode base64 data.
 
 This sample will decompress and decode base64 data, then use JMESPath pipeline expression to pass the result for decoding its JSON string.
 
-=== "powertools_json_jmespath_function.py"
-
-    ```python hl_lines="12"
-    from aws_lambda_powertools.utilities.validation import validate
-
-    import schemas
+=== "powertools_base64_gzip_jmespath_function.py"
 
-    sample_event = {
-        "data": "H4sIACZAXl8C/52PzUrEMBhFX2UILpX8tPbHXWHqIOiq3Q1F0ubrWEiakqTWofTdTYYB0YWL2d5zvnuTFellBIOedoiyKH5M0iwnlKH7HZL6dDB6ngLDfLFYctUKjie9gHFaS/sAX1xNEq525QxwFXRGGMEkx4Th491rUZdV3YiIZ6Ljfd+lfSyAtZloacQgAkqSJCGhxM6t7cwwuUGPz4N0YKyvO6I9WDeMPMSo8Z4Ca/kJ6vMEYW5f1MX7W1lVxaG8vqX8hNFdjlc0iCBBSF4ERT/3Pl7RbMGMXF2KZMh/C+gDpNS7RRsp0OaRGzx0/t8e0jgmcczyLCWEePhni/23JWalzjdu0a3ZvgEaNLXeugEAAA=="
-    }
-
-    validate(
-        event=sample_event,
-        schema=schemas.INPUT,
-        envelope="powertools_base64_gzip(data) | powertools_json(@)"
-    )
+    ```python hl_lines="9"
+    --8<-- "examples/jmespath_functions/src/powertools_base64_gzip_jmespath_function.py"
     ```
 
-=== "schemas.py"
+=== "powertools_base64_gzip_jmespath_schema.py"
 
     ```python hl_lines="7 14 16 23 39 45 47 52"
-    --8<-- "docs/shared/validation_basic_jsonschema.py"
+    --8<-- "examples/jmespath_functions/src/powertools_base64_gzip_jmespath_schema.py"
     ```
 
 ### Bring your own JMESPath function
@@ -238,31 +164,14 @@ For special binary formats that you want to decode before applying JSON Schema v
 
 In order to keep the built-in functions from Powertools, you can subclass from `PowertoolsFunctions`:
 
-=== "custom_jmespath_function.py"
-
-    ```python hl_lines="2-3 6-9 11 17"
-	from aws_lambda_powertools.utilities.jmespath_utils import (
-	    PowertoolsFunctions, extract_data_from_envelope)
-	from jmespath.functions import signature
-
-
-    class CustomFunctions(PowertoolsFunctions):
-        @signature({'types': ['string']})  # Only decode if value is a string
-        def _func_special_decoder(self, s):
-            return my_custom_decoder_logic(s)
-
-    custom_jmespath_options = {"custom_functions": CustomFunctions()}
+=== "powertools_custom_jmespath_function.py"
 
-    def handler(event, context):
-		# use the custom name after `_func_`
-		extract_data_from_envelope(data=event,
-								  envelope="special_decoder(body)",
-								  jmespath_options=**custom_jmespath_options)
-        ...
+    ```python hl_lines="3 5 8-12 15 20"
+	--8<-- "examples/jmespath_functions/src/powertools_custom_jmespath_function.py"
     ```
 
-=== "event.json"
+=== "powertools_custom_jmespath_function.json"
 
     ```json
-    	{"body": "custom_encoded_data"}
+    --8<-- "examples/jmespath_functions/src/powertools_custom_jmespath_function.json"
     ```

examples/jmespath_functions/src/extract_data_from_builtin_envelope.json

@@ -0,0 +1,20 @@
+{
+    "Records": [
+        {
+            "messageId": "19dd0b57-b21e-4ac1-bd88-01bbb068cb78",
+            "receiptHandle": "MessageReceiptHandle",
+            "body": "{\"customerId\":\"dd4649e6-2484-4993-acb8-0f9123103394\",\"booking\":{\"id\":\"5b2c4803-330b-42b7-811a-c68689425de1\",\"reference\":\"ySz7oA\",\"outboundFlightId\":\"20c0d2f2-56a3-4068-bf20-ff7703db552d\"},\"payment\":{\"receipt\":\"https:\/\/pay.stripe.com\/receipts\/acct_1Dvn7pF4aIiftV70\/ch_3JTC14F4aIiftV700iFq2CHB\/rcpt_K7QsrFln9FgFnzUuBIiNdkkRYGxUL0X\",\"amount\":100}}",
+            "attributes": {
+                "ApproximateReceiveCount": "1",
+                "SentTimestamp": "1523232000000",
+                "SenderId": "123456789012",
+                "ApproximateFirstReceiveTimestamp": "1523232000001"
+            },
+            "messageAttributes": {},
+            "md5OfBody": "7b270e59b47ff90a553787216d55d91d",
+            "eventSource": "aws:sqs",
+            "eventSourceARN": "arn:aws:sqs:us-east-1:123456789012:MyQueue",
+            "awsRegion": "us-east-1"
+        }
+    ]
+}

examples/jmespath_functions/src/extract_data_from_builtin_envelope.py

@@ -0,0 +1,9 @@
+from aws_lambda_powertools.utilities.jmespath_utils import envelopes, extract_data_from_envelope
+from aws_lambda_powertools.utilities.typing import LambdaContext
+
+
+def handler(event: dict, context: LambdaContext) -> dict:
+    payload = extract_data_from_envelope(data=event, envelope=envelopes.SQS)
+    customer_id = payload.get("customerId")  # now deserialized
+
+    return {"customer_id": customer_id, "message": "success", "statusCode": 200}

examples/jmespath_functions/src/extract_data_from_envelope.json

@@ -0,0 +1,12 @@
+{
+	"body": "{\"customerId\":\"dd4649e6-2484-4993-acb8-0f9123103394\"}",
+	"deeply_nested": [
+		{
+			"some_data": [
+				1,
+				2,
+				3
+			]
+		}
+	]
+}

examples/jmespath_functions/src/extract_data_from_envelope.py

@@ -0,0 +1,12 @@
+from aws_lambda_powertools.utilities.jmespath_utils import extract_data_from_envelope
+from aws_lambda_powertools.utilities.typing import LambdaContext
+
+
+def handler(event: dict, context: LambdaContext) -> dict:
+    payload = extract_data_from_envelope(data=event, envelope="powertools_json(body)")
+    customer_id = payload.get("customerId")  # now deserialized
+
+    # also works for fetching and flattening deeply nested data
+    some_data = extract_data_from_envelope(data=event, envelope="deeply_nested[*].some_data[]")
+
+    return {"customer_id": customer_id, "message": "success", "context": some_data, "statusCode": 200}

examples/jmespath_functions/src/powertools_base64_gzip_jmespath_function.py

@@ -0,0 +1,9 @@
+import powertools_base64_gzip_jmespath_schema as schemas
+
+from aws_lambda_powertools.utilities.validation import validate
+
+sample_event = {
+    "data": "H4sIACZAXl8C/52PzUrEMBhFX2UILpX8tPbHXWHqIOiq3Q1F0ubrWEiakqTWofTdTYYB0YWL2d5zvnuTFellBIOedoiyKH5M0iwnlKH7HZL6dDB6ngLDfLFYctUKjie9gHFaS/sAX1xNEq525QxwFXRGGMEkx4Th491rUZdV3YiIZ6Ljfd+lfSyAtZloacQgAkqSJCGhxM6t7cwwuUGPz4N0YKyvO6I9WDeMPMSo8Z4Ca/kJ6vMEYW5f1MX7W1lVxaG8vqX8hNFdjlc0iCBBSF4ERT/3Pl7RbMGMXF2KZMh/C+gDpNS7RRsp0OaRGzx0/t8e0jgmcczyLCWEePhni/23JWalzjdu0a3ZvgEaNLXeugEAAA=="  # noqa: E501
+}
+
+validate(event=sample_event, schema=schemas.INPUT, envelope="powertools_base64_gzip(data) | powertools_json(@)")

examples/jmespath_functions/src/powertools_base64_gzip_jmespath_schema.py

@@ -0,0 +1,39 @@
+INPUT = {
+    "$schema": "http://json-schema.org/draft-07/schema",
+    "$id": "http://example.com/example.json",
+    "type": "object",
+    "title": "Sample schema",
+    "description": "The root schema comprises the entire JSON document.",
+    "examples": [{"message": "hello world", "username": "lessa"}],
+    "required": ["message", "username"],
+    "properties": {
+        "message": {
+            "$id": "#/properties/message",
+            "type": "string",
+            "title": "The message",
+            "examples": ["hello world"],
+            "maxLength": 100,
+        },
+        "username": {
+            "$id": "#/properties/username",
+            "type": "string",
+            "title": "The username",
+            "examples": ["lessa"],
+            "maxLength": 30,
+        },
+    },
+}
+
+OUTPUT = {
+    "$schema": "http://json-schema.org/draft-07/schema",
+    "$id": "http://example.com/example.json",
+    "type": "object",
+    "title": "Sample outgoing schema",
+    "description": "The root schema comprises the entire JSON document.",
+    "examples": [{"statusCode": 200, "body": "response"}],
+    "required": ["statusCode", "body"],
+    "properties": {
+        "statusCode": {"$id": "#/properties/statusCode", "type": "integer", "title": "The statusCode"},
+        "body": {"$id": "#/properties/body", "type": "string", "title": "The response"},
+    },
+}

examples/jmespath_functions/src/powertools_base64_jmespath_function.py

@@ -0,0 +1,7 @@
+import powertools_base64_jmespath_schema as schemas
+
+from aws_lambda_powertools.utilities.validation import validate
+
+sample_event = {"data": "eyJtZXNzYWdlIjogImhlbGxvIGhlbGxvIiwgInVzZXJuYW1lIjogImJsYWggYmxhaCJ9="}
+
+validate(event=sample_event, schema=schemas.INPUT, envelope="powertools_json(powertools_base64(data))")