Develop to 1.0.2-GA (#93)

* Issue #4 feat: enhance the ingest event to add obsrv and source meta * issue#223: feat: Updated API endpoints with verbs * issue#223: fix: Test case modification * issue #84 : fix: API swagger doc update * #99 fix: upgrade packages to fix vulnerabilities * Command-service Vulnerabilities fixes (#56) * Release 1.3.0 into Main (#53) * issue#223: feat: Updated API endpoints with verbs * issue#223: fix: Test case modification * issue #84 : fix: API swagger doc update * #99 fix: upgrade packages to fix vulnerabilities --------- Co-authored-by: Jerald <[email protected]> Co-authored-by: harishkumar gangula <[email protected]> Co-authored-by: Praveen <[email protected]> * command-service vulnerabilities fixes --------- Co-authored-by: Manjunath Davanam <[email protected]> Co-authored-by: Jerald <[email protected]> Co-authored-by: harishkumar gangula <[email protected]> Co-authored-by: Praveen <[email protected]> * #90 fix: Resolve API Issues (#52) * #90 fix: format error messages, code cleanup * #90 fix: validate extraction config during ingest * #90 fix: add test cases for extraction key validation * #90 fix: move error handler to helpers for standard handling * #90 fix: update telemetry audit event set function (#58) * Build and deployment (#57) * build api image * Update build_and_deploy.yaml * Update build_and_deploy.yaml * Update build_and_deploy.yaml * Update build_and_deploy.yaml * build api image * Update build_and_deploy.yaml * build api image * build api image * build api image * build and deploy api image * build and deploy api image * build and deploy api image * build and deploy of api service * build and deploy of api service * modify docker file * modify docker file * modify docker file * update build and deployment * update build and deployment * Update build_and_deploy.yaml * api service build and deployment * api service build and deployment * feat: obsrv api service build and deployment github actions configuration * #0 fix: update the tag condition in actions --------- Co-authored-by: ManojCKrishna <[email protected]> Co-authored-by: Manoj Krishna <[email protected]> Co-authored-by: Harish Kumar Gangula <[email protected]> Co-authored-by: Manjunath Davanam <[email protected]> Co-authored-by: Praveen <[email protected]> * #0 fix: add azure exhaust support (#44) * #0 fix: add azure exhaust support * #0 fix: update azure exhaust service without async * develop into release 1.3.0 (#59) * Issue #4 feat: enhance the ingest event to add obsrv and source meta * #90 fix: Resolve API Issues (#52) * #90 fix: format error messages, code cleanup * #90 fix: validate extraction config during ingest * #90 fix: add test cases for extraction key validation * #90 fix: move error handler to helpers for standard handling * #90 fix: update telemetry audit event set function (#58) * Build and deployment (#57) * build api image * Update build_and_deploy.yaml * Update build_and_deploy.yaml * Update build_and_deploy.yaml * Update build_and_deploy.yaml * build api image * Update build_and_deploy.yaml * build api image * build api image * build api image * build and deploy api image * build and deploy api image * build and deploy api image * build and deploy of api service * build and deploy of api service * modify docker file * modify docker file * modify docker file * update build and deployment * update build and deployment * Update build_and_deploy.yaml * api service build and deployment * api service build and deployment * feat: obsrv api service build and deployment github actions configuration * #0 fix: update the tag condition in actions --------- Co-authored-by: ManojCKrishna <[email protected]> Co-authored-by: Manoj Krishna <[email protected]> Co-authored-by: Harish Kumar Gangula <[email protected]> Co-authored-by: Manjunath Davanam <[email protected]> Co-authored-by: Praveen <[email protected]> * #0 fix: add azure exhaust support (#44) * #0 fix: add azure exhaust support * #0 fix: update azure exhaust service without async --------- Co-authored-by: shiva-rakshith <[email protected]> Co-authored-by: Praveen Veleneni <[email protected]> Co-authored-by: Manoj Krishna <[email protected]> Co-authored-by: ManojCKrishna <[email protected]> Co-authored-by: Manoj Krishna <[email protected]> Co-authored-by: Harish Kumar Gangula <[email protected]> Co-authored-by: Manjunath Davanam <[email protected]> * #126 Feat: Add Querying on Aggregated datasources (#60) * #126 feat: enable querying on aggregated datasources * #126 feat: Fix validation issues and update routes for aggregate queries * #126 fix: remove unused methods * #126 fix: remove aggregates from endpoint * #126 fix: add validation for granularity options * #126 fix: Remove unused routes and validators * #126 feat: add new property to datasources structure and update rollup querying * #305 Feat: Add and Update dataset status (#62) * #305 feat: add retire apis and update dataset status * #305 fix: remove unused import * #305 fix: add command service api call to restart jobs * #305 fix: fix command service payload * #305 fix: update api doc * #305 fix: clear unused code * #305 fix: clear unused code * #305 fix: undo change submit ingestion * #305 fix: update test case to use enums * #305 fix: add test env file for test cases * #305 fix: add enum for string in test case * Issue #305 feat: exclude system-events data source in validation (#63) * Issue #165 feat: generate AUDIT events (#71) * Issue #fix-344 fix: defect fix (#76) * Issue #193 Feat: postman collection for Obsrv APIs (#77) * #198 feat: add validation for denorm out fields, to avoid duplicates (#78) * #198 feat: add validation for denorm out fields, to avoid duplicate out fields * #198 fix: update error message for denorm fields duplicate * #198 feat: add test cases for validation of denorm validation * #198 fix: remove console statement * Tenant ingestion (#81) * #0 feat: Add tenant_ingest API endpoint * #0 fix: Update API ID in RoutesConfig.ts * #0 fix: Update datasetId format in IngestorService.ts * #0 fix: Swap datasetId and tenantId in IngestorService.ts (#82) * Changed status enum according to latest workflow (#80) * Issue #193 Fix: changed status enum according to latest workflow * Issue #193 Fix: changed status enum according to latest workflow * Issue #0000: fix: update Dockerfile (#85) * issue #210 refactor: api endpoints refactored [removal of obsrv prefix] (#88) * issue #210 refactor: removal of obsrv prefix from API endpoints * issue #210 refactor: updated api endpoints postman and swagger documentation * issue #210 refactor: html doc for api endpoints --------- Co-authored-by: shiva-rakshith <[email protected]> Co-authored-by: Jerald <[email protected]> Co-authored-by: harishkumar gangula <[email protected]> Co-authored-by: Praveen <[email protected]> Co-authored-by: Sowmya N Dixit <[email protected]> Co-authored-by: Manoj Krishna <[email protected]> Co-authored-by: ManojCKrishna <[email protected]> Co-authored-by: Manoj Krishna <[email protected]> Co-authored-by: Ravi Mula <[email protected]> Co-authored-by: Ravinder Kumar <[email protected]> Co-authored-by: GayathriSrividya <[email protected]> Co-authored-by: Shreyas Bhaktharam <[email protected]> Co-authored-by: Jerald <[email protected]>
Sanketika-Obsrv · Jan 29, 2024 · b49ae82 · b49ae82
1 parent 058ec85
commit b49ae82
Show file tree

Hide file tree

Showing 5 changed files with 765 additions and 1,125 deletions.
diff --git a/api-service/postman-collection/Obsrv API Service.postman_collection.json b/api-service/postman-collection/Obsrv API Service.postman_collection.json
@@ -1,10 +1,10 @@
 {
 	"info": {
-		"_postman_id": "a3107544-5d3b-4af9-8002-74ce81357719",
+		"_postman_id": "eb1ecc94-340f-4cdb-a978-e52e1f454aab",
 		"name": "Obsrv API Service",
 		"schema": "https://schema.getpostman.com/json/collection/v2.0.0/collection.json",
-		"_exporter_id": "23908122",
-		"_collection_link": "https://restless-water-97719.postman.co/workspace/temp-workspace~5901a80f-1ada-4758-bf49-6f0fe63041c8/collection/23908122-a3107544-5d3b-4af9-8002-74ce81357719?action=share&source=collection_link&creator=23908122"
+		"_exporter_id": "26192103",
+		"_collection_link": "https://speeding-star-177775.postman.co/workspace/sanketika-obsrv~2ce96556-12e2-48bd-8e42-9c1dba428cc8/collection/26192103-eb1ecc94-340f-4cdb-a978-e52e1f454aab?action=share&source=collection_link&creator=26192103"
 	},
 	"item": [
 		{
@@ -27,7 +27,7 @@
 								}
 							}
 						},
-						"url": "{{HOST_IP}}/obsrv/v1/datasets/list"
+						"url": "{{HOST_IP}}/datasets/v1/list"
 					},
 					"response": []
 				},
@@ -49,14 +49,13 @@
 							}
 						},
 						"url": {
-							"raw": "{{HOST_IP}}/obsrv/v1/datasets/create?status=Live",
+							"raw": "{{HOST_IP}}/datasets/v1/create?status=Live",
 							"host": [
 								"{{HOST_IP}}"
 							],
 							"path": [
-								"obsrv",
-								"v1",
 								"datasets",
+								"v1",
 								"create"
 							],
 							"query": [
@@ -86,7 +85,7 @@
 								}
 							}
 						},
-						"url": "{{HOST_IP}}/obsrv/v1/datasets/create"
+						"url": "{{HOST_IP}}/datasets/v1/create"
 					},
 					"response": []
 				},
@@ -107,7 +106,7 @@
 								}
 							}
 						},
-						"url": "{{HOST_IP}}/obsrv/v1/datasets/update"
+						"url": "{{HOST_IP}}/datasets/v1/update"
 					},
 					"response": []
 				},
@@ -120,16 +119,15 @@
 						"method": "GET",
 						"header": [],
 						"url": {
-							"raw": "{{HOST_IP}}/obsrv/v1/datasets/get/?status=Live",
+							"raw": "{{HOST_IP}}/datasets/v1/get/dataset?status=Live",
 							"host": [
 								"{{HOST_IP}}"
 							],
 							"path": [
-								"obsrv",
-								"v1",
 								"datasets",
+								"v1",
 								"get",
-								""
+								"dataset"
 							],
 							"query": [
 								{
@@ -163,7 +161,7 @@
 								}
 							}
 						},
-						"url": "{{HOST_IP}}/obsrv/v1/datasources/list"
+						"url": "{{HOST_IP}}/datasources/v1/list"
 					},
 					"response": []
 				},
@@ -184,7 +182,7 @@
 								}
 							}
 						},
-						"url": "{{HOST_IP}}/obsrv/v1/datasources/create"
+						"url": "{{HOST_IP}}/datasources/v1/create"
 					},
 					"response": []
 				},
@@ -205,7 +203,7 @@
 								}
 							}
 						},
-						"url": "{{HOST_IP}}/obsrv/v1/datasources/update"
+						"url": "{{HOST_IP}}/datasources/v1/update"
 					},
 					"response": []
 				},
@@ -218,14 +216,13 @@
 						"method": "GET",
 						"header": [],
 						"url": {
-							"raw": "{{HOST_IP}}/obsrv/v1/datasources/get/sb-telemetry_sb-telemetry?status=Live",
+							"raw": "{{HOST_IP}}/datasources/v1/get/sb-telemetry_sb-telemetry?status=Live",
 							"host": [
 								"{{HOST_IP}}"
 							],
 							"path": [
-								"obsrv",
-								"v1",
 								"datasources",
+								"v1",
 								"get",
 								"sb-telemetry_sb-telemetry"
 							],
@@ -261,7 +258,7 @@
 								}
 							}
 						},
-						"url": "{{HOST_IP}}/obsrv/v1/datasets/source/config/list"
+						"url": "{{HOST_IP}}/datasets/v1/source/config/list"
 					},
 					"response": []
 				},
@@ -286,14 +283,13 @@
 							}
 						},
 						"url": {
-							"raw": "{{HOST_IP}}/obsrv/v1/datasets/source/config/get/6c3fc8c2-357d-489b-b0c9-afdde6e5c6c0?status=Live",
+							"raw": "{{HOST_IP}}/datasets/v1/source/config/get/6c3fc8c2-357d-489b-b0c9-afdde6e5c6c0?status=Live",
 							"host": [
 								"{{HOST_IP}}"
 							],
 							"path": [
-								"obsrv",
-								"v1",
 								"datasets",
+								"v1",
 								"source",
 								"config",
 								"get",
@@ -326,7 +322,7 @@
 								}
 							}
 						},
-						"url": "{{HOST_IP}}/obsrv/v1/datasets/source/config/create"
+						"url": "{{HOST_IP}}/datasets/v1/source/config/create"
 					},
 					"response": []
 				},
@@ -347,7 +343,7 @@
 								}
 							}
 						},
-						"url": "{{HOST_IP}}/obsrv/v1/datasets/source/config/update"
+						"url": "{{HOST_IP}}/datasets/v1/source/config/update"
 					},
 					"response": []
 				}
@@ -357,7 +353,7 @@
 			"name": "Query (Data OUT APIs)",
 			"item": [
 				{
-					"name": "native query request",
+					"name": "native query with params",
 					"request": {
 						"auth": {
 							"type": "noauth"
@@ -373,12 +369,12 @@
 								}
 							}
 						},
-						"url": "{{HOST_IP}}/obsrv/v1/data/query/dataset-1_dataset-1.1_DAY"
+						"url": "{{HOST_IP}}/data/v1/query/dataset-1_dataset-1.1_DAY"
 					},
 					"response": []
 				},
 				{
-					"name": "sql query request",
+					"name": "sql query with params",
 					"event": [
 						{
 							"listen": "prerequest",
@@ -405,7 +401,43 @@
 								}
 							}
 						},
-						"url": "{{HOST_IP}}/obsrv/v1/data/sql-query/dataset-1_dataset-1.1_DA"
+						"url": "{{HOST_IP}}/data/v1/sql-query/dataset-1_dataset-1.1_DA"
+					},
+					"response": []
+				},
+				{
+					"name": "native query",
+					"request": {
+						"method": "POST",
+						"header": [
+							{
+								"key": "Content-Type",
+								"value": "application/json"
+							}
+						],
+						"body": {
+							"mode": "raw",
+							"raw": "{\n    \"context\": {\n        \"dataSource\": \"dataset-1\"\n    },\n    \"query\": {\n        \"queryType\": \"timeseries\",\n        \"dataSource\": \"dataset-1\",\n        \"intervals\": {\n            \"type\": \"intervals\",\n            \"intervals\": [\n                \"2023-06-15/2023-07-11\"\n            ]\n        },\n        \"granularity\": \"week\",\n        \"aggregations\": [\n            {\n                \"type\": \"count\",\n                \"name\": \"total_records\"\n            }\n        ]\n    }\n}"
+						},
+						"url": "{{HOST_IP}}/data/v1/query"
+					},
+					"response": []
+				},
+				{
+					"name": "sql-query",
+					"request": {
+						"method": "POST",
+						"header": [
+							{
+								"key": "Content-Type",
+								"value": "application/json"
+							}
+						],
+						"body": {
+							"mode": "raw",
+							"raw": "{\n    \"context\": {\n        \"dataSource\": \"dataset-1\"\n    },\n    \"querySql\": {\n        \"query\": \"SELECT COUNT(*) AS totalRatingsCount FROM \\\"dataset-1\\\" LIMIT 100\"\n    }\n}"
+						},
+						"url": "{{HOST_IP}}/data/v1/sql-query"
 					},
 					"response": []
 				}
@@ -431,7 +463,7 @@
 								}
 							}
 						},
-						"url": "{{HOST_IP}}/obsrv/v1/data/create/sb-telemetry"
+						"url": "{{HOST_IP}}/data/v1/in/sb-telemetry"
 					},
 					"response": []
 				},
@@ -452,7 +484,7 @@
 								}
 							}
 						},
-						"url": "{{HOST_IP}}/obsrv/v1/data/create/sb-telemetry"
+						"url": "{{HOST_IP}}/data/v1/in/sb-telemetry"
 					},
 					"response": []
 				},
@@ -473,7 +505,7 @@
 								}
 							}
 						},
-						"url": "{{HOST_IP}}/obsrv/v1/data/create/sb-telemetry-user"
+						"url": "{{HOST_IP}}/data/v1/in/sb-telemetry-user"
 					},
 					"response": []
 				},
@@ -491,7 +523,7 @@
 								}
 							}
 						},
-						"url": "{{HOST_IP}}/obsrv/v1/data/submit/ingestion"
+						"url": "{{HOST_IP}}/data/v1/submit/ingestion"
 					},
 					"response": []
 				}
@@ -501,7 +533,7 @@
 			"name": "Exhaust APIs",
 			"item": [
 				{
-					"name": "New Request",
+					"name": "Dataset exhaust",
 					"request": {
 						"method": "GET",
 						"header": [
@@ -511,16 +543,15 @@
 							}
 						],
 						"url": {
-							"raw": "{{HOST_IP}}/obsrv/v1/data/exhaust/sb-telemetry?from=2023-07-01&to=2023-07-31&type=transformed",
+							"raw": "{{HOST_IP}}/data/v1/exhaust/dataset-1?from=2023-07-01&to=2023-07-31&type=transformed",
 							"host": [
 								"{{HOST_IP}}"
 							],
 							"path": [
-								"obsrv",
-								"v1",
 								"data",
+								"v1",
 								"exhaust",
-								"sb-telemetry"
+								"dataset-1"
 							],
 							"query": [
 								{
@@ -542,5 +573,32 @@
 				}
 			]
 		}
+	],
+	"event": [
+		{
+			"listen": "prerequest",
+			"script": {
+				"type": "text/javascript",
+				"exec": [
+					""
+				]
+			}
+		},
+		{
+			"listen": "test",
+			"script": {
+				"type": "text/javascript",
+				"exec": [
+					""
+				]
+			}
+		}
+	],
+	"variable": [
+		{
+			"key": "HOST_IP",
+			"value": "localhost:3005",
+			"type": "string"
+		}
 	]
 }