Charging (#58)

* add charging rules & billing domain

* setup billing server by config

* fix: waiting group usage for billing server

* update pdu charging data filter

* frontend: add UE CHARGING RECORD

* finish SubscriberCreate & SubscriberUpdate

* finish Charging Config in SubscriberRead.tsx

* fix: DELETE and PUT policyData.ues.chargingData when editing Subscriber's data

* fix: CheckAuth in GetChargingRecord()

* fix golangci-linter & reduce frontend vulnerabilities

* remove useless code & log in api_webui.go

* config/TLS -> cert

* fix webuicfg.yaml

* Enable to add multiple flow rules in 1 DNN

* remove SubscriberUpdate.tsx and integrate it into SubscriberCreate.tsx

* change all qfi to QosRef

* fix React Component's id naming style

* When Offline Charging, disable Quota input box

* update UI

* change default IP Filter

* update UI

* fix: when offline, quota != 0

* refactor

* implement per S-NSSAI charging config

* use frontend/build instead of ./public

* add per flow UE Charging Record

* fix: 2 flow rules use same charging config

* add expand icon

* update readme.md

* add PerFlowTableView as a React Component

* fix: undefined object use map()

* fix: the conflict of replicate IP Filter

* when delete a subscriber, remove it's CDR file

* fix: per slice charging config disappered when edit subscriber data

* fix: dnn, filter should not omit empty

* update util's hash

* fix: per slice Charging Config display error

* fix: backend pdu level quota wasn't sent to frontend

* fix: when no Charging record && sort charging record by Filter

* fix: duplicated SMF Information in REALTIME STATUS

* remove cdr file by path

* add snssai, dnn to FlowInfo & RatingGroupDataUsage

* redesign backend: GetChargingRecord()

* frontend: implemented multi snssai expand table for UE Charging Record Page

* remove redundant log

* sort flow level charging record

* Bump github.com/gin-gonic/gin from 1.9.0 to 1.9.1 (#42)

Bumps [github.com/gin-gonic/gin](https://github.com/gin-gonic/gin) from 1.9.0 to 1.9.1.
- [Release notes](https://github.com/gin-gonic/gin/releases)
- [Changelog](https://github.com/gin-gonic/gin/blob/master/CHANGELOG.md)
- [Commits](gin-gonic/gin@v1.9.0...v1.9.1)

---
updated-dependencies:
- dependency-name: github.com/gin-gonic/gin
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: tim-ywliu <[email protected]>

* Query to NRF for NFProfile instead of getting from mongodb. (#63)

* Fix: MSISDN -> GPSI (#65)

* fix: msisdn -> identityData

* change some function name to GPSI style

* SubsListIE.Msisdn -> SubsListIE.Gpsi

* continue to fix the gpsi format bug in api_webui.go

* Bump @adobe/css-tools from 4.3.1 to 4.3.2 in /frontend (#67)

Bumps [@adobe/css-tools](https://github.com/adobe/css-tools) from 4.3.1 to 4.3.2.
- [Changelog](https://github.com/adobe/css-tools/blob/main/History.md)
- [Commits](https://github.com/adobe/css-tools/commits)

---
updated-dependencies:
- dependency-name: "@adobe/css-tools"
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump golang.org/x/crypto from 0.14.0 to 0.17.0 (#68)

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.14.0 to 0.17.0.
- [Commits](golang/crypto@v0.14.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix: rebase and fix linter error

* add singleNssais sd:112233

* Refactor: refactor WebUI and fix bugs

* Feature: support offline charging

* Fix: remove unused code

* UI Modify and fix bugs

* Change default values and enhance UX

* PDU Level to silce level

* Remove comment code and add error handling

* Add postman collection

* Frontend code improvement and cleaner

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: roy19991013 <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: tim-ywliu <[email protected]>
Co-authored-by: Kunihiro Ishiguro <[email protected]>
Co-authored-by: CTFang@WireLab <[email protected]>

backend/WebUI/api_charging.go

@@ -0,0 +1,260 @@
+package WebUI
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"net/http"
+	"os"
+	"reflect"
+	"strconv"
+
+	"github.com/gin-gonic/gin"
+	"go.mongodb.org/mongo-driver/bson"
+
+	"github.com/free5gc/chf/cdr/asn"
+	"github.com/free5gc/chf/cdr/cdrFile"
+	"github.com/free5gc/chf/cdr/cdrType"
+	"github.com/free5gc/openapi/models"
+	"github.com/free5gc/util/mongoapi"
+	"github.com/free5gc/webconsole/backend/logger"
+	"github.com/free5gc/webconsole/backend/webui_context"
+)
+
+// Get vol from CDR
+// TS 32.297: Charging Data Record (CDR) file format and transfer
+func parseCDR(supi string) (map[int64]RatingGroupDataUsage, error) {
+	logger.BillingLog.Traceln("parseCDR")
+	fileName := "/tmp/webconsole/" + supi + ".cdr"
+	if _, err := os.Stat(fileName); err != nil {
+		return nil, err
+	}
+
+	newCdrFile := cdrFile.CDRFile{}
+
+	newCdrFile.Decoding(fileName)
+	dataUsage := make(map[int64]RatingGroupDataUsage)
+
+	for _, cdr := range newCdrFile.CdrList {
+		recvByte := cdr.CdrByte
+		val := reflect.New(reflect.TypeOf(&cdrType.ChargingRecord{}).Elem()).Interface()
+		err := asn.UnmarshalWithParams(recvByte, val, "")
+		if err != nil {
+			logger.BillingLog.Errorf("parseCDR error when unmarshal with params: %+v", err)
+			continue
+		}
+
+		chargingRecord := *(val.(*cdrType.ChargingRecord))
+
+		for _, multipleUnitUsage := range chargingRecord.ListOfMultipleUnitUsage {
+			rg := multipleUnitUsage.RatingGroup.Value
+			du := dataUsage[rg]
+
+			du.Snssai = fmt.Sprintf("%02d", chargingRecord.PDUSessionChargingInformation.NetworkSliceInstanceID.SST.Value) +
+				string(chargingRecord.PDUSessionChargingInformation.NetworkSliceInstanceID.SD.Value)
+
+			du.Dnn = string(chargingRecord.PDUSessionChargingInformation.DataNetworkNameIdentifier.Value)
+
+			for _, usedUnitContainer := range multipleUnitUsage.UsedUnitContainers {
+				du.TotalVol += usedUnitContainer.DataTotalVolume.Value
+				du.UlVol += usedUnitContainer.DataVolumeUplink.Value
+				du.DlVol += usedUnitContainer.DataVolumeDownlink.Value
+			}
+
+			dataUsage[rg] = du
+		}
+	}
+
+	return dataUsage, nil
+}
+
+func GetChargingData(c *gin.Context) {
+	logger.BillingLog.Info("Get Charging Data")
+	setCorsHeader(c)
+
+	if !CheckAuth(c) {
+		c.JSON(http.StatusUnauthorized, gin.H{"cause": "Illegal Token"})
+		return
+	}
+
+	chargingMethod, exist := c.Params.Get("chargingMethod")
+	if !exist {
+		c.JSON(http.StatusBadRequest, gin.H{"cause": "chargingMethod not provided"})
+		return
+	}
+	logger.BillingLog.Traceln(chargingMethod)
+
+	if chargingMethod != "Offline" && chargingMethod != "Online" {
+		c.JSON(http.StatusBadRequest, gin.H{"cause": "not support chargingMethod" + chargingMethod})
+		return
+	}
+
+	filter := bson.M{"chargingMethod": chargingMethod}
+	chargingDataInterface, err := mongoapi.RestfulAPIGetMany(chargingDataColl, filter)
+	if err != nil {
+		logger.BillingLog.Errorf("mongoapi error: %+v", err)
+	}
+
+	chargingDataBsonA := toBsonA(chargingDataInterface)
+
+	c.JSON(http.StatusOK, chargingDataBsonA)
+}
+
+func GetChargingRecord(c *gin.Context) {
+	logger.BillingLog.Info("Get Charging Record")
+	setCorsHeader(c)
+
+	if !CheckAuth(c) {
+		c.JSON(http.StatusUnauthorized, gin.H{"cause": "Illegal Token"})
+		return
+	}
+
+	webuiSelf := webui_context.GetSelf()
+	webuiSelf.UpdateNfProfiles()
+
+	// Get supi of UEs
+	var uesJsonData interface{}
+	if amfUris := webuiSelf.GetOamUris(models.NfType_AMF); amfUris != nil {
+		requestUri := fmt.Sprintf("%s/namf-oam/v1/registered-ue-context", amfUris[0])
+
+		res, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, requestUri, nil)
+		if err != nil {
+			logger.ProcLog.Error(err)
+			c.JSON(http.StatusInternalServerError, gin.H{})
+			return
+		}
+		resp, res_err := httpsClient.Do(res)
+		if res_err != nil {
+			logger.ProcLog.Error(err)
+			c.JSON(http.StatusInternalServerError, gin.H{})
+			return
+		}
+
+		defer func() {
+			if closeErr := resp.Body.Close(); closeErr != nil {
+				logger.ProcLog.Error(closeErr)
+			}
+		}()
+
+		err = json.NewDecoder(resp.Body).Decode(&uesJsonData)
+		if err != nil {
+			logger.BillingLog.Error(err)
+		}
+	}
+
+	// build charging records
+	uesBsonA := toBsonA(uesJsonData)
+	chargingRecordsBsonA := make([]interface{}, 0, len(uesBsonA))
+
+	type OfflineSliceTypeMap struct {
+		supi           string
+		snssai         string
+		dnn            string
+		unitcost       int64
+		flowTotalVolum int64
+		flowTotalUsage int64
+	}
+	// Use for sum all the flow-based charging, and add to the slice at the end.
+	offlineChargingSliceTypeMap := make(map[string]OfflineSliceTypeMap)
+
+	for _, ueData := range uesBsonA {
+		ueBsonM := toBsonM(ueData)
+
+		supi := ueBsonM["Supi"].(string)
+
+		ratingGroupDataUsages, err := parseCDR(supi)
+		if err != nil {
+			logger.BillingLog.Warnln(err)
+			continue
+		}
+
+		for rg, du := range ratingGroupDataUsages {
+			filter := bson.M{"ratingGroup": rg}
+			chargingDataInterface, err := mongoapi.RestfulAPIGetOne(chargingDataColl, filter)
+			if err != nil {
+				logger.ProcLog.Errorf("PostSubscriberByID err: %+v", err)
+			}
+			if len(chargingDataInterface) == 0 {
+				logger.BillingLog.Warningf("ratingGroup: %d not found in mongoapi, may change the rg id", rg)
+				continue
+			}
+
+			var chargingData ChargingData
+			err = json.Unmarshal(mapToByte(chargingDataInterface), &chargingData)
+			if err != nil {
+				logger.BillingLog.Error(err)
+			}
+			logger.BillingLog.Debugf("add ratingGroup: %d, supi: %s, method: %s", rg, supi, chargingData.ChargingMethod)
+
+			switch chargingData.ChargingMethod {
+			case "Offline":
+				unitcost, err := strconv.ParseInt(chargingData.UnitCost, 10, 64)
+				if err != nil {
+					logger.BillingLog.Error("Offline unitCost strconv: ", err.Error())
+					unitcost = 1
+				}
+
+				key := chargingData.UeId + chargingData.Snssai
+				pdu_level, exist := offlineChargingSliceTypeMap[key]
+				if !exist {
+					pdu_level = OfflineSliceTypeMap{}
+				}
+				if chargingData.Filter != "" {
+					// Flow-based charging
+					du.Usage = du.TotalVol * unitcost
+					pdu_level.flowTotalUsage += du.Usage
+					pdu_level.flowTotalVolum += du.TotalVol
+				} else {
+					// Slice-level charging
+					pdu_level.snssai = chargingData.Snssai
+					pdu_level.dnn = chargingData.Dnn
+					pdu_level.supi = chargingData.UeId
+					pdu_level.unitcost = unitcost
+				}
+				offlineChargingSliceTypeMap[key] = pdu_level
+			case "Online":
+				tmpInt, err1 := strconv.Atoi(chargingData.Quota)
+				if err1 != nil {
+					logger.BillingLog.Error("Quota strconv: ", err1, rg, du, chargingData)
+				}
+				du.QuotaLeft = int64(tmpInt)
+			}
+			du.Snssai = chargingData.Snssai
+			du.Dnn = chargingData.Dnn
+			du.Supi = supi
+			du.Filter = chargingData.Filter
+
+			ratingGroupDataUsages[rg] = du
+			chargingRecordsBsonA = append(chargingRecordsBsonA, toBsonM(du))
+		}
+	}
+	for idx, record := range chargingRecordsBsonA {
+		tmp, err := json.Marshal(record)
+		if err != nil {
+			logger.BillingLog.Errorln("Marshal chargingRecordsBsonA error:", err.Error())
+			continue
+		}
+
+		var rd RatingGroupDataUsage
+
+		err = json.Unmarshal(tmp, &rd)
+		if err != nil {
+			logger.BillingLog.Errorln("Unmarshall RatingGroupDataUsage error:", err.Error())
+			continue
+		}
+
+		if rd.Filter != "" {
+			// Skip the Flow-based charging
+			continue
+		}
+
+		key := rd.Supi + rd.Snssai
+		if val, exist := offlineChargingSliceTypeMap[key]; exist {
+			rd.Usage += val.flowTotalUsage
+			rd.Usage += (rd.TotalVol - val.flowTotalVolum) * val.unitcost
+			chargingRecordsBsonA[idx] = toBsonM(rd)
+		}
+	}
+
+	c.JSON(http.StatusOK, chargingRecordsBsonA)
+}