Provide standard structure for DBS errors

[vkuznet]

Per discussion in dmwm/WMCore#10961 we should adopt standard structure for DBS errors which should include DBS specific error codes.

[vkuznet]

In addition, we need common Python parser to parse DBS error structure. It can be a separate module or can be part of DBSClient codebase.

[vkuznet]

The new DBS error codes are in place starting v00.04.26 tag. The codebase can be found in dbs/errors.go and it represents the following error codes:

        GenericErrorCode        = iota + 100 // generic DBS error
        DatabaseErrorCode                    // 101 database error
        TransactionErrorCode                 // 102 transaction error
        QueryErrorCode                       // 103 query error
        RowsScanErrorCode                    // 104 row scan error
        SessionErrorCode                     // 105 db session error
        CommitErrorCode                      // 106 db commit error
        ParseErrorCode                       // 107 parser error
        LoadErrorCode                        // 108 loading error, e.g. load template
        GetIDErrorCode                       // 109 get id db error
        InsertErrorCode                      // 110 db insert error
        UpdateErrorCode                      // 111 update error
        LastInsertErrorCode                  // 112 db last insert error
        ValidateErrorCode                    // 113 validation error
        PatternErrorCode                     // 114 pattern error
        DecodeErrorCode                      // 115 decode error
        EncodeErrorCode                      // 116 encode error
        ContentTypeErrorCode                 // 117 content type error
        ParametersErrorCode                  // 118 parameters error
        NotImplementedApiCode                // 119 not implemented API error
        ReaderErrorCode                      // 120 io reader error
        WriterErrorCode                      // 121 io writer error
        UnmarshalErrorCode                   // 122 json unmarshal error
        MarshalErrorCode                     // 123 marshal error
        HttpRequestErrorCode                 // 124 HTTP request error
        MigrationErrorCode                   // 125 Migration error
        RemoveErrorCode                      // 126 remove error
        InvalidRequestErrorCode              // 127 invalid request error

The DBS web handler now wraps HTTP failure request with two common structures: HTTPError and DBSError which are part of ServerError, see

// HTTPError represents HTTP error structure
type HTTPError struct {
        Method         string `json:"method"`           // HTTP method
        HTTPCode       int    `json:"code"`             // HTTP status code from IANA
        Timestamp      string `json:"timestamp"`        // timestamp of the error
        Path           string `json:"path"`             // URL path
        UserAgent      string `json:"user_agent"`       // http user-agent field
        XForwardedHost string `json:"x_forwarded_host"` // http.Request X-Forwarded-Host
        XForwardedFor  string `json:"x_forwarded_for"`  // http.Request X-Forwarded-For
        RemoteAddr     string `json:"remote_addr"`      // http.Request remote address
}

// ServerError represents HTTP server error structure
type ServerError struct {
        DBSError  error     `json:"error"`     // DBS error
        HTTPError HTTPError `json:"http"`      // HTTP section of the error
        Exception int       `json:"exception"` // for compatibility with Python server
        Type      string    `json:"type"`      // for compatibility with Python server
        Message   string    `json:"message"`   // for compatibility with Python server 
}

Finally, on a client side a particular error will look like this:

curl .. https://cmsweb-testbed.cern.ch/dbs2go/datatiers?data_tier_name=1

[
  {
    "error": {
      "reason": "DBSError Code:114 Description:DBS validation error when wrong pattern is provided Function:dbs.validator.Check Message:unable to match 'data_tier_name' value '1' Error: invalid parameter(s)",
      "message": "not str type",
      "function": "dbs.Validate",
      "code": 113
    },
    "http": {
      "method": "GET",
      "code": 400,
      "timestamp": "2022-02-04 14:47:47.058650325 +0000 UTC m=+86568.954530362",
      "path": "/dbs2go/datatiers?data_tier_name=1",
      "user_agent": "curl/7.59.0",
      "x_forwarded_host": "cmsweb-testbed.cern.ch",
      "x_forwarded_for": "188.185.79.81",
      "remote_addr": "188.184.75.219:20274"
    },
    "exception": 400,
    "type": "HTTPError",
    "message": "DBSError Code:113 Description:DBS validation error, e.g. input parameter does not match lexicon rules Function:dbs.Validate Message:not str type Error: nested DBSError Code:114 Description:DBS validation error when wrong pattern is provided Function:dbs.validator.Check Message:unable to match 'data_tier_name' value '1' Error: invalid parameter(s)"
  }
]

The returned HTTP response contains all relevant information to identify DBS error, its code, and user client info (such as host, user-agent, etc).

Each DBSError can be wrapped into another one to provide relevant information how error was originated (similar to Python traceback).