The functions, entities, and methods in this library have the wide goal of providing access to vCD functionality using Go clients. A more focused goal is to support the Terraform Provider for vCD. When in doubt about the direction of development, we should facilitate the path towards making the code usable and maintainable in the above project.
A new entity must have its type defined in types/56/types.go. If the type is not already there, it should be
added using the vCD API, and possibly reusing components already defined
in types.go.
The new entity should have a structure in entity.go as
type Entity struct {
Entity *types.Entity
client *VCDClient
// Optional, in some cases: Parent *Parent
}The entity should have at least the following:
(parent *Parent) CreateEntityAsync(input *types.Entity) (Task, error)
(parent *Parent) CreateEntity(input *types.Entity) (*Entity, error)
The second form will invoke the *Async method, run task.WaitCompletion(), and then retrieving the new entity
from the parent and returning it.
If the API does not provide a task, the second method will be sufficient.
If the structure is exceedingly complex, we can use two approaches:
- if the parameters needed to create the entity are less than 4, we can pass them as argument
(parent *Parent) CreateEntityAsync(field1, field2 string, field3 bool) (Task, error)- If there are too many parameters to pass, we can create a simplified structure:
type EntityInput struct {
field1 string
field2 string
field3 bool
field4 bool
field5 int
field6 string
field7 []string
}
(parent *Parent) CreateEntityAsync(simple EntityInput) (Task, error)The latter approach should be preferred when the simplified structure would be a one-to-one match with the corresponding resource in Terraform.
Calls to the vCD API should not be sent directly, but using one of the following functions from `api.go:
// Helper function creates request, runs it, check responses and parses out interface from response.
// pathURL - request URL
// requestType - HTTP method type
// contentType - value to set for "Content-Type"
// errorMessage - error message to return when error happens
// payload - XML struct which will be marshalled and added as body/payload
// out - structure to be used for unmarshalling xml
// E.g. unmarshalledAdminOrg := &types.AdminOrg{}
// client.ExecuteRequest(adminOrg.AdminOrg.HREF, http.MethodGet, "", "error refreshing organization: %s", nil, unmarshalledAdminOrg)
func (client *Client) ExecuteRequest(pathURL, requestType, contentType, errorMessage string, payload, out interface{}) (*http.Response, error)// Helper function creates request, runs it, checks response and parses task from response.
// pathURL - request URL
// requestType - HTTP method type
// contentType - value to set for "Content-Type"
// errorMessage - error message to return when error happens
// payload - XML struct which will be marshalled and added as body/payload
// E.g. client.ExecuteTaskRequest(updateDiskLink.HREF, http.MethodPut, updateDiskLink.Type, "error updating disk: %s", xmlPayload)
func (client *Client) ExecuteTaskRequest(pathURL, requestType, contentType, errorMessage string, payload interface{}) (Task, error) // Helper function creates request, runs it, checks response and do not expect any values from it.
// pathURL - request URL
// requestType - HTTP method type
// contentType - value to set for "Content-Type"
// errorMessage - error message to return when error happens
// payload - XML struct which will be marshalled and added as body/payload
// E.g. client.ExecuteRequestWithoutResponse(catalogItemHREF.String(), http.MethodDelete, "", "error deleting Catalog item: %s", nil)
func (client *Client) ExecuteRequestWithoutResponse(pathURL, requestType, contentType, errorMessage string, payload interface{}) error // ExecuteRequestWithCustomError sends the request and checks for 2xx response. If the returned status code
// was not as expected - the returned error will be unmarshaled to `errType` which implements Go's standard `error`
// interface.
func (client *Client) ExecuteRequestWithCustomError(pathURL, requestType, contentType, errorMessage string,
payload interface{}, errType error) (*http.Response, error) In addition to saving code and time by reducing the boilerplate, these functions also trigger debugging calls that make the code
easier to monitor.
Using any of the above calls will result in the standard log i
(See LOGGING.md) recording all the requests and responses
on demand, and also triggering debug output for specific calls (see enableDebugShowRequest and enableDebugShowResponse
and the corresponding disable* in api.go).
Each entity should have the following methods:
// OPTIONAL
(parent *Parent) GetEntityByHref(href string) (*Entity, error)
// ALWAYS
(parent *Parent) GetEntityByName(name string) (*Entity, error)
(parent *Parent) GetEntityById(id string) (*Entity, error)
(parent *Parent) GetEntityByNameOrId(identifier string) (*Entity, error)
For example, the parent for Vdc is Org, the parent for EdgeGateway is Vdc.
If the entity is at the top level (such as Org, ExternalNetwork), the parent is VCDClient.
These methods return a pointer to the entity's structure and a nil error when the search was successful,
a nil pointer and an error in every other case.
When the method can establish that the entity was not found because it did not appear in the
parent's list of entities, the method will return ErrorEntityNotFound.
In no cases we return a nil error when the method fails to find the entity.
The "ALWAYS" methods can optionally add a Boolean refresh argument, signifying that the parent should be refreshed
prior to attempting a search.
Note: We are in the process of replacing methods that don't adhere to the above principles (for example, return a structure instead of a pointer, return a nil error on not-found, etc).
Functions dealing with different versions should use a matrix structure to identify which calls to run according to the highest API version supported by vCD. An example can be found in adminvdc.go.
Note: use this pattern for adding new vCD functionality, which is not available in the earliest API version supported
by the code base (as indicated by Client.APIVersion).
type vdcVersionedFunc struct {
SupportedVersion string
CreateVdc func(adminOrg *AdminOrg, vdcConfiguration *types.VdcConfiguration) (*Vdc, error)
CreateVdcAsync func(adminOrg *AdminOrg, vdcConfiguration *types.VdcConfiguration) (Task, error)
UpdateVdc func(adminVdc *AdminVdc) (*AdminVdc, error)
UpdateVdcAsync func(adminVdc *AdminVdc) (Task, error)
}
var vdcVersionedFuncsV95 = vdcVersionedFuncs{
SupportedVersion: "31.0",
CreateVdc: createVdc,
CreateVdcAsync: createVdcAsync,
UpdateVdc: updateVdc,
UpdateVdcAsync: updateVdcAsync,
}
var vdcVersionedFuncsV97 = vdcVersionedFuncs{
SupportedVersion: "32.0",
CreateVdc: createVdcV97,
CreateVdcAsync: createVdcAsyncV97,
UpdateVdc: updateVdcV97,
UpdateVdcAsync: updateVdcAsyncV97,
}
var vdcVersionedFuncsByVcdVersion = map[string]vdcVersionedFuncs{
"vdc9.5": vdcVersionedFuncsV95,
"vdc9.7": vdcVersionedFuncsV97,
"vdc10.0": vdcVersionedFuncsV97
}
func (adminOrg *AdminOrg) CreateOrgVdc(vdcConfiguration *types.VdcConfiguration) (*Vdc, error) {
apiVersion, err := adminOrg.client.maxSupportedVersion()
if err != nil {
return nil, err
}
vdcFunctions, ok := vdcVersionedFuncsByVcdVersion["vdc"+apiVersionToVcdVersion[apiVersion]]
if !ok {
return nil, fmt.Errorf("no entity type found %s", "vdc"+apiVersion)
}
if vdcFunctions.CreateVdc == nil {
return nil, fmt.Errorf("function CreateVdc is not defined for %s", "vdc"+apiVersion)
}
util.Logger.Printf("[DEBUG] CreateOrgVdc call function for version %s", vdcFunctions.SupportedVersion)
return vdcFunctions.CreateVdc(adminOrg, vdcConfiguration)
}
The query engine is a search engine that is based on queries (see query.go) with additional filters.
The query runs through the function client.SearchByFilter (filter_engine.go), which requires a queryType (string),
and a set of criteria (*FilterDef).
We can search by one of the types handled by queryFieldsOnDemand (query_metadata.go), such as
const (
QtVappTemplate = "vappTemplate" // vApp template
QtAdminVappTemplate = "adminVAppTemplate" // vApp template as admin
QtEdgeGateway = "edgeGateway" // edge gateway
QtOrgVdcNetwork = "orgVdcNetwork" // Org VDC network
QtAdminCatalog = "adminCatalog" // catalog
QtCatalogItem = "catalogItem" // catalog item
QtAdminCatalogItem = "adminCatalogItem" // catalog item as admin
QtAdminMedia = "adminMedia" // media item as admin
QtMedia = "media" // media item
)There are two reasons for this limitation:
- If we want to include metadata, we need to add the metadata fields to the list of fields we want the query to fetch.
- Unfortunately, not all fields defined in the corresponding type is accepted by the
fieldsparameter in a query. The fields returned byqueryFieldsOnDemandare the one that have been proven to be accepted.
The FilterDef type is defined as follows (filter_utils.go)
type FilterDef struct {
// A collection of filters (with keys from SupportedFilters)
Filters map[string]string
// A list of metadata filters
Metadata []MetadataDef
// If true, the query will include metadata fields and search for exact values.
// Otherwise, the engine will collect metadata fields and search by regexp
UseMetadataApiFilter bool
}A FilterDef may contain several filters, such as:
criteria := &govcd.FilterDef{
Filters: {
"name": "^Centos",
"date": "> 2020-02-02",
"latest": "true",
},
Metadata: {
{
Key: "dept",
Type: "STRING",
Value: "ST\\w+",
IsSystem: false,
},
},
UseMetadataApiFilter: false,
}The set of criteria above will find an item with name starting with "Centos", created after February 2nd, 2020, with
a metadata key "dept" associated with a value starting with "ST". If more than one item is found, the engine will return
the newest one (because of "latest": "true")
The argument UseMetadataApiFilter, when true, instructs the engine to run the search with metadata values. Meaning that
the query will contain a clause filter=metadata:KeyName==TYPE:Value. If IsSystem is true, the clause will become
filter=metadata@SYSTEM:KeyName==TYPE:Value. This search can't evaluate regular expressions, because it goes directly
to vCD.
An example of SYSTEM metadata values is the set of annotations that the vCD adds to a vApp template when we save a
vApp to a catalog.
"metadata" = {
"vapp.origin.id" = "deadbeef-2913-4ed7-b943-79a91620fd52" // vApp ID
"vapp.origin.name" = "my_vapp_name"
"vapp.origin.type" = "com.vmware.vcloud.entity.vapp"
}
The engine returns a list of QueryItem, and interface that defines several methods used to help evaluate the search
conditions.
Here is an example of how to retrieve a media item. The criteria ask for the newest item created after the 2nd of February 2020, containing a metadata field named "abc", with a non-empty value.
criteria := &govcd.FilterDef{
Filters: map[string]string{
"date":"> 2020-02-02",
"latest": "true",
},
Metadata: []govcd.MetadataDef{
{
Key: "abc",
Type: "STRING",
Value: "\\S+",
IsSystem: false,
},
},
UseMetadataApiFilter: false,
}
queryType := govcd.QtMedia
if vcdClient.Client.IsSysAdmin {
queryType = govcd.QtAdminMedia
}
queryItems, explanation, err := vcdClient.Client.SearchByFilter(queryType, criteria)
if err != nil {
return err
}
if len(queryItems) == 0 {
return fmt.Errorf("no media found with given criteria (%s)", explanation)
}
if len(queryItems) > 1 {
// deal with several items
var itemNames = make([]string, len(queryItems))
for i, item := range queryItems {
itemNames[i] = item.GetName()
}
return fmt.Errorf("more than one media item found by given criteria: %v", itemNames)
}
// retrieve the full entity for the item found
media, err = catalog.GetMediaByHref(queryItems[0].GetHref())The explanation returned by SearchByFilter contains the details of the criteria as they were understood by the
engine, and the detail of how each comparison with other items was evaluated. This is useful to create meaningful error
messages.
To add a type to the search engine, we need the following:
- Add the type to
types.QueryResultRecordsType(types.go), or, if the type exists, make sure it includesMetadata - Add the list of supported fields to
queryFieldsOnDemand(query_metadata.go) - Implement the interface
QueryItem(filter_interface.go), which requires a type localization (such astype QueryMedia types.MediaRecordType) - Add a clause to
resultsToQueryItems(filter_interface.go)
Logs should not be cluttered with excessive detail. However, sometimes we need to provide such detail when hunting for bugs.
We can introduce data inspection points, regulated by the environment variable GOVCD_INSPECT, which uses a convenient
code to activate the inspection at different points.
For example, we can mark the inspection points in the query engine with labels "QE1", "QE2", etc., in the network creation
they will be "NET1", "NET2", etc, and then activate them using
GOVCD_INSPECT=QE2,NET1.
In the code, we use the function dataInspectionRequested(code) that will check whether the environment variable contains
the given code.
Every feature in the library must include testing. See TESTING.md for more info.