Subnet creation fails with 400 Bad Request

[vgarg]

I have an existing VNET called "virtualNetwork1" with CIDR range of 10.0.0.0/19. I am trying to create a new Subnet called "vivekazcluster1-subnet" with CIDR 10.0.0.0/24 within this VNET using the Go SDK.

Here is the code snippet that I am using:

// create a subnet
snProps := network.Subnet{}
snProps.Properties.AddressPrefix = cidrBlock
subnet, err := subnetsClient.CreateOrUpdate(
    resGroup, vnetName, name, snProps)
if err != nil {
    return nil, err
}

The subnet creation however fails with the following error:
autorest:DoErrorUnlessStatusCode PUT https://management.azure.com/subscriptions/****-****-445a-****/resourceGroups/vivekmockpdc/providers/Microsoft.Network/virtualnetworks/virtualNetwork1/subnets/vivekazcluster1-subnet?api-version=2015-05-01-preview failed with 400 Bad Request

Audit logs provide a little more insight, but not enough to pin point the problem from my perspective:

statusCode:BadRequest
serviceRequestId:09d84c77-8c3d-44c3-a1c8-a38e836c69ca
statusMessage:{"error":{"code":"InvalidRequestFormat","message":"Cannot parse the request.","details":[{"code":"MissingJsonReferenceId","message":"Value for reference id is missing. Path properties.networkSecurityGroup."}]}}

Note, I have no issues creating a subnet using the azure CLI. Any workaround would be greatly appreciated.

Vivek

[brendandixon]

@vgarg The returned message (thank you for including it!) says that, despite the documentation stating otherwise (see https://msdn.microsoft.com/en-us/library/azure/mt163621.aspx), the service is expecting an Id within the NetworkSecurityGroup of the Properties portion of the Subnet struct.

Did you try providing an Id? Can you trace the JSON sent from the CLI? Does it include the Id?

Either way, this appears to be not strictly a bug with the SDK, but a lack of clarity regarding which fields are or are not required.

Make sense?

[ahmetb]

Thanks for reporting @vgarg. Looks like this is the same issue as #206, please subscribe to the discussion there.

[ahmetb]

Oops reopening I might be wrong.

[brendandixon]

@ahmetalpbalkan Different issue than #206.

[vgarg]

@brendandixon I did try setting the Id value ( snProps.Properties.NetworkSecurityGroup.Id = "None") and (snProps.Properties.NetworkSecurityGroup.Id = "") but both times it failed with the same error.

I haven't tried tracing the output from CLI. Is there an option I can turn on? I tried providing -v, but that doesn't provide the JSON.

[ahmetb]

@vgarg please try -vv on cli.

[vgarg]

@ahmetalpbalkan thanks. Looks like azure CLI is not sending any Id. Here is the output:

warn: using default address space 10.0.0.0/22
verbose: Creating subnet "ghi"
silly: requestOptions
silly: {
silly: rawResponse: false,
silly: queryString: {},
silly: method: 'PUT',
silly: headers: {
silly: Content-Type: 'application/json',
silly: Content-Length: 46,
silly: user-agent: 'WindowsAzureXplatCLI/0.9.6'
silly: },
silly: url: 'https://management.azure.com/subscriptions/****-******/resourceGroups/vivekmockpdc/providers/Microsoft.Network/virtualnetworks/virtualNetwork1/subnets/ghi?api-version=2015-05-01-preview',
silly: body: '{"properties":{"addressPrefix":"10.0.0.0/22"}}'
silly: }

[brendandixon]

@ahmetalpbalkan @vgarg I wonder if we have a Swagger error. The URL the CLI uses differs slightly from that used in the SDK (note the final path component is not submitted in the CLI). This makes some sense: The PUT to create, in a sense, does not yet have a name whereas an update would. If the server made assumptions, such as expecting an Id for an Update, the error becomes clear.

@vgarg We can test this. Open the file azure-sdk-for-go/arm/network/subnets.go and remove the final "{subnetName} parameter from line 260 (within the CreateOrUpdateRequestPreparer). Then try your Create request again. Updates will fail, but let's see if this makes the Create succeed. If it does, then we've found our error: A bad URL specified in the Swagger REST API files that guide the generation of the SDK.

[vgarg]

@brendandixon perhaps I am misunderstanding your comment above, but from the output I posted from CLI above, the CLI URL does include the subnet name as a final segment in the URL. Its "ghi".

I tried removing the last segment and got the following error:
autorest:DoErrorUnlessStatusCode PUT https://management.azure.com/subscriptions/***/resourceGroups/vivekmockpdc/providers/Microsoft.Network/virtualnetworks/virtualNetwork1/subnets?api-version=2015-05-01-preview failed with 405 Method Not Allowed

DId I try the right change?

[brendandixon]

@vgarg My mistake. I apologize. I misread the URL. Let's get the JSON sent from the SDK.

First, create an "inspector" in your code:

import (
    "io/ioutil"
    "net/http"
)

type inspector struct{}

func (i inspector) WithInspection() autorest.PrepareDecorator {
    return func(p autorest.Preparer) autorest.Preparer {
        return autorest.PreparerFunc(func(r *http.Request) (*http.Request, error) {
            b, _ := ioutil.ReadAll(r.Body)
            fmt.Printf("Body: %s\n", string(b))
            r.Body = ioutil.NopCloser(bytes.NewReader(b))
            return p.Prepare(r)
        })
    }
}

Then, add the inspector to the client:

c := network.NewSubnetsClient(xxxxx)
c.RequestInspector := inspector{}

Then, run the code and save the output. It should emit the JSON to stdout.

Thank you,

[vgarg]

@brendandixon thanks. I tried two different tests. One with no value for Id. And another with Id="None". Both cases fail with the same error. Here is the output:

Case 1:
Body: {"properties":{"addressPrefix":"10.0.0.0/24","networkSecurityGroup":{}}}
Error: autorest:DoErrorUnlessStatusCode PUT https://management.azu
re.com/subscriptions/***/resourceGroups/vivekmockpdc/providers/Microsoft.Network/virtualnetworks/virtualNetw
ork1/subnets/vivekazcluster1-subnet?api-version=2015-05-01-preview failed with 400 Bad Request

Case 2:
Body: {"properties":{"addressPrefix":"10.0.0.0/24","networkSecurityGroup":{"id":"None"}}}
Error: autorest:DoErrorUnlessStatusCode PUT https://management.azu
re.com/subscriptions/****/resourceGroups/vivekmockpdc/providers/Microsoft.Network/virtualnetworks/virtualNetw
ork1/subnets/vivekazcluster1-subnet?api-version=2015-05-01-preview failed with 400 Bad Request

I wonder if the issue is indeed same as #206 in that an empty security group in JSON is not liked by the the REST API.

Btw: Is there another value I should try to represent "None" as a way to get around?

[vgarg]

Just as a way to temporarily get around, I changed the NetworkSecurityGroup to be a pointer based on the suggestion in #206 as follows:

NetworkSecurityGroup *struct {
Id string json:"id,omitempty"
} json:"networkSecurityGroup,omitempty"

With this change, the subnet got created, however I still received the following error:

autorest:WithErrorUnlessStatusCode PUT https://management.azure.com/subscriptions/ddc30f29-55a6-445a-8e84-f43efa7ec295/resourceGroups/vivekmockpdc/providers/Microsoft.Network/virtualnetworks/virtualNetwork1/subnets/vivekazcluster1-subnet?api-version=2015-05-01-preview failed with 201 Created

[ahmetb]

@vgarg this new error sounds like #205.

[brendandixon]

@vgarg Great. I am working on both #206 and #205. Fixing #206 will be a breaking change. The #205 is a simple error that misinterprets a success code.

Since there are no issues, I suspect you can close this and track them.

[vgarg]

@brendandixon @ahmetalpbalkan thanks guys. Marking it closed.

I am currently thinking of temporarily working around both #206 and #205 while waiting for the real fixes. Is there a change I can make in my copy of the code for that would prevent #205?

[brendandixon]

@vgarg Yes. Change line 218 from:

autorest.WithErrorUnlessOK(),

to

autorest.WithErrorUnlessStatusCode(http.StatusOK, http.StatusCreated)

That should do it!

[vgarg]

Thanks @brendandixon . That worked.

Do you have any ballpark guidance on when the #205 and #206 may be fixed? Hours or days or weeks?

[brendandixon]

@vgarg There are a number of changes pending I'd like to roll up into a tagged beta release. I'd guess maybe a week?

[vgarg]

OK. Thanks @brendandixon