[QLDB] Unable to create QLDB Stream in .NET #10513

t1agob · 2020-09-24T15:03:46Z

I am getting an error creating a QLDB Stream from CDK using .NET. The CfnStream class requires a InclusiveStartTime property of type string, which is a UTC date in ISO8601 format. ex: 2019-11-05T13:15:30Z.

I am using the following code to get to this format but I have also tried specifying a regular string in the correct format. It didn't work either.

InclusiveStartTime = $"{DateTime.UtcNow.ToString("s")}Z"

This is the exception I am getting.

Unhandled exception. Amazon.JSII.Runtime.JsiiException: Resolution error: System.ArgumentException: Value has unexpected token type Date (Parameter 'value')

I tried replicating the same CDK stack using TypeScript and it worked.

Reproduction Steps

namespace Cdk
{
    public class MyStack : Stack
    {
        internal MyStack(Construct scope, string id, IStackProps props = null) : base(scope, id, props)
        {
            CfnLedger qldb = new CfnLedger(this, "myLedger", new CfnLedgerProps{
                Name = "myLedger",
                PermissionsMode = "ALLOW_ALL"
            });

            Role QldbToKinesisRole = new Role(this, "qldbToKinesisRole", new RoleProps{
                RoleName = "QldbToKinesisRole",
                AssumedBy = new ServicePrincipal("qldb.amazonaws.com")
            });

            QldbToKinesisRole.AddManagedPolicy(ManagedPolicy.FromAwsManagedPolicyName("AmazonKinesisFullAccess"));

            Stream qldbStream = new Stream(this, "qldbStream", new StreamProps{
                StreamName = "MyKinesisStream",
                ShardCount = 1
            });
            
            Amazon.CDK.AWS.QLDB.CfnStream qldbStreamConfig = new Amazon.CDK.AWS.QLDB.CfnStream(this, "qldbStreamConfig", new Amazon.CDK.AWS.QLDB.CfnStreamProps{
                LedgerName = "myLedger",
                StreamName = "MyKinesisStream",
                InclusiveStartTime = $"{DateTime.UtcNow.ToString("s")}Z",
                RoleArn = QldbToKinesisRole.RoleArn,
                KinesisConfiguration = new Amazon.CDK.AWS.QLDB.CfnStream.KinesisConfigurationProperty{
                    AggregationEnabled = true,
                    StreamArn = qldbStream.StreamArn
                } 
            });
        }
    }
}

What did you expect to happen?

I expected the cdk deploy command to succeed.

What actually happened?

Unhandled exception. Amazon.JSII.Runtime.JsiiException: Resolution error: System.ArgumentException: Value has unexpected token type Date (Parameter 'value')
   at Amazon.JSII.Runtime.Services.Converters.JsiiToFrameworkConverter.InferType(IReferenceMap referenceMap, Object value)
   at Amazon.JSII.Runtime.Services.Converters.ValueConverter.ConvertAny(Type type, IReferenceMap referenceMap, Object value)
   at Amazon.JSII.Runtime.Services.Converters.ValueConverter.TryConvertPrimitive(Type type, IReferenceMap referenceMap, Object value, Boolean isOptional, PrimitiveType primitiveType, Object& result)
   at Amazon.JSII.Runtime.Services.Converters.ValueConverter.TryConvert(IOptionalValue optionalValue, Type type, IReferenceMap referenceMap, Object value, Object& result)
   at Amazon.JSII.Runtime.Services.Converters.JsiiToFrameworkConverter.TryConvertMap(IReferenceMap referenceMap, TypeReference elementTypeInstance, Object value, Object& result)
   at Amazon.JSII.Runtime.Services.Converters.ValueConverter.TryConvertCollection(IReferenceMap referenceMap, Object value, Boolean isOptional, CollectionTypeReference collectionType, Object& result)
   at Amazon.JSII.Runtime.Services.Converters.ValueConverter.TryConvert(IOptionalValue optionalValue, Type type, IReferenceMap referenceMap, Object value, Object& result)
   at Amazon.JSII.Runtime.CallbackExtensions.<>c__DisplayClass2_0.<InvokeMethod>b__0(Int32 n)
   at System.Linq.Enumerable.SelectRangeIterator`1.ToArray()
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Amazon.JSII.Runtime.CallbackExtensions.InvokeMethod(InvokeRequest request, IReferenceMap referenceMap)
   at Amazon.JSII.Runtime.CallbackExtensions.InvokeCallbackCore(Callback callback, IReferenceMap referenceMap)
   at Amazon.JSII.Runtime.CallbackExtensions.InvokeCallback(Callback callback, IReferenceMap referenceMap, IFrameworkToJsiiConverter converter, String& error).
   at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
   at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]()
   at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)
   at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
   at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]()
   at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)
   at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
   at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]()
   at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)
   at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
   at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]()
   at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)
   at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
   at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]()
   at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)
   at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
   at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]()
   at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)
   at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
   at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]()
   at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)
   at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
   at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]()
   at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)
   at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
   at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]()
   at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)
   at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
   at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]()
   at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)
   at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
   at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]()
   at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)
   at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
   at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]()
   at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)
   at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
   at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]()
   at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)
   at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
   at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]()
   at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)
   at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
   at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]()
   at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)
   at Amazon.JSII.Runtime.Services.Client.Invoke(InvokeRequest request)
   at Amazon.JSII.Runtime.Services.Client.Invoke(ObjectReference objectReference, String method, Object[] arguments)
   at Amazon.JSII.Runtime.Deputy.DeputyBase.<>c__DisplayClass17_0`1.<InvokeInstanceMethod>b__1(IClient client, Object[] args)
   at Amazon.JSII.Runtime.Deputy.DeputyBase.<InvokeMethodCore>g__GetResult|18_0[T](<>c__DisplayClass18_0`1& )
   at Amazon.JSII.Runtime.Deputy.DeputyBase.InvokeMethodCore[T](JsiiMethodAttribute methodAttribute, Object[] arguments, Func`3 beginFunc, Func`3 invokeFunc)
   at Amazon.JSII.Runtime.Deputy.DeputyBase.InvokeInstanceMethod[T](Type[] parameterTypes, Object[] arguments, String methodName)
   at Amazon.CDK.Stage.Synth(IStageSynthesisOptions options)
   at Cdk.Program.Main(String[] args) in /Users/tiagobar/Documents/GitHub/eventsourcing-qldb/cdk/src/Cdk/Program.cs:line 14
Subprocess exited with error 134

Environment

CLI Version : aws-cli/2.0.34 Python/3.7.4 Darwin/19.6.0 botocore/2.0.0dev38
Framework Version: 1.64.0 (build 9510201)
OS : MacOs
Language (Version): netcoreapp3.1

Other

I see that there are other issues that seem related to this one 7392 and 5317

This is 🐛 Bug Report

The text was updated successfully, but these errors were encountered:

peron · 2020-09-26T15:29:56Z

We have the same problem. We have tried multiple formats, and either get the same error during synth as the OP, or it fails during deploy when CloudFormation complains about the value being in the wrong format.

shivlaks · 2020-09-29T07:43:25Z

@t1agob thanks for reporting, can you share the regular string you've used? I'm not familiar with QLDB yet, but I'm going to start by trying to deploy the sample app. does an InclusiveStartTime with the values in the example succeed?

@peron - was your issue also specific to .NET or were you using another language?

t1agob · 2020-09-29T07:47:40Z

@shivlaks no, the values in the example do not work. On the code I pasted when opening the issue I am getting the date in the specific format mentioned on the example but I have also tried to use a string like the one in the example. Got the same error.

shivlaks · 2020-09-29T07:52:04Z

@t1agob got it, thanks for confirming, I'll start with that example in TypeScript first and then build up. I'll provide an update once I take a look.

t1agob · 2020-09-29T07:53:56Z

@shivlaks In Typescript it works without any issue. I moved all my CDK code from .Net to Typescript due to this issue.

shivlaks · 2020-09-29T07:55:23Z

@t1agob all right that saves me some debug time. so when using the same string value directly, does the failure occur during synthesis for .NET?

t1agob · 2020-09-29T07:57:33Z

@shivlaks both synthesis and deploy. The stacktrace above is from a cdk deploy command.

RomainMuller · 2020-09-29T08:07:58Z

The value appears to be round-trip-ing though a jsii Callback. I would not be surprised if the .NET JSON serializer actually deserializes the ISO-8601 string into a Date, and it's all broken from there on. I think this qualifies as a bug in the .NET runtime library for jsii.

When a ISO-8601-encoded date is passed from JavaScript to the .NET app, the JSON deserializer would interpret that as a `DateTime` instead of passing the string un-touched. This is a problem, since the jsii kernel protocol has a dedicated wrapper key for DateTime values (`$jsii$date`). This PR adds a compliance test around this particular scenario, and disabled `DateTime` handling from the standard deserializer (the wrapped values are still processed correctly). This bug was identified in aws/aws-cdk#10513

When a ISO-8601-encoded date is passed from JavaScript to the .NET app, the JSON deserializer would interpret that as a `DateTime` instead of passing the string un-touched. This is a problem, since the jsii kernel protocol has a dedicated wrapper key for DateTime values (`$jsii$date`). This PR adds a compliance test around this particular scenario, and disabled `DateTime` handling from the standard deserializer (the wrapped values are still processed correctly). > This PR also changes the way the .NET test package generates the > tested libraries (`jsii-calc` & dependencies) so that instead of > generating NuGet packages, it only generates C# projects. This makes > the experience of debugging the tests **much** nicer, and reduces the > likelihood of cached build artifacts getting in the way. This bug was identified in aws/aws-cdk#10513

RomainMuller · 2020-09-29T09:41:00Z

My assumption was confirmed and I am looking to release the fix today 🎉

When a ISO-8601-encoded date is passed from JavaScript to the .NET app, the JSON deserializer would interpret that as a `DateTime` instead of passing the string un-touched. This is a problem, since the jsii kernel protocol has a dedicated wrapper key for DateTime values (`$jsii$date`). This PR adds a compliance test around this particular scenario, and disabled `DateTime` handling from the standard deserializer (the wrapped values are still processed correctly). > This PR also changes the way the .NET test package generates the > tested libraries (`jsii-calc` & dependencies) so that instead of > generating NuGet packages, it only generates C# projects. This makes > the experience of debugging the tests **much** nicer, and reduces the > likelihood of cached build artifacts getting in the way. Fixes aws/aws-cdk#10513

peron · 2020-10-01T06:50:17Z

I see you got around this quickly! So my answer might not be that important anymore, but yes, we also use .NET.

Since the problem occurred during synthesis, we ended up doing this in the stack file:

InclusiveStartTime = "<replace-me>"

and ended Program.Main with

app.Synth();

var fileName = $"./cdk.out/{stackName}.template.json";
var fileContents = File.ReadAllText(fileName);
fileContents = fileContents.Replace("<replace-me>", "2012-04-23T18:25:00Z");
File.WriteAllText(fileName, fileContents);

and that worked for us.

t1agob added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Sep 24, 2020

github-actions bot assigned shivlaks Sep 24, 2020

github-actions bot added the @aws-cdk/aws-qldb Related to Amazon Quantum Ledger Database (Amazon QLDB) label Sep 24, 2020

shivlaks added the p1 label Sep 29, 2020

shivlaks added effort/small Small work item – less than a day of effort and removed needs-triage This issue or PR still needs to be triaged. labels Sep 29, 2020

RomainMuller added jsii This issue originates in jsii, or this feature must be implemented in jsii. language/dotnet Related to .NET bindings labels Sep 29, 2020

RomainMuller mentioned this issue Sep 29, 2020

fix(dontet): iso-8601 date strings get turned into DateTime aws/jsii#2058

Merged

RomainMuller closed this as completed in aws/jsii#2058 Sep 29, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[QLDB] Unable to create QLDB Stream in .NET #10513

[QLDB] Unable to create QLDB Stream in .NET #10513

t1agob commented Sep 24, 2020

peron commented Sep 26, 2020

shivlaks commented Sep 29, 2020

t1agob commented Sep 29, 2020

shivlaks commented Sep 29, 2020

t1agob commented Sep 29, 2020

shivlaks commented Sep 29, 2020

t1agob commented Sep 29, 2020

RomainMuller commented Sep 29, 2020

RomainMuller commented Sep 29, 2020

peron commented Oct 1, 2020

[QLDB] Unable to create QLDB Stream in .NET #10513

[QLDB] Unable to create QLDB Stream in .NET #10513

Comments

t1agob commented Sep 24, 2020

Reproduction Steps

What did you expect to happen?

What actually happened?

Environment

Other

peron commented Sep 26, 2020

shivlaks commented Sep 29, 2020

t1agob commented Sep 29, 2020

shivlaks commented Sep 29, 2020

t1agob commented Sep 29, 2020

shivlaks commented Sep 29, 2020

t1agob commented Sep 29, 2020

RomainMuller commented Sep 29, 2020

RomainMuller commented Sep 29, 2020

peron commented Oct 1, 2020