Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add serde support to date time #2646

Merged
merged 35 commits into from
Jun 5, 2023

Conversation

thomas-k-cameron
Copy link
Contributor

@thomas-k-cameron thomas-k-cameron commented Apr 27, 2023

Motivation and Context

This is a child PR of #2616

The changes that this PR introduces is same as the ones that were merged to unstable-serde-support branch before.

Initially, we tried to make commit to unstable-serde-support branch and merge changes one by one in small PRs. However, in order to make it up to date with the main branch, we would need to go through a large PR of over 700 files.

Thus, I decided to create individual PRs that commits directly to main branch.

Description

  • Implements serde support to DateTime

Testing

  • Test checks whether the serialized/de-serialized data matches with the expected value

Checklist

NA


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

This was referenced Apr 27, 2023
@thomas-k-cameron thomas-k-cameron marked this pull request as ready for review April 27, 2023 02:32
@thomas-k-cameron thomas-k-cameron requested review from a team as code owners April 27, 2023 02:32
Copy link
Collaborator

@jdisanti jdisanti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Just some minor things.

Could you also add a cargo invocation to this script that runs tests with aws_sdk_unstable and the two serde features enabled? https://github.com/awslabs/smithy-rs/blob/cbbbec4a4892e301ebde3af5cc852aa48d82e608/rust-runtime/aws-smithy-types/additional-ci#L7

rust-runtime/aws-smithy-types/src/date_time/mod.rs Outdated Show resolved Hide resolved
rust-runtime/aws-smithy-types/src/date_time/mod.rs Outdated Show resolved Hide resolved
rust-runtime/aws-smithy-types/src/date_time/de.rs Outdated Show resolved Hide resolved
@thomas-k-cameron
Copy link
Contributor Author

I don't know where the error is coming from.

    Checking serde v1.0.0
error[E0432]: unresolved import `core::nonzero`
   --> /opt/cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/serde-1.0.0/src/lib.rs:195:19
    |
195 |     pub use core::nonzero::{NonZero, Zeroable};
    |                   ^^^^^^^ could not find `nonzero` in `core`

error[E0432]: unresolved import `core::num::Zero`
   --> /opt/cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/serde-1.0.0/src/lib.rs:198:13
    |
198 |     pub use core::num::Zero;
    |             ^^^^^^^^^^^^^^^ no `Zero` in `num`
    |
help: consider importing one of these items instead
    |
198 |     pub use core::num::FpCategory::Zero;
    |             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
198 |     pub use core::num::IntErrorKind::Zero;
    |             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
198 |     pub use core::num::flt2dec::FullDecoded::Zero;
    |             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
198 |     pub use core::num::fmt::Part::Zero;
    |             ~~~~~~~~~~~~~~~~~~~~~~~~~~~

error[E0635]: unknown feature `zero_one`
  --> /opt/cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/serde-1.0.0/src/lib.rs:91:68
   |
91 | #![cfg_attr(feature = "unstable", feature(nonzero, specialization, zero_one))]
   |                                                                    ^^^^^^^^

Some errors have detailed explanations: E0432, E0635.
For more information about an error, try `rustc --explain E0432`.
error: could not compile `serde` due to 3 previous errors
warning: build failed, waiting for other jobs to finish...
error: process didn't exit successfully: `/opt/rustup/toolchains/nightly-2022-11-16-x86_64-unknown-linux-gnu/bin/cargo check --all-features --manifest-path aws-inlineable/Cargo.toml` (exit status: 101)
error: process didn't exit successfully: `/opt/rustup/toolchains/nightly-2022-11-16-x86_64-unknown-linux-gnu/bin/cargo hack check --all-features` (exit status: 1) 

The nightly compiler that you are using seems to be pretty old, maybe an update?

@thomas-k-cameron
Copy link
Contributor Author

OK, it's working now.

rust-runtime/aws-smithy-runtime/Cargo.toml Outdated Show resolved Hide resolved
E: serde::de::Error,
{
match self.state {
VisitorState::Unexpected => fail(VisitorState::UNEXPECTED_VISITOR_STATE),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this compiles. Could you also add a cargo invocation to this script that runs tests with aws_sdk_unstable and the two serde features enabled? https://github.com/awslabs/smithy-rs/blob/cbbbec4a4892e301ebde3af5cc852aa48d82e608/rust-runtime/aws-smithy-types/additional-ci#L7

That should make CI test it.

Copy link
Contributor Author

@thomas-k-cameron thomas-k-cameron May 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for letting me know!

I thought it is already covered by the existing tests.
I completely missed it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I figured out why it wasn't showing up any errors.

This branch doesn't have [features] key on Cargo.toml.
It seems like cargo just ignored anything that was feature-gated behind serde-serialize and serde-deserialize

@jdisanti
Copy link
Collaborator

jdisanti commented Jun 1, 2023

What was the problem? Does it not call the different visit methods? We could just make them both i64 if that's the case, and it would still be 5 bytes less than the string version.

@thomas-k-cameron
Copy link
Contributor Author

You are right!

It's working now. I don't know what I was thinking.

@thomas-k-cameron
Copy link
Contributor Author

thomas-k-cameron commented Jun 2, 2023

No idea where linker error is coming from.

+ popd
+ for example in '*'
+ echo -e '\033[1;33mChecking examples/test-utils...\033[0m'
+ pushd test-utils
Checking examples/test-utils...
+ cargo check
    Updating crates.io index
   Compiling autocfg v1.1.0
   Compiling proc-macro2 v1.0.59
   Compiling unicode-ident v1.0.9
   Compiling quote v1.0.28
   Compiling libc v0.2.144
   Compiling syn v2.0.18
   Compiling version_check v0.9.4
    Checking cfg-if v1.0.0
    Checking bytes v1.4.0
   Compiling lock_api v0.4.9
   Compiling parking_lot_core v0.9.7
    Checking pin-project-lite v0.2.9
    Checking once_cell v1.17.2
    Checking scopeguard v1.1.0
   Compiling futures-core v0.3.28
    Checking smallvec v1.10.0
    Checking itoa v1.0.6
   Compiling log v0.4.18
   Compiling tokio v1.28.2
    Checking parking_lot v0.12.1
    Checking mio v0.8.8
    Checking socket2 v0.4.9
    Checking signal-hook-registry v1.4.1
    Checking num_cpus v1.15.0
   Compiling futures-task v0.3.28
   Compiling futures-util v0.3.28
    Checking tracing-core v0.1.31
   Compiling num-traits v0.2.15
    Checking ryu v1.0.13
    Checking pin-utils v0.1.0
   Compiling tokio-macros v2.1.0
   Compiling tracing-attributes v0.1.24
error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-Wl,--version-script=/tmp/rustcabi7DK/list" "-m64" "/tmp/rustcabi7DK/symbols.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.0.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.1.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.10.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.11.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.12.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.13.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.14.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.15.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.2.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.3.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.4.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.5.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.6.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.7.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.8.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.tracing_attributes.62e245ff-cgu.9.rcgu.o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.4n73859jkoxdxj5m.rcgu.rmeta" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/tracing_attributes-9b799731f627d96b.53km001gtyljpks3.rcgu.o" "-Wl,--as-needed" "-L" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps" "-L" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/libsyn-5ff6d53c13b63fe1.rlib" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/libquote-522855fad33e69c5.rlib" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/libproc_macro2-d0cb96a081d001ab.rlib" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/libunicode_ident-6f8d5f8ff8a5490f.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libproc_macro-df9180a12451ef7e.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-c6192dd4c4d410ac.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-ff7377081f1949cc.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-1cb453c5b22d87c0.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-ef80a3f609e31ec2.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-a690952b7d7394ed.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-33a38bd462dea42b.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-2804c2e5e3dadb10.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-6cea4a991902c072.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-3f9cac5a3a01ae1f.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-be29f577ccd2e0c7.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-40cbcc82de628eb3.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-ce9e735ebca3a756.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-52730b1d9cce8f6d.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-dbd9b472d3582c61.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-8f1d87098c577a9e.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-72ca4bc0b0597905.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-cdff4f89b0741001.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-92115cdd0b50e8be.rlib" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-7f2d6606c0ceff89.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" "/opt/rustup/toolchains/1.67.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/build/workspace/aws-sdk/examples/test-utils/target/debug/deps/libtracing_attributes-9b799731f627d96b.so" "-Wl,--gc-sections" "-shared" "-Wl,-zrelro,-znow" "-nodefaultlibs"
  = note: collect2: error: ld returned 1 exit status

@thomas-k-cameron
Copy link
Contributor Author

thomas-k-cameron commented Jun 4, 2023

I tried to reproduce this but it hasn't been working.

Maybe something to do with the docker image or the instance?

@Velfi
Copy link
Contributor

Velfi commented Jun 5, 2023

No idea where linker error is coming from.

I've seen this happen as part of the minimal versions check we do. Tracing at v1.0 has some issue with an attribute macro, so when the minimal versions check downgrades tracing, this error can show up. I think we should start using a version of tracing that's newer.

@thomas-k-cameron
Copy link
Contributor Author

The latest version of tracing is v0.1.38.

What do you mean by tracing v1.0?

https://docs.rs/tracing/latest/tracing/index.html

@jdisanti
Copy link
Collaborator

jdisanti commented Jun 5, 2023

You don't need to worry about CI on the examples. That's just a notification to us so that we can fix the examples before doing a release. It doesn't block merge. The important CI job that must pass before merge is ci / Matrix Success, which did pass here.

@thomas-k-cameron
Copy link
Contributor Author

You don't need to worry about CI on the examples. That's just a notification to us so that we can fix the examples before doing a release. It doesn't block merge. The important CI job that must pass before merge is ci / Matrix Success, which did pass here.

Cool. Thanks!

Copy link
Collaborator

@jdisanti jdisanti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great. Just some minor comments.

impl<'de> Visitor<'de> for NonHumanReadableDateTimeVisitor {
type Value = DateTime;
fn expecting(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter.write_str("Datat must be number")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make this error message clearer?

}

/// check for human redable format
#[test]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should be in a #[cfg(test)] mod tests {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

}

/// check for human redable format
#[test]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same with these

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

}
}

#[test]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#[test]
#[cfg(test)]

rust-runtime/aws-smithy-types/src/date_time/de.rs Outdated Show resolved Hide resolved
@jdisanti
Copy link
Collaborator

jdisanti commented Jun 5, 2023

I recommend installing the pre-commit commit git hook to the git repo by running pre-commit install so that it automatically fixes style issues when committing 😄

@thomas-k-cameron
Copy link
Contributor Author

I recommend installing the pre-commit commit git hook to the git repo by running pre-commit install so that it automatically fixes style issues when committing 😄

Actually, I have pre-commit but it takes for ever to finish (like 10 to 20 minutes). So I usually just push it and see if it works.

I will make sure to pre-commit before I push it from now on 😄

@jdisanti
Copy link
Collaborator

jdisanti commented Jun 5, 2023

but it takes for ever to finish (like 10 to 20 minutes).

Huh. That's odd. It takes about 10 seconds on my machine if the tools are already compiled, and maybe 2 minutes otherwise.

@thomas-k-cameron
Copy link
Contributor Author

thomas-k-cameron commented Jun 5, 2023

but it takes for ever to finish (like 10 to 20 minutes).

Huh. That's odd. It takes about 10 seconds on my machine if the tools are already compiled, and maybe 2 minutes otherwise.

Maybe because I'm using VSC on windows inside a docker.

It was lot faster on github codespace so.

@jdisanti jdisanti added this pull request to the merge queue Jun 5, 2023
Merged via the queue into smithy-lang:main with commit 74a7204 Jun 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants