refactor: removed duplicate and some useless optional dependencies

[frol]

• tempdir is deprecated since merge into tempfile (I would not touch it would not it introduce the third version of rand crate into nearcore... we are stuck using two versions due to the epoch manager being rand-sensitive)
• I integrated cargo-deny (see deny.toml config for the details about the whitelisted duplicates) to check for duplicate dependencies on CI (updated Buildkite pipeline); we may also use it to check licenses and advisories
• I downgraded ansi_term to 0.11 since it is used in clap, and there are no major fixes in 0.12, so it makes sense to avoid duplication
• I did not update base64 crate to 0.12 as it would cause duplication since actix-http uses 0.11 version
• I did not update rocksdb to 0.14 since it requires some migration changes and I don't want to deal with it in this PR

[gitpod-io]

[frol]

@ailisp I had added the following command to the "sanity checks" step on CI:

      if [ -e deny.toml ]; then
        wget "https://github.com/EmbarkStudios/cargo-deny/releases/download/$${CARGO_DENY_VERSION}/$${CARGO_DENY_PKG}.tar.gz"
        tar xzf "$${CARGO_DENY_PKG}.tar.gz"
        "$${CARGO_DENY_PKG}/cargo-deny" check bans
      fi

NOTE: I had to escape $ using $$ to force Buildkite to resolve environment variables in runtime.

UPD: Can we include cargo-deny installation step to the base image?

UPD2: Can we run pytest suite against PR?

[bowenwang1996]

Can we run pytest suite against PR?

Do you want to run all of them?

[ailisp]

UPD: Can we include cargo-deny installation step to the base image?

Yes but not very urgent as this step is usually faster than the other steps so speed up this step but does not increase the user wait time

UPD2: Can we run pytest suite against PR?

Plan to do that soon. currently how nightly run is not very ideal. My plan:

• parallelize pytest, as like parallel_run_test.py
• finish setup https://github.com/icerove/buildkite-autoscaler and https://github.com/ailisp/fastautoscaler, since tests are getting wait for agent so long as parallel test steps increased, some 30 min+.
• will also add cargo-deny in image at time of setup new autoscaler agent
• run 3 nightly suite against ci, if it's reasonably fast, or pytest only.

[ailisp]

Nice cleanup! I was wondering how would different version of a crate coexist and the behave in the single neard binary

[frol]

@ailisp Different versions of the same crate just have different symbol names like rand__0_6_5__new (it is more packed than that, but they don't collide). The problem is when you try to pass a structure actix__0_8__Message to a function that expects actix__0_9__Message, and those messages in Rust were a little bit cryptic (they got improved AFAIK).

Some of the crates still are duplicated, but they are now explicitly whitelisted and documented in deny.toml and checked on CI, so we won't introduce new duplicates without noticing that.

[frol]

Do you want to run all of them?

Well, this is a massive dependencies update, so I would rather catch problems in PR, than on master.

Also, "nearlib test" was flaky on CI, so I just want to test it thoroughly, if possible.

@ailisp Do we have a facility to trigger pytest manually? (I mean, I can start it running on my laptop, but I guess, that might take ages to finish.

[MaksymZavershynskyi]

This is great! Thank you!