Test proxy & storage tests improvements

[Jinming-Hu]

In this PR, we:

1. Removed unused definitions in sdk/core/azure-core-test/inc/azure/core/test/network_models.hpp.
2. Removed unused functions TestBase::ValidateSkippingTest() and TestBase::IsValidTime().
3. Added an overload of function TestBase::InitClientOptions().
4. Removed unused functions TestProxyManager::SetStopRecordMode(), TestProxyManager::SetStopPlaybackMode().
5. Rewrote proxy matcher setting logic.
   1. Use json object rather than hard-coded json string, for better readability and maintenance.
   2. Exclude Expect and Connection headers so that recording made by libcurl can be used by WinHTTP and also the other way around
   3. Removed some unnecessary ignored headers
   4. Added some ignored query parameters
6. Fixed an issue in libcurl transport layer. Closes Bug: Host header is not correctly set for non-default port #4213
7. Fixed an issue where libcurl transport layer adds Content-Length header for HEAD/GET/DELETE requests, which caused a behavior misalignment between libcurl and WinHTTP, making the requests recorded by one of the transport layer unusable by the other.
   1. Also removed Content-Length header for GET requests from existing recordings.
8. Fixed a bug where test proxy sometimes discarded request body.
9. Fixed a bug where requests were not properly redirected if Host was set in the original request *
10. Enabled some storage test cases that were disabled in Test proxy #4118. Closes LIVEONLY tests in Storage  #4147
11. Removed CHECK_SKIP_TEST macro. We check if test cases is live-only in SetUpTestBase() function. With this change, the only thing we need to do to mark a test case live-only is to add _LIVEONLY_ suffix in case name. There's no need to call CHECK_SKIP_TEST macro anymore.

* A little more explanation of this bug:
Transport layer accepts a request with a URL and request headers in it. If Host header is not in request headers, libcurl transport will extract host from URL and set Host header. If Host is already in request, libcurl transport does nothing.
Test-proxy policy modifies host in URL to localhost:5001, but it doesn't modify Host header (I think it's a negligence).
For most of the requests we don't set Host header. Test-proxy modifies host in the URL to localhost:5001, libcurl extracts localhost:5001 from URL and set Host header. Everything works perfectly.
For very few requests, Host header is already set. Test-proxy only modifies host in the URL, libcurl doesn't set Host header because there's one already. Now the host in Host header and URL don't match. These requests cannot be recorded & played back.

Pull Request Checklist

Please leverage this checklist as a reminder to address commonly occurring feedback when submitting a pull request to make sure your PR can be reviewed quickly:

See the detailed list in the contributing guide.

• C++ Guidelines
• Doxygen docs
• Unit tests
• No unwanted commits/changes
• Descriptive title/description
  • PR is single purpose
  • Related issue listed
• Comments in source
• No typos
• Update changelog
• Not work-in-progress
• External references or docs updated
• Self review of PR done
• Any breaking changes?

[antkmsft]

You probably want George's review, but as for me, changes look good.
You are touching a lot of tests and as a result, code coverage may improve. I suggest to grab code coverage numbers from a successful run closer to the end of code review, and update the numbers in ci.ymls.

[Jinming-Hu]

Thanks @antkmsft !

@gearama Can you please help review this PR? and also take a look at the sanitizer issue #4147?

[gearama]

/azp run cpp - keyvault - ci

[azure-pipelines]

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

[gearama]

/azp run cpp - attestation - ci

[azure-pipelines]

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

[gearama]

need to make sure all the other piplines run , make a change in the read me in the folder of the services to kick in the test runs

[Jinming-Hu]

need to make sure all the other piplines run , make a change in the read me in the folder of the services to kick in the test runs

@gearama What change do I need to make to enable the tests?

[antkmsft]

@Jinming-Hu, I think @gearama meant that you can touch some files in each of the library, so the CI would trigger a run of cpp - (library) - ci. The other way to do this is probably by manually kicking off these pipelines.

[antkmsft]

@Jinming-Hu, you can trigger CI pipelines manually, as I just did for identity pipeline, but the runs won't be visible here.

Maybe it is better to just add a space into each library's inc/library.hpp header (undoing it after the PR is done).

I wonder if running non-CI pipelines would give us the same level of verification (although taking longer). These ones you can run via /azp run cpp - (library).

Next week I'll bring this up, I think we should allow the /azp command to trigger runs for CI pipelines, even if none of the files from that library was modified in the PR. Because 1. a responsible human triggers the command knowing best what he wants; 2. This PR demonstrates it could be useful/needed sometimes.

[Jinming-Hu]

@antkmsft Really? There's no way to trigger the pipeline without actually making changes? Is that how pipeline works? It's kind of inconvenient.

So we'll need to add an extra empty line or something in the README.md to trigger the pipeline. If all the tests pass, we revert the changes. It that correct?

[LarryOsterman]

It looks good to me, don't forget to handle #4372 at some point :).