User agent 2.1: Track client features

[hssyoo]

Description

This PR implements scaffolding for tracking client feature usage in the user agent header. Features that we track are defined in botocore.useragent._USERAGENT_FEATURE_MAPPINGS. Whenever a trackable feature is used, its associated metric id is stored. When the user agent string is built, it retrieves all registered features and creates a comma-separated string with the m/ prefix. For example, if a request is made using SigV4A signing, the user agent string may look like the following:

Boto3/1.36.13 md/Botocore#1.36.13 md/awscrt#0.23.8 ua/2.0 os/macos#24.2.0 md/arch#arm64 lang/python#3.11.10 md/pyimpl#CPython m/S cfg/retry-mode#legacy Botocore/1.36.13

Note the m/S component.

Design

I prioritized an approach that tries to minimize invasiveness to the current codebase. Specifically, I wanted to avoid having to plumb some features variable to every method that registers a function, mainly so we wouldn't need to update the function signature of potentially all methods. This would also require higher level packages (Boto3, S3Transfer, AWS CLI) to plumb features they register.

Initial features to track

There are 5 features tracked to start:

• WAITER: An operation called using a waiter.
• PAGINATOR: An operation called using a paginator.
• S3_TRANSFER: An operation called using the S3 Transfer Manager.
• ENDPOINT_OVERRIDE: An operation called using a user provided endpoint URL.
• SIGV4A_SIGNING: An operation called using sigv4a signing.

S3Transfer usage will need to be registered in boto3 and will require a small update to s3transfer(to pass context from parent to child threads).

• S3Transfer: User agent 2.1: Propagate Botocore context to child threads s3transfer#335
• Boto3: User agent 2.1: Track S3Transfer usage boto3#4441

Caveats

• Features are tracked in a set. Since sets don't preserve order in which elements were added, if the feature metadata string exceeds 1KB and it's truncated, the truncation strategy won't be true LIFO. I thought this was an acceptable tradeoff since we're nowhere near the limit and we can explore storage solutions or even metric collection strategy in the future if this becomes a real issue.
• Some features are "cacheable". For example, if a Session object is creating a new Client and there are cached credentials, it will skip invoking the method that registers the relevant credentials feature. This PR doesn't prescribe a single way to deal with cacheable features since the way feature values are cached and gotten can vary. For cached credentials, I imagine we would inspect the METHOD attributes to determine the correct feature to register.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 98.05825% with 2 lines in your changes missing coverage. Please review.

Please upload report for BASE (develop@da7d707). Learn more about missing BASE report.
Report is 200 commits behind head on develop.

Files with missing lines	Patch %	Lines
botocore/crt/auth.py	0.00%	2 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             develop    #3389   +/-   ##
==========================================
  Coverage           ?   93.08%           
==========================================
  Files              ?       67           
  Lines              ?    14588           
  Branches           ?        0           
==========================================
  Hits               ?    13579           
  Misses             ?     1009           
  Partials           ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[hssyoo]

This should be called reset_context. Will fix in later revision.