Move tokenstorage (#1065)

* Move tokenstorage v2 out of experimental

* Move TokenStorage doc -> Storage Adapters (Legacy)

* Document tokenstorage interface

* Docstring pedantry

changelog.d/20240920_144134_derek_move_tokenstorage.rst

@@ -0,0 +1,6 @@
+
+Changed
+~~~~~~~
+
+- TokenStorages have been moved from ``globus_sdk.experimental.tokenstorage``
+  to ``globus_sdk.tokenstorage``. (:pr:`NUMBER`)

docs/authorization/index.rst

@@ -9,4 +9,5 @@ Components of the Globus SDK which handle application authorization.
     globus_authorizers
     scopes_and_consents/index
     login_flows
+    token_caching/index
     gare

docs/authorization/token_caching/index.rst

@@ -0,0 +1,20 @@
+
+Token Caching
+=============
+
+The documentation in this section provides references for interfaces and standard
+implementations for caching OAuth2 tokens. While there are two distinct class
+hierarchies, :ref:`token_storages` and its predecessor :ref:`storage_adapters`, we
+recommend using the former. ``TokenStorage`` is a newer iteration of the token storage
+interface and includes a superset of the functionality previously supported in
+``StorageAdapter``.
+
+All constructs from both hierarchies are importable from the ``globus_sdk.tokenstorage``
+namespace.
+
+.. toctree::
+    :maxdepth: 1
+
+    token_storages
+    storage_adapters
+

docs/authorization/token_caching/storage_adapters.rst

@@ -0,0 +1,133 @@
+.. _storage_adapters:
+
+Storage Adapters (Legacy)
+=========================
+
+.. warning::
+
+    The class hierarchy documented here is legacy.
+
+    We recommend using the newer class hierarchy documented at :ref:`token_storages`.
+
+The StorageAdapter component provides a way of storing and loading the tokens
+received from authentication and token refreshes.
+
+Usage
+-----
+
+StorageAdapter is available under the name ``globus_sdk.tokenstorage``.
+
+Storage adapters are the main objects of this subpackage. Primarily, usage
+should revolve around creating a storage adapter, potentially loading data from
+it, and using it as the ``on_refresh`` handler for an authorizer.
+
+For example:
+
+.. code-block:: python
+
+    import os
+    import globus_sdk
+    from globus_sdk.tokenstorage import SimpleJSONFileAdapter
+
+    my_file_adapter = SimpleJSONFileAdapter(os.path.expanduser("~/mytokens.json"))
+
+    if not my_file_adapter.file_exists():
+        # ... do a login flow, getting back initial tokens
+        # elided for simplicity here
+        token_response = ...
+        # now store the tokens, and pull out the tokens for the
+        # resource server we want
+        my_file_adapter.store(token_response)
+        by_rs = token_response.by_resource_server
+        tokens = by_rs["transfer.api.globus.org"]
+    else:
+        # otherwise, we already did this whole song-and-dance, so just
+        # load the tokens from that file
+        tokens = my_file_adapter.get_token_data("transfer.api.globus.org")
+
+
+    # RereshTokenAuthorizer and ClientCredentialsAuthorizer both use
+    # `on_refresh` callbacks
+    # this feature is therefore only relevant for those auth types
+    #
+    # auth_client is the internal auth client used for refreshes,
+    # and which was used in the login flow
+    # note that this is all normal authorizer usage wherein
+    # my_file_adapter is providing the on_refresh callback
+    auth_client = ...
+    authorizer = globus_sdk.RefreshTokenAuthorizer(
+        tokens["refresh_token"],
+        auth_client,
+        access_token=tokens["access_token"],
+        expires_at=tokens["expires_at_seconds"],
+        on_refresh=my_file_adapter.on_refresh,
+    )
+
+    # or, for client credentials
+    authorizer = globus_sdk.ClientCredentialsAuthorizer(
+        auth_client,
+        ["urn:globus:auth:transfer.api.globus.org:all"],
+        access_token=tokens["access_token"],
+        expires_at=tokens["expires_at_seconds"],
+        on_refresh=my_file_adapter.on_refresh,
+    )
+
+    # and then use the authorizer on a client!
+    tc = globus_sdk.TransferClient(authorizer=authorizer)
+
+
+Complete Example Usage
+~~~~~~~~~~~~~~~~~~~~~~
+
+The :ref:`Group Listing With Token Storage Script <example_group_listing_with_token_storage>`
+provides a complete and runnable example which leverages ``tokenstorage``.
+
+
+Adapter Types
+-------------
+
+.. module:: globus_sdk.tokenstorage
+
+``globus_sdk.tokenstorage`` provides base classes for building your own storage
+adapters, and several complete adapters.
+
+The :class:`SimpleJSONFileAdapter` is good for the "simplest possible"
+persistent storage, using a JSON file to store token data.
+
+:class:`MemoryAdapter` is even simpler still, and is great for writing and
+testing code which uses the `StorageAdapter` interface backed by an in-memory
+structure.
+
+The :class:`SQLiteAdapter` is more complex, for applications like the
+globus-cli which need to store various tokens and additional configuration. In
+addition to basic token storage, the :class:`SQLiteAdapter` provides for namespacing
+of the token data, and for additional configuration storage.
+
+Reference
+---------
+
+.. autoclass:: StorageAdapter
+   :members:
+   :member-order: bysource
+   :show-inheritance:
+
+.. autoclass:: MemoryAdapter
+   :members:
+   :member-order: bysource
+   :show-inheritance:
+
+.. autoclass:: FileAdapter
+   :members:
+   :member-order: bysource
+   :show-inheritance:
+
+.. autoclass:: SimpleJSONFileAdapter
+   :members:
+   :member-order: bysource
+   :show-inheritance:
+
+.. autoclass:: SQLiteAdapter
+   :members:
+   :member-order: bysource
+   :show-inheritance:
+

docs/authorization/token_caching/token_storages.rst

@@ -0,0 +1,46 @@
+.. _token_storages:
+
+.. currentmodule:: globus_sdk.tokenstorage
+
+Token Storages
+==============
+
+Interacting with Globus services requires the use of Globus Auth-issued OAuth2 tokens.
+To assist in reuse of these tokens, the SDK provides an interface to store and
+retrieve this data across different storage backends.
+
+In addition to the interface, :class:`TokenStorage`, the SDK provides concrete
+implementations for some of the most common storage backends:
+
+-   :class:`JSONTokenStorage` for storing tokens in a local JSON file.
+-   :class:`SQLiteTokenStorage` for storing tokens in a local SQLite database.
+-   :class:`MemoryTokenStorage` for storing tokens in process memory.
+
+
+Reference
+---------
+
+.. autoclass:: TokenStorage
+    :members:
+    :member-order: bysource
+
+
+.. autoclass:: TokenStorageData
+    :members:
+    :member-order: bysource
+
+
+File-based Token Storages
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. autoclass:: JSONTokenStorage
+
+.. autoclass:: SQLiteTokenStorage
+    :members: close, iter_namespaces
+
+
+Ephemeral Token Storages
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. autoclass:: MemoryTokenStorage
+

docs/index.rst

@@ -34,7 +34,6 @@ Table of Contents
     services/index
     local_endpoints
     Authorization <authorization/index>
-    tokenstorage
     config
     core/index
 

docs/tokenstorage.rst

@@ -1,126 +1,3 @@
-.. _tokenstorage:
+:orphan:
 
-TokenStorage
-============
-
-The TokenStorage component provides a way of storing and loading the tokens
-received from authentication and token refreshes.
-
-Usage
------
-
-TokenStorage is available under the name ``globus_sdk.tokenstorage``.
-
-Storage adapters are the main objects of this subpackage. Primarily, usage
-should revolve around creating a storage adapter, potentially loading data from
-it, and using it as the ``on_refresh`` handler for an authorizer.
-
-For example:
-
-.. code-block:: python
-
-    import os
-    import globus_sdk
-    from globus_sdk.tokenstorage import SimpleJSONFileAdapter
-
-    my_file_adapter = SimpleJSONFileAdapter(os.path.expanduser("~/mytokens.json"))
-
-    if not my_file_adapter.file_exists():
-        # ... do a login flow, getting back initial tokens
-        # elided for simplicity here
-        token_response = ...
-        # now store the tokens, and pull out the tokens for the
-        # resource server we want
-        my_file_adapter.store(token_response)
-        by_rs = token_response.by_resource_server
-        tokens = by_rs["transfer.api.globus.org"]
-    else:
-        # otherwise, we already did this whole song-and-dance, so just
-        # load the tokens from that file
-        tokens = my_file_adapter.get_token_data("transfer.api.globus.org")
-
-
-    # RereshTokenAuthorizer and ClientCredentialsAuthorizer both use
-    # `on_refresh` callbacks
-    # this feature is therefore only relevant for those auth types
-    #
-    # auth_client is the internal auth client used for refreshes,
-    # and which was used in the login flow
-    # note that this is all normal authorizer usage wherein
-    # my_file_adapter is providing the on_refresh callback
-    auth_client = ...
-    authorizer = globus_sdk.RefreshTokenAuthorizer(
-        tokens["refresh_token"],
-        auth_client,
-        access_token=tokens["access_token"],
-        expires_at=tokens["expires_at_seconds"],
-        on_refresh=my_file_adapter.on_refresh,
-    )
-
-    # or, for client credentials
-    authorizer = globus_sdk.ClientCredentialsAuthorizer(
-        auth_client,
-        ["urn:globus:auth:transfer.api.globus.org:all"],
-        access_token=tokens["access_token"],
-        expires_at=tokens["expires_at_seconds"],
-        on_refresh=my_file_adapter.on_refresh,
-    )
-
-    # and then use the authorizer on a client!
-    tc = globus_sdk.TransferClient(authorizer=authorizer)
-
-
-Complete Example Usage
-~~~~~~~~~~~~~~~~~~~~~~
-
-The :ref:`Group Listing With Token Storage Script <example_group_listing_with_token_storage>`
-provides complete and runnable example which leverages ``tokenstorage``.
-
-
-Adapter Types
--------------
-
-.. module:: globus_sdk.tokenstorage
-
-``globus_sdk.tokenstorage`` provides base classes for building your own storage
-adapters, and several complete adapters.
-
-The :class:`SimpleJSONFileAdapter` is good for the "simplest possible"
-persistent storage, using a JSON file to store token data.
-
-:class:`MemoryAdapter` is even simpler still, and is great for writing and
-testing code which uses the `StorageAdapter` interface backed by an in-memory
-structure.
-
-The :class:`SQLiteAdapter` is more complex, for applications like the
-globus-cli which need to store various tokens and additional configuration. In
-addition to basic token storage, the :class:`SQLiteAdapter` provides for namespacing
-of the token data, and for additional configuration storage.
-
-Reference
----------
-
-.. autoclass:: StorageAdapter
-   :members:
-   :member-order: bysource
-   :show-inheritance:
-
-.. autoclass:: MemoryAdapter
-   :members:
-   :member-order: bysource
-   :show-inheritance:
-
-.. autoclass:: FileAdapter
-   :members:
-   :member-order: bysource
-   :show-inheritance:
-
-.. autoclass:: SimpleJSONFileAdapter
-   :members:
-   :member-order: bysource
-   :show-inheritance:
-
-.. autoclass:: SQLiteAdapter
-   :members:
-   :member-order: bysource
-   :show-inheritance:
+The documentation which was found on this page has moved to :ref:`storage_adapters`.

docs/tutorial.rst

@@ -123,7 +123,7 @@ are useful for further reading:
 * :ref:`using GlobusAuthorizer objects <globus_authorizers>` handle passing tokens to Globus,
   and may handle token expiration
 
-* :ref:`TokenStorage <tokenstorage>` objects handle storage of tokens
+* :ref:`storage_adapters` objects handle storage of tokens
 
 These are covered by several of the available :ref:`Examples <examples>` as
 well.

src/globus_sdk/experimental/globus_app/_types.py

@@ -5,8 +5,8 @@
 
 from globus_sdk import AuthLoginClient
 from globus_sdk._types import UUIDLike
-from globus_sdk.experimental.tokenstorage import TokenStorage
 from globus_sdk.login_flows import LoginFlowManager
+from globus_sdk.tokenstorage import TokenStorage
 
 if sys.version_info < (3, 8):
     from typing_extensions import Protocol, runtime_checkable

src/globus_sdk/experimental/globus_app/_validating_token_storage.py

@@ -3,8 +3,8 @@
 import typing as t
 
 from globus_sdk import AuthClient, OAuthRefreshTokenResponse, OAuthTokenResponse, Scope
-from globus_sdk.experimental.tokenstorage import TokenStorage, TokenStorageData
 from globus_sdk.scopes.consents import ConsentForest
+from globus_sdk.tokenstorage import TokenStorage, TokenStorageData
 
 from .errors import (
     IdentityMismatchError,

src/globus_sdk/experimental/globus_app/app.py

@@ -8,9 +8,9 @@
 from globus_sdk import AuthClient, AuthLoginClient, GlobusSDKUsageError, Scope
 from globus_sdk._types import ScopeCollectionType, UUIDLike
 from globus_sdk.authorizers import GlobusAuthorizer
-from globus_sdk.experimental.tokenstorage import TokenStorage
 from globus_sdk.gare import GlobusAuthorizationParameters
 from globus_sdk.scopes import AuthScopes, scopes_to_scope_list
+from globus_sdk.tokenstorage import TokenStorage
 
 from ._types import TokenStorageProvider
 from ._validating_token_storage import ValidatingTokenStorage