Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Cosmos] Fix failing unit tests #24287

Merged
merged 88 commits into from
May 13, 2022
Merged
Changes from 1 commit
Commits
Show all changes
88 commits
Select commit Hold shift + click to select a range
61ba8d1
initial commit
simorenoh Aug 13, 2021
15dcceb
Client Constructor (#20310)
annatisch Aug 20, 2021
bda95c3
read database
simorenoh Aug 27, 2021
c9648ab
Update simon_testfile.py
simorenoh Aug 27, 2021
80540dc
with coroutine
simorenoh Aug 30, 2021
1285438
Update simon_testfile.py
simorenoh Aug 30, 2021
992b0cd
small changes
simorenoh Aug 31, 2021
47cb688
async with returns no exceptions
simorenoh Aug 31, 2021
f3fa79f
Merge pull request #1 from Azure/simonmoreno/async
simorenoh Aug 31, 2021
0c49739
async read container
simorenoh Sep 1, 2021
47f4af5
async item read
simorenoh Sep 2, 2021
c97c946
cleaning up
simorenoh Sep 3, 2021
fcd95db
create item/ database methods
simorenoh Sep 13, 2021
36c5b90
item delete working
simorenoh Sep 13, 2021
44db2a2
docs replace functionality
simorenoh Sep 16, 2021
ec5b6ed
upsert functionality
simorenoh Sep 17, 2021
d63d052
Merge pull request #2 from simorenoh/item-read
simorenoh Oct 8, 2021
5d74c8f
missing query methods
simorenoh Oct 11, 2021
89fc2f7
CRUD for udf, sproc, triggers
simorenoh Oct 12, 2021
fdaa880
Merge branch 'Azure:main' into async-client
simorenoh Oct 12, 2021
3f9baf2
Merge branch 'Azure:main' into async-client
simorenoh Oct 12, 2021
d6650bc
Merge branch 'Azure:main' into query-functionality
simorenoh Oct 12, 2021
043dfe0
initial query logic + container methods
simorenoh Oct 13, 2021
befdb41
Merge branch 'async-client' into query-functionality
simorenoh Oct 13, 2021
8cffbe2
Merge pull request #3 from simorenoh/query-functionality
simorenoh Oct 13, 2021
72de7c8
missing some execution logic and tests
simorenoh Oct 21, 2021
5b805b8
oops
simorenoh Oct 21, 2021
8d8d0c4
fully working queries
simorenoh Oct 22, 2021
b597ca8
small fix to query_items()
simorenoh Oct 22, 2021
18319df
Update _cosmos_client_connection_async.py
simorenoh Oct 22, 2021
162c44d
Update _cosmos_client_connection.py
simorenoh Oct 22, 2021
ebbac51
documentation update
simorenoh Oct 22, 2021
43f78e6
Merge branch 'Azure:main' into main
simorenoh Oct 22, 2021
470aa5b
updated MIT dates and get_user_client() description
simorenoh Oct 22, 2021
74da690
Update CHANGELOG.md
simorenoh Oct 22, 2021
7104d63
Merge branch 'Azure:main' into main
simorenoh Oct 25, 2021
20718c7
Delete simon_testfile.py
simorenoh Oct 25, 2021
d825eaa
Merge pull request #4 from simorenoh/async-client
simorenoh Oct 25, 2021
e3c27a5
leftover retry utility
simorenoh Oct 25, 2021
3b778ad
Update README.md
simorenoh Oct 25, 2021
c6e352e
docs and removed six package
simorenoh Oct 28, 2021
8971a25
Merge remote-tracking branch 'upstream/main'
simorenoh Oct 28, 2021
52736ac
changes based on comments
simorenoh Nov 4, 2021
ad98039
small change in type hints
simorenoh Nov 4, 2021
f76c595
updated readme
simorenoh Nov 9, 2021
3f02a65
fixes based on conversations
simorenoh Nov 10, 2021
e719869
added missing type comments
simorenoh Nov 11, 2021
d03ee05
Merge branch 'Azure:main' into main
simorenoh Nov 11, 2021
02c52ee
update changelog for ci pipeline
simorenoh Nov 23, 2021
2cb4551
added typehints, moved params into keywords, added decorators, made _…
simorenoh Nov 29, 2021
cf20d35
changes based on sync with central sdk
simorenoh Dec 2, 2021
f456817
remove is_system_key from scripts (only used in execute_sproc)
simorenoh Dec 3, 2021
ea9bd16
Revert "remove is_system_key from scripts (only used in execute_sproc)"
simorenoh Dec 3, 2021
709d2eb
async script proxy using composition
simorenoh Dec 3, 2021
3277dd8
pylint
simorenoh Dec 3, 2021
a57cb4d
capitalized constants
simorenoh Dec 6, 2021
014578b
Apply suggestions from code review
simorenoh Dec 6, 2021
0d79695
closing python code snippet
simorenoh Dec 6, 2021
fdabea1
last doc updates
simorenoh Dec 7, 2021
016d0dd
Update sdk/cosmos/azure-cosmos/CHANGELOG.md
tjprescott Dec 7, 2021
8228aa9
version update
simorenoh Dec 7, 2021
7ae1cd0
Merge branch 'Azure:main' into main
simorenoh Dec 13, 2021
7e8e953
cosmos updates for release
simorenoh Dec 13, 2021
99997e9
Merge branch 'main' of https://github.com/Azure/azure-sdk-for-python …
simorenoh Jan 12, 2022
f446308
Merge branch 'Azure-main'
simorenoh Jan 12, 2022
bc466d8
Merge branch 'Azure-main'
simorenoh Jan 12, 2022
bca2da7
Merge branch 'Azure:main' into main
simorenoh Jan 13, 2022
437895f
fix connection string comma
simorenoh Jan 18, 2022
dd4b4a4
Update CHANGELOG.md
simorenoh Jan 18, 2022
79e0b28
Merge branch 'main' of https://github.com/Azure/azure-sdk-for-python
simorenoh Jan 25, 2022
399f8bc
Merge branch 'Azure:main' into main
simorenoh Jan 25, 2022
a3bd2e8
fixing extra await keyword in sample
simorenoh Feb 25, 2022
dbf1532
Update CHANGELOG.md
simorenoh Feb 25, 2022
c275953
Merge branch 'main' of https://github.com/Azure/azure-sdk-for-python
simorenoh Mar 10, 2022
140b042
Update CHANGELOG.md
simorenoh Mar 10, 2022
426a616
Merge branch 'main' of https://github.com/Azure/azure-sdk-for-python
simorenoh Apr 5, 2022
7c551dc
Merge branch 'Azure:main' into main
simorenoh Apr 6, 2022
0678376
Merge branch 'Azure:main' into main
simorenoh Apr 6, 2022
6830ed6
Merge branch 'Azure:main' into main
simorenoh Apr 28, 2022
b2e9c92
first round of fixes
simorenoh May 3, 2022
6d5f921
Update test_config.py
simorenoh May 3, 2022
6e2e9db
round 2
simorenoh May 4, 2022
9975b19
raising failed result in get_Database_accounts
simorenoh May 5, 2022
6c13409
small changes
simorenoh May 10, 2022
1af63f2
more small fixes
simorenoh May 11, 2022
da17adb
Update test_query.py
simorenoh May 12, 2022
1953c90
Update test_query.py
simorenoh May 12, 2022
94c1bda
Update _global_endpoint_manager.py
simorenoh May 12, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
docs replace functionality
missing upsert and other resources
  • Loading branch information
simorenoh committed Sep 16, 2021

Verified

This commit was signed with the committer’s verified signature.
manusa Marc Nuri
commit 44db2a2f5aa37acba1d1c15b861682454899bf27
Original file line number Diff line number Diff line change
@@ -370,7 +370,7 @@ async def Create(self, body, path, typ, id, initial_headers, options=None, **kwa
return result

async def __Post(self, path, request_params, body, req_headers, **kwargs):
"""Azure Cosmos 'POST' http request.
"""Azure Cosmos 'POST' async http request.

:params str url:
:params str path:
@@ -483,7 +483,7 @@ async def Read(self, path, typ, id, initial_headers, options=None, **kwargs): #
return result

async def __Get(self, path, request_params, req_headers, **kwargs):
"""Azure Cosmos 'GET' async http request.
"""Azure Cosmos 'GET' async async http request.

:params str url:
:params str path:
@@ -507,6 +507,121 @@ async def __Get(self, path, request_params, req_headers, **kwargs):
**kwargs
)

async def ReplaceContainer(self, collection_link, collection, options=None, **kwargs):
"""Replaces a collection and return it.

:param str collection_link:
The link to the collection entity.
:param dict collection:
The collection to be used.
:param dict options:
The request options for the request.

:return:
The new Collection.
:rtype:
dict

"""
if options is None:
options = {}

CosmosClientConnection.__ValidateResource(collection)
path = base.GetPathFromLink(collection_link)
collection_id = base.GetResourceIdOrFullNameFromLink(collection_link)
return await self.Replace(collection, path, "colls", collection_id, None, options, **kwargs)

async def ReplaceItem(self, document_link, new_document, options=None, **kwargs):
"""Replaces a document and returns it.

:param str document_link:
The link to the document.
:param dict new_document:
:param dict options:
The request options for the request.

:return:
The new Document.
:rtype:
dict

"""
CosmosClientConnection.__ValidateResource(new_document)
path = base.GetPathFromLink(document_link)
document_id = base.GetResourceIdOrFullNameFromLink(document_link)

# Python's default arguments are evaluated once when the function is defined,
# not each time the function is called (like it is in say, Ruby). This means
# that if you use a mutable default argument and mutate it, you will and have
# mutated that object for all future calls to the function as well. So, using
# a non-mutable deafult in this case(None) and assigning an empty dict(mutable)
# inside the function so that it remains local For more details on this gotcha,
# please refer http://docs.python-guide.org/en/latest/writing/gotchas/
if options is None:
options = {}

# Extract the document collection link and add the partition key to options
collection_link = base.GetItemContainerLink(document_link)
options = await self._AddPartitionKey(collection_link, new_document, options)

return await self.Replace(new_document, path, "docs", document_id, None, options, **kwargs)

async def Replace(self, resource, path, typ, id, initial_headers, options=None, **kwargs): # pylint: disable=redefined-builtin
"""Replaces a Azure Cosmos resource and returns it.

:param dict resource:
:param str path:
:param str typ:
:param str id:
:param dict initial_headers:
:param dict options:
The request options for the request.

:return:
The new Azure Cosmos resource.
:rtype:
dict

"""
if options is None:
options = {}

initial_headers = initial_headers or self.default_headers
headers = base.GetHeaders(self, initial_headers, "put", path, id, typ, options)
# Replace will use WriteEndpoint since it uses PUT operation
request_params = _request_object.RequestObject(typ, documents._OperationType.Replace)
result, self.last_response_headers = await self.__Put(path, request_params, resource, headers, **kwargs)

# update session for request mutates data on server side
self._UpdateSessionIfRequired(headers, result, self.last_response_headers)
return result

async def __Put(self, path, request_params, body, req_headers, **kwargs):
"""Azure Cosmos 'PUT' async http request.

:params str url:
:params str path:
:params (str, unicode, dict) body:
:params dict req_headers:

:return:
Tuple of (result, headers).
:rtype:
tuple of (dict, dict)

"""
request = self.pipeline_client.put(url=path, headers=req_headers)
return await asynchronous_request.AsynchronousRequest(
client=self,
request_params=request_params,
global_endpoint_manager=self._global_endpoint_manager,
connection_policy=self.connection_policy,
pipeline_client=self.pipeline_client,
request=request,
request_data=body,
**kwargs
)

async def DeleteDatabase(self, database_link, options=None, **kwargs):
"""Deletes a database.

@@ -602,7 +717,7 @@ async def DeleteResource(self, path, typ, id, initial_headers, options=None, **k
return result

async def __Delete(self, path, request_params, req_headers, **kwargs):
"""Azure Cosmos 'DELETE' http request.
"""Azure Cosmos 'DELETE' async http request.

:params str url:
:params str path:
49 changes: 49 additions & 0 deletions sdk/cosmos/azure-cosmos/azure/cosmos/aio/container.py
Original file line number Diff line number Diff line change
@@ -241,6 +241,55 @@ async def read_item(
response_hook(self.client_connection.last_response_headers, result)
return result

@distributed_trace_async
async def replace_item(
self,
item, # type: Union[str, Dict[str, Any]]
body, # type: Dict[str, Any]
populate_query_metrics=None, # type: Optional[bool]
pre_trigger_include=None, # type: Optional[str]
post_trigger_include=None, # type: Optional[str]
**kwargs # type: Any
):
# type: (...) -> Dict[str, str]
"""Replaces the specified item if it exists in the container.

If the item does not already exist in the container, an exception is raised.

:param item: The ID (name) or dict representing item to be replaced.
:param body: A dict-like object representing the item to replace.
:param populate_query_metrics: Enable returning query metrics in response headers.
:param pre_trigger_include: trigger id to be used as pre operation trigger.
:param post_trigger_include: trigger id to be used as post operation trigger.
:keyword str session_token: Token for use with Session consistency.
:keyword dict[str,str] initial_headers: Initial headers to be sent as part of the request.
:keyword str etag: An ETag value, or the wildcard character (*). Used to check if the resource
has changed, and act according to the condition specified by the `match_condition` parameter.
:keyword ~azure.core.MatchConditions match_condition: The match condition to use upon the etag.
:keyword Callable response_hook: A callable invoked with the response metadata.
:returns: A dict representing the item after replace went through.
:raises ~azure.cosmos.exceptions.CosmosHttpResponseError: The replace failed or the item with
given id does not exist.
:rtype: dict[str, Any]
"""
item_link = self._get_document_link(item)
request_options = build_options(kwargs)
response_hook = kwargs.pop('response_hook', None)
request_options["disableIdGeneration"] = True
if populate_query_metrics is not None:
request_options["populateQueryMetrics"] = populate_query_metrics
if pre_trigger_include is not None:
request_options["preTriggerInclude"] = pre_trigger_include
if post_trigger_include is not None:
request_options["postTriggerInclude"] = post_trigger_include

result = await self.client_connection.ReplaceItem(
document_link=item_link, new_document=body, options=request_options, **kwargs
)
if response_hook:
response_hook(self.client_connection.last_response_headers, result)
return result

@distributed_trace_async
async def delete_item(
self,
136 changes: 106 additions & 30 deletions sdk/cosmos/azure-cosmos/azure/cosmos/aio/database.py
Original file line number Diff line number Diff line change
@@ -98,6 +98,36 @@ def _get_container_link(self, container_or_id):
# type: (Union[str, ContainerProxy, Dict[str, Any]]) -> str
return u"{}/colls/{}".format(self.database_link, self._get_container_id(container_or_id))

@distributed_trace_async
async def read(self, populate_query_metrics=None, **kwargs):
# type: (Optional[bool], Any) -> Dict[str, Any]
"""Read the database properties.

:param bool populate_query_metrics: Enable returning query metrics in response headers.
:keyword str session_token: Token for use with Session consistency.
:keyword dict[str,str] initial_headers: Initial headers to be sent as part of the request.
:keyword Callable response_hook: A callable invoked with the response metadata.
:rtype: Dict[Str, Any]
:raises ~azure.cosmos.exceptions.CosmosHttpResponseError: If the given database couldn't be retrieved.
"""
# TODO this helper function should be extracted from CosmosClient
from .cosmos_client import CosmosClient

database_link = CosmosClient._get_database_link(self)
request_options = build_options(kwargs)
response_hook = kwargs.pop('response_hook', None)
if populate_query_metrics is not None:
request_options["populateQueryMetrics"] = populate_query_metrics

self._properties = await self.client_connection.ReadDatabase(
database_link, options=request_options, **kwargs
)

if response_hook:
response_hook(self.client_connection.last_response_headers, self._properties)

return cast('Dict[str, Any]', self._properties)

@distributed_trace_async
async def create_container(
self,
@@ -255,36 +285,6 @@ async def create_container_if_not_exists(
analytical_storage_ttl=analytical_storage_ttl
)

@distributed_trace_async
async def read(self, populate_query_metrics=None, **kwargs):
# type: (Optional[bool], Any) -> Dict[str, Any]
"""Read the database properties.

:param bool populate_query_metrics: Enable returning query metrics in response headers.
:keyword str session_token: Token for use with Session consistency.
:keyword dict[str,str] initial_headers: Initial headers to be sent as part of the request.
:keyword Callable response_hook: A callable invoked with the response metadata.
:rtype: Dict[Str, Any]
:raises ~azure.cosmos.exceptions.CosmosHttpResponseError: If the given database couldn't be retrieved.
"""
# TODO this helper function should be extracted from CosmosClient
from .cosmos_client import CosmosClient

database_link = CosmosClient._get_database_link(self)
request_options = build_options(kwargs)
response_hook = kwargs.pop('response_hook', None)
if populate_query_metrics is not None:
request_options["populateQueryMetrics"] = populate_query_metrics

self._properties = await self.client_connection.ReadDatabase(
database_link, options=request_options, **kwargs
)

if response_hook:
response_hook(self.client_connection.last_response_headers, self._properties)

return cast('Dict[str, Any]', self._properties)

def get_container_client(self, container):
# type: (Union[str, ContainerProxy, Dict[str, Any]]) -> ContainerProxy
"""Get a `ContainerProxy` for a container with specified ID (name).
@@ -313,6 +313,82 @@ def get_container_client(self, container):

return ContainerProxy(self.client_connection, self.database_link, id_value)

@distributed_trace_async
async def replace_container(
self,
container, # type: Union[str, ContainerProxy, Dict[str, Any]]
partition_key, # type: Any
indexing_policy=None, # type: Optional[Dict[str, Any]]
default_ttl=None, # type: Optional[int]
conflict_resolution_policy=None, # type: Optional[Dict[str, Any]]
populate_query_metrics=None, # type: Optional[bool]
**kwargs # type: Any
):
# type: (...) -> ContainerProxy
"""Reset the properties of the container.

Property changes are persisted immediately. Any properties not specified
will be reset to their default values.

:param container: The ID (name), dict representing the properties or
:class:`ContainerProxy` instance of the container to be replaced.
:param partition_key: The partition key to use for the container.
:param indexing_policy: The indexing policy to apply to the container.
:param default_ttl: Default time to live (TTL) for items in the container.
If unspecified, items do not expire.
:param conflict_resolution_policy: The conflict resolution policy to apply to the container.
:param populate_query_metrics: Enable returning query metrics in response headers.
:keyword str session_token: Token for use with Session consistency.
:keyword str etag: An ETag value, or the wildcard character (*). Used to check if the resource
has changed, and act according to the condition specified by the `match_condition` parameter.
:keyword ~azure.core.MatchConditions match_condition: The match condition to use upon the etag.
:keyword dict[str,str] initial_headers: Initial headers to be sent as part of the request.
:keyword Callable response_hook: A callable invoked with the response metadata.
:raises ~azure.cosmos.exceptions.CosmosHttpResponseError: Raised if the container couldn't be replaced.
This includes if the container with given id does not exist.
:returns: A `ContainerProxy` instance representing the container after replace completed.
:rtype: ~azure.cosmos.ContainerProxy

.. admonition:: Example:

.. literalinclude:: ../samples/examples.py
:start-after: [START reset_container_properties]
:end-before: [END reset_container_properties]
:language: python
:dedent: 0
:caption: Reset the TTL property on a container, and display the updated properties:
:name: reset_container_properties
"""
request_options = build_options(kwargs)
response_hook = kwargs.pop('response_hook', None)
if populate_query_metrics is not None:
request_options["populateQueryMetrics"] = populate_query_metrics

container_id = self._get_container_id(container)
container_link = self._get_container_link(container_id)
parameters = {
key: value
for key, value in {
"id": container_id,
"partitionKey": partition_key,
"indexingPolicy": indexing_policy,
"defaultTtl": default_ttl,
"conflictResolutionPolicy": conflict_resolution_policy,
}.items()
if value is not None
}

container_properties = await self.client_connection.ReplaceContainer(
container_link, collection=parameters, options=request_options, **kwargs
)

if response_hook:
response_hook(self.client_connection.last_response_headers, container_properties)

return ContainerProxy(
self.client_connection, self.database_link, container_properties["id"], properties=container_properties
)

@distributed_trace_async
async def delete_container(
self,
Loading