Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to use timespan #20233

Merged
merged 7 commits into from
Aug 16, 2021
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions sdk/monitor/azure-monitor-query/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
- Rename `LogsBatchQueryRequest` to `LogsBatchQuery`.
- `include_render` is now renamed to `include_visualization` in the query API.
- `LogsQueryResult` and `LogsBatchQueryResult` now return `visualization` instead of `render`.
- `start_time`, `duration` and `end_time` are now replaced with a single param called `timespan`

### Bugs Fixed

Expand Down
25 changes: 12 additions & 13 deletions sdk/monitor/azure-monitor-query/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ Each set of metric values is a time series with the following characteristics:
## Examples

- [Single logs query](#single-logs-query)
- [Specify duration](#specify-duration)
- [Specify timespan](#specify-timespan)
- [Set logs query timeout](#set-logs-query-timeout)
- [Batch logs query](#batch-logs-query)
- [Query metrics](#query-metrics)
Expand All @@ -113,9 +113,9 @@ Each set of metric values is a time series with the following characteristics:

This example shows getting a log query. To handle the response and view it in a tabular form, the [pandas](https://pypi.org/project/pandas/) library is used. See the [samples][python-query-samples] if you choose not to use pandas.

#### Specify duration
#### Specify timespan

The `duration` parameter specifies the time duration for which to query the data. This argument can also be accompanied with either `start_time` or `end_time`. If either `start_time` or `end_time` aren't provided, the current time is used as the end time. As an alternative, the `start_time` and `end_time` arguments can be provided together instead of the `duration` argument. For example:
The `timespan` parameter specifies the time duration for which to query the data. The timespan for which to query the data. This can be a timedelta, a timedelta and a start datetime, or a start datetime/end datetime. For example:

```python
import os
Expand All @@ -132,12 +132,14 @@ client = LogsQueryClient(credential)
query = """AppRequests |
summarize avgRequestDuration=avg(DurationMs) by bin(TimeGenerated, 10m), _ResourceId"""

start_time=datetime(2021, 7, 2)
end_time=datetime.now()

# returns LogsQueryResult
response = client.query(
os.environ['LOG_WORKSPACE_ID'],
query,
start_time=datetime(2021, 6, 2),
end_time=datetime.now()
timespan=(start_time, end_time)
)

if not response.tables:
Expand Down Expand Up @@ -185,14 +187,13 @@ client = LogsQueryClient(credential)
requests = [
LogsBatchQuery(
query="AzureActivity | summarize count()",
duration=timedelta(hours=1),
timespan=timedelta(hours=1),
workspace_id=os.environ['LOG_WORKSPACE_ID']
),
LogsBatchQuery(
query= """AppRequests | take 10 |
summarize avgRequestDuration=avg(DurationMs) by bin(TimeGenerated, 10m), _ResourceId""",
duration=timedelta(hours=1),
start_time=datetime(2021, 6, 2),
timespan=(datetime(2021, 6, 2), timedelta(hours=1)),
workspace_id=os.environ['LOG_WORKSPACE_ID']
),
LogsBatchQueryRequest(
Expand Down Expand Up @@ -270,13 +271,13 @@ from azure.identity import DefaultAzureCredential

credential = DefaultAzureCredential()
client = MetricsQueryClient(credential)

start_time = datetime(2021, 5, 25)
duration = timedelta(days=1)
metrics_uri = os.environ['METRICS_RESOURCE_URI']
response = client.query(
metrics_uri,
metric_names=["PublishSuccessCount"],
start_time=datetime(2021, 5, 25),
duration=timedelta(days=1),
timespan=(start_time, duration)
)

for metric in response.metrics:
Expand Down Expand Up @@ -322,8 +323,6 @@ metrics_uri = os.environ['METRICS_RESOURCE_URI']
response = client.query(
metrics_uri,
metric_names=["MatchedEventCount"],
start_time=datetime(2021, 6, 21),
duration=timedelta(days=1),
aggregations=[AggregationType.COUNT]
)

Expand Down
27 changes: 18 additions & 9 deletions sdk/monitor/azure-monitor-query/azure/monitor/query/_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
# Licensed under the MIT License. See License.txt in the project root for
# license information.
# --------------------------------------------------------------------------
from datetime import datetime, timedelta
from typing import TYPE_CHECKING
from msrest import Serializer
from azure.core.exceptions import HttpResponseError
Expand Down Expand Up @@ -49,26 +50,34 @@ def order_results(request_order, responses):
ordered = [mapping[id] for id in request_order]
return ordered

def construct_iso8601(start=None, end=None, duration=None):
def construct_iso8601(timespan=None):
if not timespan:
return None
try:
start, end, duration = None, None, None
if isinstance(timespan[1], datetime):
start, end = timespan[0], timespan[1]
elif isinstance(timespan[1], timedelta):
start, duration = timespan[0], timespan[1]
else:
raise ValueError('Tuple must be a start datetime with a timedelta or an end datetime.')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add some comments to describe the situation?
e.g. if isinstance(timespan[1], datetime): # if timespan is tuple[datetime, datetime], we treat it as [start time, end time].

except TypeError:
duration = timespan
if duration is not None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can just use if duration: :)

duration = 'PT{}S'.format(duration.total_seconds())
try:
duration = 'PT{}S'.format(duration.total_seconds())
except AttributeError:
raise ValueError('timespan must be a timedelta or a tuple.')
iso_str = None
if start is not None:
start = Serializer.serialize_iso(start)
if end and duration:
raise ValueError("start_time can only be provided with duration or end_time, but not both.")
if end is not None:
end = Serializer.serialize_iso(end)
iso_str = start + '/' + end
elif duration is not None:
iso_str = start + '/' + duration
else:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In which case will enter here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

premeptively raising in case start is provided with something other than a timedelta or a datetime

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks to me here is the case start is not None while end and duration are None?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

improved the message to warn about none explicitly

raise ValueError("Start time must be provided along with duration or end time.")
elif end is not None:
if not duration:
raise ValueError("End time must be provided along with duration or start time.")
end = Serializer.serialize_iso(end)
iso_str = duration + '/' + end
else:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, in which case will enter here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in case timespan is a timedelta and no start is provided

iso_str = duration
return iso_str
Original file line number Diff line number Diff line change
Expand Up @@ -49,28 +49,22 @@ def __init__(self, credential, **kwargs):
)
self._query_op = self._client.query

def query(self, workspace_id, query, duration=None, **kwargs):
def query(self, workspace_id, query, timespan=None, **kwargs):
# type: (str, str, Optional[timedelta], Any) -> LogsQueryResult
"""Execute an Analytics query.

Executes an Analytics query for data.

**Note**: Although the start_time, end_time, duration are optional parameters, it is highly
recommended to specify the timespan. If not, the entire dataset is queried.

:param workspace_id: ID of the workspace. This is Workspace ID from the Properties blade in the
Azure portal.
:type workspace_id: str
:param query: The Analytics query. Learn more about the `Analytics query syntax
<https://azure.microsoft.com/documentation/articles/app-insights-analytics-reference/>`_.
:type query: str
:param ~datetime.timedelta duration: The duration for which to query the data. This can also be accompanied
with either start_time or end_time. If start_time or end_time is not provided, the current time is
taken as the end time.
:keyword datetime start_time: The start time from which to query the data. This should be accompanied
with either end_time or duration.
:keyword datetime end_time: The end time till which to query the data. This should be accompanied
with either start_time or duration.
:param timespan: The timespan for which to query the data. This can be a timedelta,
a timedelta and a start datetime, or a start datetime/end datetime.
:type timespan: ~datetime.timedelta or tuple[~datetime.datetime, ~datetime.timedelta]
or tuple[~datetime.datetime, ~datetime.datetime]
:keyword int server_timeout: the server timeout in seconds. The default timeout is 3 minutes,
and the maximum timeout is 10 minutes.
:keyword bool include_statistics: To get information about query statistics.
Expand All @@ -93,9 +87,7 @@ def query(self, workspace_id, query, duration=None, **kwargs):
:dedent: 0
:caption: Get a response for a single Log Query
"""
start = kwargs.pop('start_time', None)
end = kwargs.pop('end_time', None)
timespan = construct_iso8601(start, end, duration)
timespan = construct_iso8601(timespan)
include_statistics = kwargs.pop("include_statistics", False)
include_visualization = kwargs.pop("include_visualization", False)
server_timeout = kwargs.pop("server_timeout", None)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ def __init__(self, credential, **kwargs):
self._namespace_op = self._client.metric_namespaces
self._definitions_op = self._client.metric_definitions

def query(self, resource_uri, metric_names, duration=None, **kwargs):
def query(self, resource_uri, metric_names, **kwargs):
# type: (str, list, Optional[timedelta], Any) -> MetricsResult
"""Lists the metric values for a resource.

Expand All @@ -64,13 +64,10 @@ def query(self, resource_uri, metric_names, duration=None, **kwargs):
:type resource_uri: str
:param metric_names: The names of the metrics to retrieve.
:type metric_names: list[str]
:param ~datetime.timedelta duration: The duration for which to query the data. This can also be accompanied
with either start_time or end_time. If start_time or end_time is not provided, the current time is
taken as the end time.
:keyword datetime start_time: The start time from which to query the data. This should be accompanied
with either end_time or duration.
:keyword datetime end_time: The end time till which to query the data. This should be accompanied
with either start_time or duration.
:keyword timespan: The timespan for which to query the data. This can be a timedelta,
a timedelta and a start datetime, or a start datetime/end datetime.
:paramtype timespan: ~datetime.timedelta or tuple[~datetime.datetime, ~datetime.timedelta]
or tuple[~datetime.datetime, ~datetime.datetime]
:keyword interval: The interval (i.e. timegrain) of the query.
:paramtype interval: ~datetime.timedelta
:keyword aggregations: The list of aggregation types to retrieve. Use `azure.monitor.query.AggregationType`
Expand Down Expand Up @@ -112,12 +109,11 @@ def query(self, resource_uri, metric_names, duration=None, **kwargs):
:dedent: 0
:caption: Get a response for a single Metrics Query
"""
start = kwargs.pop('start_time', None)
end = kwargs.pop('end_time', None)

aggregations = kwargs.pop("aggregations", None)
if aggregations:
kwargs.setdefault("aggregation", ",".join(aggregations))
timespan = construct_iso8601(start, end, duration)
timespan = construct_iso8601(kwargs.pop("timespan", None))
kwargs.setdefault("metricnames", ",".join(metric_names))
kwargs.setdefault("timespan", timespan)
generated = self._metrics_op.list(resource_uri, connection_verify=False, **kwargs)
Expand Down
17 changes: 6 additions & 11 deletions sdk/monitor/azure-monitor-query/azure/monitor/query/_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -158,13 +158,10 @@ class LogsBatchQuery(object):
:param query: The Analytics query. Learn more about the `Analytics query syntax
<https://azure.microsoft.com/documentation/articles/app-insights-analytics-reference/>`_.
:type query: str
:param ~datetime.timedelta duration: The duration for which to query the data. This can also be accompanied
with either start_time or end_time. If start_time or end_time is not provided, the current time is
taken as the end time.
:keyword datetime start_time: The start time from which to query the data. This should be accompanied
with either end_time or duration.
:keyword datetime end_time: The end time till which to query the data. This should be accompanied
with either start_time or duration.
:param timespan: The timespan for which to query the data. This can be a timedelta,
a timedelta and a start datetime, or a start datetime/end datetime.
:type timespan: ~datetime.timedelta or tuple[~datetime.datetime, ~datetime.timedelta]
or tuple[~datetime.datetime, ~datetime.datetime]
:keyword additional_workspaces: A list of workspaces that are included in the query.
These can be qualified workspace names, workspace Ids, or Azure resource Ids.
:paramtype additional_workspaces: list[str]
Expand All @@ -180,7 +177,7 @@ class LogsBatchQuery(object):
:paramtype headers: dict[str, str]
"""

def __init__(self, query, workspace_id, duration=None, **kwargs): #pylint: disable=super-init-not-called
def __init__(self, query, workspace_id, timespan, **kwargs): #pylint: disable=super-init-not-called
# type: (str, str, Optional[str], Any) -> None
include_statistics = kwargs.pop("include_statistics", False)
include_visualization = kwargs.pop("include_visualization", False)
Expand All @@ -202,9 +199,7 @@ def __init__(self, query, workspace_id, duration=None, **kwargs): #pylint: disab
headers['Prefer'] = prefer
except TypeError:
headers = {'Prefer': prefer}
start = kwargs.pop('start_time', None)
end = kwargs.pop('end_time', None)
timespan = construct_iso8601(start, end, duration)
timespan = construct_iso8601(timespan)
additional_workspaces = kwargs.pop("additional_workspaces", None)
self.id = kwargs.get("request_id", str(uuid.uuid4()))
self.body = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
# license information.
# --------------------------------------------------------------------------

from datetime import timedelta
from typing import Any, Union, Sequence, Dict, Optional, TYPE_CHECKING
from datetime import datetime, timedelta
from typing import Any, Tuple, Union, Sequence, Dict, Optional, TYPE_CHECKING
from azure.core.exceptions import HttpResponseError
from .._generated.aio._monitor_query_client import MonitorQueryClient

Expand Down Expand Up @@ -42,28 +42,22 @@ async def query(
self,
workspace_id: str,
query: str,
duration: Optional[timedelta] = None,
timespan: Optional[Union[timedelta, Tuple[datetime, timedelta], Tuple[datetime, datetime]]] = None,
**kwargs: Any) -> LogsQueryResult:
"""Execute an Analytics query.

Executes an Analytics query for data.

**Note**: Although the start_time, end_time, duration are optional parameters, it is highly
recommended to specify the timespan. If not, the entire dataset is queried.

:param workspace_id: ID of the workspace. This is Workspace ID from the Properties blade in the
Azure portal.
:type workspace_id: str
:param query: The Analytics query. Learn more about the `Analytics query syntax
<https://azure.microsoft.com/documentation/articles/app-insights-analytics-reference/>`_.
:type query: str
:param ~datetime.timedelta duration: The duration for which to query the data. This can also be accompanied
with either start_time or end_time. If start_time or end_time is not provided, the current time is
taken as the end time.
:keyword datetime start_time: The start time from which to query the data. This should be accompanied
with either end_time or duration.
:keyword datetime end_time: The end time till which to query the data. This should be accompanied
with either start_time or duration.
:param timespan: The timespan for which to query the data. This can be a timedelta,
a timedelta and a start datetime, or a start datetime/end datetime.
:type timespan: ~datetime.timedelta or tuple[~datetime.datetime, ~datetime.timedelta]
or tuple[~datetime.datetime, ~datetime.datetime]
:keyword int server_timeout: the server timeout. The default timeout is 3 minutes,
and the maximum timeout is 10 minutes.
:keyword bool include_statistics: To get information about query statistics.
Expand All @@ -77,9 +71,7 @@ async def query(
:rtype: ~azure.monitor.query.LogsQueryResult
:raises: ~azure.core.exceptions.HttpResponseError
"""
start = kwargs.pop('start_time', None)
end = kwargs.pop('end_time', None)
timespan = construct_iso8601(start, end, duration)
timespan = construct_iso8601(timespan)
include_statistics = kwargs.pop("include_statistics", False)
include_visualization = kwargs.pop("include_visualization", False)
server_timeout = kwargs.pop("server_timeout", None)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ async def query(
self,
resource_uri: str,
metric_names: List,
duration: Optional[timedelta] = None,
**kwargs: Any
) -> MetricsResult:
"""Lists the metric values for a resource.
Expand All @@ -59,13 +58,10 @@ async def query(
:type resource_uri: str
:param metric_names: The names of the metrics to retrieve.
:type metric_names: list
:param ~datetime.timedelta duration: The duration for which to query the data. This can also be accompanied
with either start_time or end_time. If start_time or end_time is not provided, the current time is
taken as the end time.
:keyword datetime start_time: The start time from which to query the data. This should be accompanied
with either end_time or duration.
:keyword datetime end_time: The end time till which to query the data. This should be accompanied
with either start_time or duration.
:keyword timespan: The timespan for which to query the data. This can be a timedelta,
a timedelta and a start datetime, or a start datetime/end datetime.
:paramtype timespan: ~datetime.timedelta or tuple[~datetime.datetime, ~datetime.timedelta]
or tuple[~datetime.datetime, ~datetime.datetime]
:keyword interval: The interval (i.e. timegrain) of the query.
:paramtype interval: ~datetime.timedelta
:keyword aggregations: The list of aggregation types to retrieve. Use `azure.monitor.query.AggregationType`
Expand Down Expand Up @@ -98,9 +94,7 @@ async def query(
:rtype: ~azure.monitor.query.MetricsResult
:raises: ~azure.core.exceptions.HttpResponseError
"""
start = kwargs.pop('start_time', None)
end = kwargs.pop('end_time', None)
timespan = construct_iso8601(start, end, duration)
timespan = construct_iso8601(kwargs.pop('timespan', None))
kwargs.setdefault("metricnames", ",".join(metric_names))
kwargs.setdefault("timespan", timespan)
aggregations = kwargs.pop("aggregations", None)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ async def logs_query():

# returns LogsQueryResult
async with client:
response = await client.query(os.environ['LOG_WORKSPACE_ID'], query)
response = await client.query(os.environ['LOG_WORKSPACE_ID'], query, timespan=None)

if not response.tables:
print("No results for the query")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,8 @@
query = """AppRequests |
summarize avgRequestDuration=avg(DurationMs) by bin(TimeGenerated, 10m), _ResourceId"""

end_time = datetime.now(UTC())

# returns LogsQueryResult
response = client.query(os.environ['LOG_WORKSPACE_ID'], query, duration=timedelta(days=1), end_time=end_time)
response = client.query(os.environ['LOG_WORKSPACE_ID'], query, timespan=timedelta(days=1))

if not response.tables:
print("No results for the query")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,8 @@
query = """AppRequests |
summarize avgRequestDuration=avg(DurationMs) by bin(TimeGenerated, 10m), _ResourceId"""

end_time = datetime.now(UTC())

# returns LogsQueryResult
response = client.query(os.environ['LOG_WORKSPACE_ID'], query, duration=timedelta(hours=1), end_time=end_time)
response = client.query(os.environ['LOG_WORKSPACE_ID'], query, timespan=timedelta(hours=1))

if not response.tables:
print("No results for the query")
Expand Down
Loading