Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set Custom Status #110

Merged
merged 11 commits into from
Apr 27, 2020
Merged

Set Custom Status #110

merged 11 commits into from
Apr 27, 2020

Conversation

priyaananthasankar
Copy link
Collaborator

@priyaananthasankar priyaananthasankar commented Apr 16, 2020

Be able to set custom status field while the orchestration is in progress. Added the following:
Partially addresses #35

  1. API
  2. Sample with setcustomstatus and function chaining
  3. Unit tests that test
    • string status
    • object status
      The custom status itself is of Any type.

This is useful across all orchestration patterns. Especially useful for upcoming monitoring pattern.

@priyaananthasankar
Copy link
Collaborator Author

priyaananthasankar commented Apr 16, 2020

Last 2 commits fixed linter errors FYI - which nox did not catch during dev run. This is a separate issue I have raised #111

And couple more commits to fix some minor README.md issues etc.

Copy link
Collaborator

@davidmrdavid davidmrdavid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some questions, please review my comments below 😉

@priyaananthasankar priyaananthasankar self-assigned this Apr 18, 2020
Copy link
Collaborator

@davidmrdavid davidmrdavid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved on the condition that the latest updates I requested are incorporated, please feel free to merge after that 👍
Thank you for this work!

@@ -0,0 +1,27 @@
import logging

def main(name: str) -> str:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change name to key? The name parameter seems more appropiate to a hello world function



def orchestrator_function(context: df.DurableOrchestrationContext):
"""This function provides the core function chaining orchestration logic
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comments here refer to function chaining which we're doing, but I would want them to also highlight that they are setting custom statuses, which I believe the user can see if they click on the returned 'status URL'....just to differentiate this sample's comments from function_chaining

@priyaananthasankar priyaananthasankar merged commit bc0b459 into Azure:dev Apr 27, 2020
@priyaananthasankar
Copy link
Collaborator Author

Addresses #117

priyaananthasankar added a commit that referenced this pull request Jun 18, 2020
* task_utility docstrings

* ActivityType docstrings

Contains refactoring to conform to python naming conventions to

* Doc strings for Actions

Call Activity and CallActivity with Retry

* Doc Strings History

Included refactor for enum naming conventions in python

* Final docstring bits

docstrings for tasks and json utilities

* add waitforexternalevent basic files

* fix bugs to make waitForExternalEvent working

* add waitforexternalevent samples

* remove explicit binding_info imp

defined in conftest as a fixture, explicit import not required.

* demo sample for function chaining with docstring and json changes

* demo sample for function chaining with docstring and json changes

* added dev pipeline status in README

* minor fixes(variable name, delete comment)

* flake8 fixes

* add docstrings

* implement task_any function

* change pip install library and add docstrings to samples

* change pip install library and add docstrings to samples

* unittest for waitforexternalevents

* fix bugs after merging dev

* fix flake8

* Base implementation of tests

test initial call
test get activity and build of task set

* parrot values success

* test full complete flow

* test failed scenario

add missing bits in task_all to account for a failed task

* docstring to numpy format

* minor changes (rename, remove logging)

* unittest for task_any, added tasks_test_utils

* add class __eq__ function for Waitforexternalevent actions

* add samples readme doc

* fix flake8

* Refactoring HistoryEvent

Get rid of all of those hardcoded property value references

* add docstrings for HistoryEvent class

* Refactor json conversion

refactor the classes that are parsing json strings to remove all of the hardcoded property names
also allows for the classes dangle additional attributes that may be present but not explicitly used for construction

* simple Fan out fan in sample

* Fix flake errors

* Remove local debugging bits

* remove state in task_any

* add handle faulted task_any case +unittest

* Undo De Morgan's Law

cause it's really hard to read

* replace filters with list comprehension

more concise/readable

* Add documentation for tracking API implementation

* move datetime format string to azure package

* replace filter with list comprehension

more concise method

* remove extra zimezone from format

causing parsing warnings

* Push context initialization our of handle method

have this dependency injected instead of built within

* able to pass in tasksets to task_any and task_all

* update unittest for adding timestamp to taskset, add unittest for passing taskset to task_any as args

* fix bugs in task_all(when all tasks fail), and fix unittest for that scenario

* fix flake8

* test from orchestrator level(draft)

* Remove IFunctionContext abstraction

unneeded layer of abstraction with a DUCK typing language like Python

* Starting of schema validation bits

* createCheckStatusResponse()

* wire up schema validation into the orchestrator tests

* Test commit

* fix flake 8 issues

* fix pytest, remove task_any_tests from orchestrator level

* fix flake8

* implement raise_event api, fix docstring

* add unittest, create separate methods, fix naming style, handle placeholder in url validation

* Fan Out Fan In sample

uses tensorflow to predict whether images from bing are dogs or cats

* fix flake8

* add aiohttp to requirement.txt and setup.py

* add async await to start_new and raise_event api, flake8

* update api_implementation_status

* fix variable naming style in docstring

* update sample for external events, update readme in sample/external_events

* Refactoring and docstrings

removed the get generation data function. With the configurability, we can increase the volume of images to simulate the high CPU scenario. No need to fake it now.
refactored some of the larger, multi task functions to create better readability
added some docstrings on the functions to aid in understanding what is going on without the visual aid of powerpoint

* Continue as new implementation

* new_guid implementation

* Fix flake8 issues

* update sample to simple version

* add func.httpresponse in durableorchestratorClient.py

* update docstring for createcheckstatusresponse api

* fix flake8

* update sample for create_check_status_response fixes

* update pytest for the changes in createcheckstatusresponse api, add azure-function to requirement.txt for pytest

* Implementation of call_http

* Remove traceback print statement

* remove url validator

* Updates from demo feedback

rename guid to uuid
attach additional context attributes to an attribute of the durable context, not directly to the durable context

* Fix flake8

* copy paste error

giving a more descriptive orchestrator name

* remove abcd from sample

* Update API_IMPLEMENTATION_STATUS.md

Sprint 7 update

* Unit tests for call_http

found a couple of bugs with these

* merged shervyna changes

* Fix the squiggles

mostly formatting/inspection surpession

* Move iAction

moving and renaming IAction to Action

* Update actions to implement action base class

* remove the squiggles

some minor formatting updates

* Fix format of docstring

* update to use rpc_base_url for start new and raise event urls

* minor updates

docstring on new function
add Action to __init__ for actions module

* flake8 bits

* Add action_type property of base class

* update docstrings per convention

* docstring convention updates

made sure docstrings conformed to the same format and indentions

* Base class implementations for get_status

* refactor massive string into a Dict[str, Any] object

makes for a more readable bit of code

* Move duplicate test rpc url constants to centralized location

* adding updated contributing and getting started guides

* get_status implementation

includes refactoring of aiohttp calls to be able to test the rest of the bits without this server requirement
tests for GetStatusOptions included
base test for get_status_success

* test for DurableOrchestrationStatus parsing

* Add non ok message tests

* add raises exception test

* get_status_by and get_status_all implementation

* flake8 fixes

* fix docstring for get_status_by parameters

copy paste error

* addig nox to the project - updated requirements file

* Updated noxfile to include flake, move getting started to investigations
folder

* typo in the commit from Jscript to python

* PR recommendations

remove PyCharm # noinspection tags
move get and post async functions to util module
replace abc123 stand-in test value with a uuid stand-in

* add docstrings for new public functions

* Continued Client API implementation

purge_instance_history
purge_instance_history_by
terminate
wait_for_completion_or_create_check_status_response

* bug fixed for retry function.

* refactor tests

remove the redundant function declarations

* unit tests for purge history and terminate

* unit tests for wait or create check response

includes bug fixes found from tests

* update implementation status with features included in branch

* update name match the name of the class under test

* change to use asynio sleep

don't want a blocking sleep call here

* Use azure-functions>=1.2.0

* Make DurableOrchestrationContext available for import

* Update start_new type hints to return str

* Add get_input to DurableOrchestrationContext

* Allow passing OrchestrationContext to create

- maintain backwards compatibility when JSON body is passed

* Change str to Awaitable[str]

* Add get_input tests, get_input always returns str

* Fix flake8 errors

* add furl to setup.py

* Fix merge error

* # This is a combination of 3 commits. (#58)

Fix samples to use the latest func.OrchestrationContext annotation

Use new syntax to replace main endpoint

Fix old python_durable_bindings samples

* Replace badge with the new Azure Functions Python pipeline

* edited contributor guide

* furl is required for this project - added it to the requirements.doc

* fixed pytest-asyncio in requirements.txt

* fixed pytest-asyncio in requirements.txt

* fixed pytest-asyncio in requirements.txt

* Remove static method tag (#72)

Remove static method tag (#72): residual bits left over from when a part of the client class. Causing failures in current state. Also removing unnecessary wrapper of http helper calls

* Remove furl from RpcManagementOptions

only durable class that is requiring this package, and is only using it to construct a url query string. Lighter requirements list to just use string joining

* release artifacts

* Added DateUtil and Furl to install_requires

* changelog and setup.py changes

* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipelines

* tested version of pipeline yaml

* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipeline

* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipelines

* setup.py CD testing

* setup.py CD testing

* PyPI testing

* added info logging to the external event sample

* Correctness and documentation of the samples (#85)

* validated all samples except tensorflow

* deleted non-samples, tested tensorflow sample, added types and comments

* new readmes for fan-out-fan-in and function-chaining

* renamed some functions so they are easier to reason about

* disable aiohttp's input validation... potentially unsafe. ExternalEvents now shows no warnings!

* Fixed Contributor Guide Issues

* Fixed Contributor Guide Issues

* Input value not required for start_new

* input values not required for start_new

* Add limitations, link to quickstart, update samples

* Update example

* Add link

* Update words

* Add versions

* Update links

* versioning-via-tags is enabled (#101)

* updated readme for external events (#103)

* Readme file for FanOutFanIn (#104)

* Readme file for FanOutFanIn

* Readme file for FanOutFanIn

* Readme file for FanOutFanIn

* Add activity trigger return type sample (#105)

* Add sample for activity trigger type checks

* Remove extensions.csproj

Co-authored-by: Priya Ananthasankar <[email protected]>

* misc contributions (#108)

* Set Custom Status (#110)

* custom status

* custom status

* custom status

* custom status

* custom status

* custom status

* custom status

* custom status

* set custom status rework comments

* Enabled nox to recognize docstring formatting errors (#122)

* enables trivial orchestrators

* added flake8-docstring, fixed arg to flake8 ,fixed formatting on noxfile

* removed accidental commit from other PR

* enables trivial orchestrators (#121)

* removed grpc folder (#120)

* [WIP] Semi-automatic type-serialization  (#109)

* Promote to master (#99) (#126)

Co-authored-by: Shawn Gaul <[email protected]>
Co-authored-by: Shervyna Ruan <[email protected]>
Co-authored-by: Shawn Gaul <[email protected]>
Co-authored-by: asedighi <[email protected]>
Co-authored-by: Anthony <[email protected]>
Co-authored-by: Anthony Chu <[email protected]>
Co-authored-by: Hanzhang Zeng (Roger) <[email protected]>
Co-authored-by: Hanzhang Zeng (Roger) <[email protected]>
Co-authored-by: David Justo <[email protected]>

Co-authored-by: Priya Ananthasankar <[email protected]>
Co-authored-by: Shawn Gaul <[email protected]>
Co-authored-by: Shervyna Ruan <[email protected]>
Co-authored-by: Shawn Gaul <[email protected]>
Co-authored-by: asedighi <[email protected]>
Co-authored-by: Anthony <[email protected]>
Co-authored-by: Anthony Chu <[email protected]>
Co-authored-by: Hanzhang Zeng (Roger) <[email protected]>
Co-authored-by: Hanzhang Zeng (Roger) <[email protected]>

* Promote to master (#99) (#127)

* Monitoring Feature (#119)

* updated CHANGELOG (#128)

* Changelog includes links (#129)

* updated CHANGELOG

* improved changelog descriptions w/ issue links

Co-authored-by: Shawn Gaul <[email protected]>
Co-authored-by: Shervyna Ruan <[email protected]>
Co-authored-by: Priya Ananthasankar <[email protected]>
Co-authored-by: Shawn Gaul <[email protected]>
Co-authored-by: asedighi <[email protected]>
Co-authored-by: Anthony <[email protected]>
Co-authored-by: Anthony Chu <[email protected]>
Co-authored-by: Hanzhang Zeng (Roger) <[email protected]>
Co-authored-by: Hanzhang Zeng (Roger) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants