Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Search attributes #43

Merged
merged 11 commits into from
Jun 10, 2022
Merged

Search attributes #43

merged 11 commits into from
Jun 10, 2022

Conversation

cretz
Copy link
Member

@cretz cretz commented Jun 8, 2022

What was changed

Checklist

  1. Closes Search attributes #26

cretz added 2 commits June 9, 2022 12:43
# Conflicts:
#	.github/workflows/ci.yml
#	README.md
#	temporalio/worker/workflow.py
#	temporalio/workflow.py
#	tests/worker/test_workflow.py
@cretz cretz requested review from bergundy, lorensr and a team June 9, 2022 21:48
@cretz cretz marked this pull request as ready for review June 9, 2022 21:48
temporalio/workflow.py Outdated Show resolved Hide resolved
tests/worker/test_workflow.py Outdated Show resolved Hide resolved
) -> None:
"""Encode search attributes as bridge payloads."""
for k, vals in attrs.items():
payloads[k].CopyFrom(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this create the key k if it doesn't exist?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or do you always make sure it exists before sending to this method?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a Protobuf oddity. They lazily create everything on first scalar (i.e. non-message) update. See https://developers.google.com/protocol-buffers/docs/reference/python-generated#map-fields. If I tried to do payloads[k] = some_message, I'd get an error because no Python Protobuf code allows assignment to a message.

raise TypeError("Search attribute values must be lists")
# Convert dates to strings
safe_vals = []
for v in vals:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should assert that vals is a homogenous list

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could also limit this at the type level.
We should have done the same in TS @lorensr (still not too late).

type SearchAttributeValueArray = string[] | number[] | boolean[] | Date[];
type SearchAttributes = Record<string, SearchAttributeVAlueArray>;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, will do.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, will do

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am afraid that Union[List[str], List[int], List[float], List[bool], List[datetime]] doesn't work as we'd like. You can't assign an empty list to it for example (this is a known MyPy and Python typing issue, see issue 3283 in https://github.com/python/mypy). We're gonna have to keep the types as is and only do runtime checks.

Copy link
Member

@bergundy bergundy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should decode the SAs in the wrapped describe response, I didn't see that.

@cretz
Copy link
Member Author

cretz commented Jun 10, 2022

You should decode the SAs in the wrapped describe response, I didn't see that.

@bergundy - I think we need more details on what should be in the describe response. Right now I don't surface anything but the status and the raw gRPC message (so they can decode if they want). Why did TypeScript make WorkflowExecutionDescription.searchAttributes available but not, say, workflowRunTimeout? I think we should decide what needs to be available at temporalio/features#17, and then I can expose it all on the describe call.

@bergundy
Copy link
Member

I think we should decide what needs to be available at temporalio/features#17

Don't really see a reason why not same for workflowRunTimeout but we never got to that, could document in the sdk-features issue.

@cretz cretz merged commit 8ffa58e into temporalio:main Jun 10, 2022
@cretz cretz deleted the search-attrs branch June 10, 2022 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Search attributes
4 participants