Add messages implementation for python #165

elchupanebrej · 2023-07-18T18:08:37Z

🤔 What's changed?

Add python implementation

🏷️ What kind of change is this?

⚡ New feature (non-breaking change which adds new behaviour)

📋 Checklist:

I agree to respect and uphold the Cucumber Community Code of Conduct
I've changed the behaviour of the code
- I have added/updated tests to cover my changes.
Users should know about my change
- I have added an entry to the "Unreleased" section of the CHANGELOG, linking to this pull request.

This text was originally generated from a template, then edited by hand. You can modify the template here.

elchupanebrej · 2023-07-18T18:11:58Z

This address to #162

mpkorstanje

At a glance this doesn't follow the pattern used by the other language implementations in quite a few ways. Please follow up the directions from #162 around code generation.

I also don't understand the purpose of the samples directory.

elchupanebrej · 2023-07-19T12:50:14Z

@mpkorstanje

For Python exists a tool that allows generating Pydantic models directly from json schema https://github.com/koxudaxi/datamodel-code-generator - So this allows not including an extra layer with templating. If you insist - I'll rewrite this by that approach.
Samples are taken from gherkin repository to validate if serialization/deserialization works well. Adding external data to a python package is always an egg-chicken problem. I don't like to add external files by makefiles or any kind of scripts because they are always platform dependent. If another approach is used in cucumber - please let me know, and I'll adapt this PR

mpkorstanje · 2023-07-19T15:34:16Z

For Python exists a tool that allows generating Pydantic models directly from json schema

You can use Pydantic if you can make it fit into the make clean-all generate-all workflow. Though I suspect your manual edits might pose a problem.

Samples are taken from gherkin repository to validate if serialization/deserialization works well.

Consider narrowing this down to a few representative examples. Currently it is hard to see the forest for the trees.

luke-hill

If you're going to copy lots of the cck it would be better to fetch the data using some form of call rather than C+P as this is currently being rapidly updated

python/pyproject.toml

.github/workflows/test-python.yml

python/RELEASING.md

mpkorstanje · 2024-01-04T12:55:15Z

@elchupanebrej

Samples are used in tests. More complex tests could exist. I insist to include them for now

What purpose do these tests serve? They'll be a hassle to update if/when the schema changes.

luke-hill · 2024-04-04T14:46:02Z

Hi @elchupanebrej - Just checking in to see where you're up to with this. Is this something you're still working on?

elchupanebrej · 2024-09-04T17:13:54Z

Hi @elchupanebrej - Just checking in to see where you're up to with this. Is this something you're still working on?

Hi @luke-hill, sorry for the long response, hadn't time to work on the project. I'll try to create another merge request that will conform to the building process.

elchupanebrej · 2024-09-04T20:59:59Z

The PR was updated with Makefile. Model is stable, so generated code is totally same to version, which was generated at first try

@mpkorstanje I kindly ask you to review the code and take a release part. I didn't get into all deps&relations between release tools.

python/tests/test_model_load.py

mpkorstanje · 2024-09-04T21:26:39Z

Left a few quick remarks, will have to take a deeper look later.

python/Makefile

luke-hill · 2024-09-05T13:22:06Z

python/src/message_samples/minimal/minimal.feature.ndjson

@@ -0,0 +1,12 @@
+{"meta":{"ci":{"buildNumber":"154666429","git":{"remote":"https://github.com/cucumber-ltd/shouty.rb.git","revision":"99684bcacf01d95875834d87903dcb072306c9ad"},"name":"GitHub Actions","url":"https://github.com/cucumber-ltd/shouty.rb/actions/runs/154666429"},"cpu":{"name":"x64"},"implementation":{"name":"fake-cucumber","version":"16.3.0"},"os":{"name":"darwin","version":"22.4.0"},"protocolVersion":"22.0.0","runtime":{"name":"node.js","version":"19.7.0"}}}


I think this is a good process what you've done here. Just commenting for documentation.

I think as/when you have gotten this all working, it would be good to migrate this and others to the CCK proper. WDYT? (Maybe something for 2025?)

migrate this and others to the CCK proper
It must work with CCK now in all possible cases. If it doesn't - let write tests & fix

@elchupanebrej As a test I'm not happy with a "sample test". As said before this create a circular dependency between the code that generates the samples and messages.

Tests for messages can be limited to testing whether the code was generated and serialization works correctly. This is does not test those things specifically while still testing many other - less relevant things.

@luke-hill what exactly do you mean by "migrating this and others to the cck"?

@mpkorstanje sorry for bothering you, it seems I can't catch a point:

Samples of messages in the CCK repository are stored as examples. Every tool that uses messages has to use them (at least serialize when some event is emitted, and deserialize when this message comes to some reporter). So I took the full suite of test data from the CCK repo and checked that the models generated were successfully parsed that messages into the model, and after that deserialized them to totally the same JSON. Could you please describe more precisely what kind of tests would be OK: would be enough if some model(for every kind of message) would be created, serialized and deserialized perfectly to the totally same model?

The CCK uses the messages to generate the output of a canonical cucumber execution. For this is needs the messages. The value the CCK adds isn't that it generates a sample of each messages, but rather that the collection of messages as a whole. So it can for example express relationships between messages.

This dependency also means that it can't be used as test data in messages. That would result in a circular dependency.

Now for messages the exact testing strategy depends on the framework and language used.

For example for Javascript, the object and it's json representation are almost identical so there is little to test at all. And because the code is generated, it doesn't seem nesesary to test every message either.

So you can see we do a round trip test of one moderately complex message and not much more.

https://github.com/cucumber/messages/blob/main/javascript/test/messagesTest.ts

For Java serialization is more complicated. It does not have a concept of undefined. So we got tests to check for that.

https://github.com/cucumber/messages/blob/main/java/src/test/java/io/cucumber/messages/NdjsonSerializationTest.java

Now I don't know enough about Python to tell you exactly what to test. I can't tell you about pitfalls I don't know about. But I imagine if third party code generator is used, a simple round trip should be enough.

To add as well. For the ruby implementation we don't even do something that complex. All we do is "open" the message (Which is an Envelope class), then reclose it again. And check that in doing so we don't change anything. https://github.com/cucumber/messages/blob/main/ruby/spec/cucumber/messages/acceptance_spec.rb

If you're finding that you're going down the rabbit hole of doing lots of testing here, then something is likely going wrong. Messages are the building block that everything else is based off, not the other way around.

Copied ruby approach

mpkorstanje

There seems to have been a misunderstanding.

So just to clarify.

Either:

Source is generated by the Ruby codegen script
Generated source is checked in

Or:

Source is generated by the python build process.
Generated source is not checked in.
Make targets print a message that code code gen is handled by Python.

Which option are you going for now?

.github/workflows/test-python.yml

python/Makefile

python/pyproject.toml

mpkorstanje · 2024-09-08T11:39:41Z

python/src/message_samples/minimal/minimal.feature.ndjson

@@ -0,0 +1,12 @@
+{"meta":{"ci":{"buildNumber":"154666429","git":{"remote":"https://github.com/cucumber-ltd/shouty.rb.git","revision":"99684bcacf01d95875834d87903dcb072306c9ad"},"name":"GitHub Actions","url":"https://github.com/cucumber-ltd/shouty.rb/actions/runs/154666429"},"cpu":{"name":"x64"},"implementation":{"name":"fake-cucumber","version":"16.3.0"},"os":{"name":"darwin","version":"22.4.0"},"protocolVersion":"22.0.0","runtime":{"name":"node.js","version":"19.7.0"}}}


@elchupanebrej As a test I'm not happy with a "sample test". As said before this create a circular dependency between the code that generates the samples and messages.

Tests for messages can be limited to testing whether the code was generated and serialization works correctly. This is does not test those things specifically while still testing many other - less relevant things.

@luke-hill what exactly do you mean by "migrating this and others to the cck"?

python/src/message_samples/minimal/minimal.feature.ts

python/src/messages.py

python/src/_messages.py

python/src/message_samples/__init__.py

youtux · 2024-09-15T16:01:09Z

python/pyproject.toml

+]
+dependencies = [
+  "importlib_resources",
+  "pydantic>=2.0.3"


Is it really necessary to use and add pydantic as a dependency ?
Many people are still on pydantic v1, and this would require pytest-bdd users to upgrade to pydantic v2 since pytest-bdd will soon depend on gherkin

Aren’t stdlib dataclasses enough?

importlib_resources is also, from what I can see, only used for tests which I'm not sure is needed either

@youtux Yes, this is technically possible, but such realization will be dependent on some library like https://github.com/lidatong/dataclasses-json (the best option for now), which are not as good supported as pydantic

From another perspective - testing utilities are selected at the start of a project, so if the messages package will be used somewhere - it most probably would be dependent on the new version of Pydantic

but there are many projects using pytest-bdd for years, and this would be an issue.
We can do without pydantic in a very simple way. We can use data classes, then when we need to serialise to json we call asdict(model). If we need custom encoders (e.g. for date times) we can implement a simple JsONEncoder and pass that to json.dumps(asdict(model), encoder=…).

Or also just implement custom serialiser for each object

in this case, we have to implement dict_factory for dataclass.asdict, which will have to take in count Enums, or there would be an issue with serialization to JSON. And deserialisation to the dataclass also will be an issue (Enums again)
And pydantic covers both of this issues

I really think we should not bring in a big dependency like pydantic here, especially since it has made a big API change in v2, and I can see it make it difficult for users to adopt this library if it conflicts with their pydantic v1 requirement.

What's the use of pydantic here? I don't see it being used for serialisation / deserialisation here.
What's the API of this library going to look like?

Messages library is hardly used for serialization/deserialization, for example:

Test runner must produce messages in the ndjson format, so it uses model of "messages" lib to represent outcomes, messages lib serializes and validates against Json schema (non-directly).

Test reporter consumes ndjson stream of messages and uses "messages" library to deserialize inputs and validate them.

So "messages" lib is a bridge between test runner and test reporter (potentially from different languages ecosystems)

ok, but how is the API of this lib supposed to look like?

from cucumber_messages import ??? ???

@youtux , please check python/tests/test_model_load.py test in this PR (I'll rework tests later).

For example reporting in the pytest-bdd-ng uses this particular model:
https://github.com/elchupanebrej/pytest-bdd-ng/blob/default/src/pytest_bdd/message_plugin.py

Exrta dependencies were declined

elchupanebrej · 2024-09-17T05:41:08Z

Thanks for great review, return later this week and will update all things accordingly 😀

elchupanebrej · 2024-12-13T18:23:36Z

This would use lidatong/dataclasses-json#442 in future

elchupanebrej · 2024-12-19T00:26:10Z

@mpkorstanje @luke-hill @jsa34 @youtux
Please make a new review for this PR:

Model doesn't use Pydantic anymore (dataclasses are used)
Model generation depends on ruby templates (so no more Makefile workarounds)
Tests are provided in the same manner as at other language implementations

luke-hill

I've reviewed 2/3rds of this PR so far and only got a couple of questions. So I'll pass that in for now. The other items are a bit more complex and I'll review at a later date (Might be after xmas now).

.github/workflows/test-codegen.yml

.pre-commit-config.yaml

codegen/templates/python.py.erb

python/pyproject.toml

luke-hill

Reviewed and signed off 17/21 files, still got a few things to query

.github/workflows/test-python.yml

codegen/templates/python.py.erb

python/src/cucumber_messages/_messages.py

youtux · 2025-01-11T21:35:57Z

I must have missed something. Why are we not using the code generator from https://github.com/koxudaxi/datamodel-code-generator anymore?

It would look more maintainable to me to use that one, as supporting the current proposed solution requires people that both both Ruby and Python, rather than just Python.

luke-hill

There's a few other minor things, but it'll be easier for me to just run rubocop or something on it afterwards as it's just syntax-y stuff.

I've ignored the generated files and the remaining 3 files to sign off have review stuff, so I've finished on this now. Great stuff.

.github/workflows/test-python.yml

codegen/generators/python.rb

luke-hill · 2025-01-21T14:33:09Z

I've resolved the generator issues I raised because I'd rather get this merged in and then just tackle those as a patch release later on myself as it'll speed things up and allow the python people to work on other items such as expressions and suchlike

* Fixup property type definitions * Fixup property descriptions * Descriptions inlined where possible * Property descriptions are placed after properties per se * Remove redundant double-quotes at type definitions * Split enums and model templates * Simplify gh-action test matrix * Fixup empty project.toml settings

elchupanebrej · 2025-01-22T17:54:38Z

I must have missed something. Why are we not using the code generator from https://github.com/koxudaxi/datamodel-code-generator anymore?

It would look more maintainable to me to use that one, as supporting the current proposed solution requires people that both Ruby and Python, rather than just Python.

Python environment to generate code was unwanted
There was an extra dependency

luke-hill

22/23 files I'm good with. Few questions raised on final one - but might all be redundant. Top work.

As an aside - as/when I come to refactor the templates/generators, do you want the messages in 1 file per class like they are in some of the other languages. Not important for now, but just thought I'd make you aware that's quite easy with a bit of chopping that we do (Not required for this PR)

python/pyproject.toml

Co-authored-by: Luke Hill <[email protected]>

youtux · 2025-01-23T12:23:27Z

I must have missed something. Why are we not using the code generator from https://github.com/koxudaxi/datamodel-code-generator anymore?

It would look more maintainable to me to use that one, as supporting the current proposed solution requires people that both Ruby and Python, rather than just Python.

Python environment to generate code was unwanted

There was an extra dependency

The extra dependency would only be a build-time dependency, so it wouldn't be needed by downstream users.

luke-hill

Top stuff. Please get a review from Rien/David also as they are the primary maintainers for Java/JS.

I don't foresee any major issues with other flavours such as go/dotnet e.t.c.

Review items implemented

elchupanebrej force-pushed the python-impl branch from 18b7d09 to 6a26520 Compare July 18, 2023 18:10

mpkorstanje requested changes Jul 18, 2023

View reviewed changes

elchupanebrej marked this pull request as draft July 19, 2023 15:08

luke-hill requested changes Nov 1, 2023

View reviewed changes

python/pyproject.toml Show resolved Hide resolved

.github/workflows/test-python.yml Outdated Show resolved Hide resolved

elchupanebrej force-pushed the python-impl branch 2 times, most recently from ee63f2a to 358b36b Compare December 31, 2023 18:50

luke-hill reviewed Jan 4, 2024

View reviewed changes

python/RELEASING.md Outdated Show resolved Hide resolved

elchupanebrej mentioned this pull request Jan 12, 2024

Add python implemeatation to official messages library elchupanebrej/pytest-bdd-ng#104

Open

elchupanebrej force-pushed the python-impl branch 2 times, most recently from 3256104 to effdd2b Compare September 4, 2024 20:54

mpkorstanje reviewed Sep 4, 2024

View reviewed changes

python/tests/test_model_load.py Show resolved Hide resolved

mpkorstanje marked this pull request as ready for review September 4, 2024 21:26

mpkorstanje reviewed Sep 4, 2024

View reviewed changes

python/Makefile Outdated Show resolved Hide resolved

luke-hill reviewed Sep 5, 2024

View reviewed changes

elchupanebrej force-pushed the python-impl branch 2 times, most recently from ae519d7 to 99e72d6 Compare September 7, 2024 15:39

mpkorstanje requested changes Sep 8, 2024

View reviewed changes

mpkorstanje reviewed Sep 8, 2024

View reviewed changes

python/src/_messages.py Outdated Show resolved Hide resolved

jsa34 reviewed Sep 15, 2024

View reviewed changes

python/src/message_samples/__init__.py Outdated Show resolved Hide resolved

youtux reviewed Sep 15, 2024

View reviewed changes

elchupanebrej force-pushed the python-impl branch from c127cc1 to f6ecf72 Compare September 21, 2024 10:55

elchupanebrej force-pushed the python-impl branch from 55c3a88 to 575bfbb Compare December 11, 2024 19:59

elchupanebrej force-pushed the python-impl branch from 7821dbc to b818731 Compare December 18, 2024 23:13

luke-hill reviewed Dec 19, 2024

View reviewed changes

.github/workflows/test-codegen.yml Outdated Show resolved Hide resolved

.pre-commit-config.yaml Show resolved Hide resolved

codegen/templates/python.py.erb Show resolved Hide resolved

elchupanebrej force-pushed the python-impl branch from e926466 to fe353a2 Compare December 19, 2024 14:18

mpkorstanje self-requested a review December 20, 2024 21:09

kieran-ryan reviewed Jan 9, 2025

View reviewed changes

python/pyproject.toml Outdated Show resolved Hide resolved

luke-hill requested changes Jan 9, 2025

View reviewed changes

.github/workflows/test-python.yml Outdated Show resolved Hide resolved

codegen/templates/python.py.erb Show resolved Hide resolved

youtux reviewed Jan 11, 2025

View reviewed changes

python/src/cucumber_messages/_messages.py Outdated Show resolved Hide resolved

youtux reviewed Jan 11, 2025

View reviewed changes

python/src/cucumber_messages/_messages.py Outdated Show resolved Hide resolved

youtux reviewed Jan 11, 2025

View reviewed changes

python/src/cucumber_messages/_messages.py Outdated Show resolved Hide resolved

youtux reviewed Jan 11, 2025

View reviewed changes

python/src/cucumber_messages/_messages.py Outdated Show resolved Hide resolved

luke-hill requested changes Jan 12, 2025

View reviewed changes

.github/workflows/test-python.yml Show resolved Hide resolved

codegen/generators/python.rb Outdated Show resolved Hide resolved

codegen/generators/python.rb Outdated Show resolved Hide resolved

codegen/generators/python.rb Outdated Show resolved Hide resolved

elchupanebrej added 2 commits January 22, 2025 19:45

[python] Add messages implementation for python

9b8bffc

elchupanebrej force-pushed the python-impl branch from 6a547ee to caee375 Compare January 22, 2025 17:45

elchupanebrej requested a review from luke-hill January 22, 2025 17:55

luke-hill reviewed Jan 23, 2025

View reviewed changes

python/pyproject.toml Show resolved Hide resolved

python/pyproject.toml Outdated Show resolved Hide resolved

python/pyproject.toml Show resolved Hide resolved

Update python/pyproject.toml

bc38183

Co-authored-by: Luke Hill <[email protected]>

luke-hill approved these changes Jan 23, 2025

View reviewed changes

luke-hill mentioned this pull request Jan 23, 2025

Revisit python generator templates #272

Open

davidjgoss added 2 commits January 29, 2025 13:07

Merge branch 'main' into python-impl

9d1e77a

Update CHANGELOG.md

0a03aac

luke-hill merged commit 4ed7f02 into cucumber:main Jan 29, 2025
38 checks passed

		@@ -0,0 +1,12 @@
		{"meta":{"ci":{"buildNumber":"154666429","git":{"remote":"https://github.com/cucumber-ltd/shouty.rb.git","revision":"99684bcacf01d95875834d87903dcb072306c9ad"},"name":"GitHub Actions","url":"https://github.com/cucumber-ltd/shouty.rb/actions/runs/154666429"},"cpu":{"name":"x64"},"implementation":{"name":"fake-cucumber","version":"16.3.0"},"os":{"name":"darwin","version":"22.4.0"},"protocolVersion":"22.0.0","runtime":{"name":"node.js","version":"19.7.0"}}}

Add messages implementation for python #165

Add messages implementation for python #165

Conversation

elchupanebrej commented Jul 18, 2023

🤔 What's changed?

🏷️ What kind of change is this?

📋 Checklist:

elchupanebrej commented Jul 18, 2023

mpkorstanje left a comment • edited Loading

Choose a reason for hiding this comment

elchupanebrej commented Jul 19, 2023

mpkorstanje commented Jul 19, 2023 • edited Loading

luke-hill left a comment

Choose a reason for hiding this comment

mpkorstanje commented Jan 4, 2024 • edited Loading

luke-hill commented Apr 4, 2024

elchupanebrej commented Sep 4, 2024

elchupanebrej commented Sep 4, 2024

mpkorstanje commented Sep 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mpkorstanje Oct 28, 2024 • edited Loading

Choose a reason for hiding this comment

luke-hill Nov 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mpkorstanje left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elchupanebrej commented Sep 17, 2024

elchupanebrej commented Dec 13, 2024

elchupanebrej commented Dec 19, 2024 • edited Loading

luke-hill left a comment

Choose a reason for hiding this comment

luke-hill left a comment

Choose a reason for hiding this comment

youtux commented Jan 11, 2025

luke-hill left a comment

Choose a reason for hiding this comment

luke-hill commented Jan 21, 2025

elchupanebrej commented Jan 22, 2025

luke-hill left a comment

Choose a reason for hiding this comment

youtux commented Jan 23, 2025

luke-hill left a comment

Choose a reason for hiding this comment

mpkorstanje left a comment •

edited

Loading

mpkorstanje commented Jul 19, 2023 •

edited

Loading

mpkorstanje commented Jan 4, 2024 •

edited

Loading

mpkorstanje Oct 28, 2024 •

edited

Loading

luke-hill Nov 21, 2024 •

edited

Loading

elchupanebrej commented Dec 19, 2024 •

edited

Loading