Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

types: support working with binary for Python 3 #211

Merged
merged 2 commits into from
Apr 4, 2022

Conversation

DifferentialOrange
Copy link
Member

@DifferentialOrange DifferentialOrange commented Mar 15, 2022

Merge after #213

This is a breaking change.

Before this patch, both bytes and str were encoded as mp_str. It was
possible to work with utf and non-utf strings, but not with
varbinary (mp_bin) [1]. This patch adds varbinary support for Python 3
by default. With Python 2 the behavior of the connector remains the same.

Before this patch:

  • encoding="utf-8" (default)

    Python 3 -> Tarantool -> Python 3
    str -> mp_str (string) -> str
    bytes -> mp_str (string) -> str
    mp_bin (varbinary) -> bytes
  • encoding=None

    Python 3 -> Tarantool -> Python 3
    bytes -> mp_str (string) -> bytes
    str -> mp_str (string) -> bytes
    mp_bin (varbinary) -> bytes

Using bytes as key was not supported by several methods (delete,
update, select).

After this patch:

  • encoding="utf-8" (default)

    Python 3 -> Tarantool -> Python 3
    str -> mp_str (string) -> str
    bytes -> mp_bin (varbinary) -> bytes
  • encoding=None

    Python 3 -> Tarantool -> Python 3
    bytes -> mp_str (string) -> bytes
    str -> mp_str (string) -> bytes
    mp_bin (varbinary) -> bytes

Using bytes as key are now supported by all methods.

Thus, encoding="utf-8" connection may be used to work with
utf-8 strings and varbinary and encodine=None connection
may be used to work with non-utf-8 strings.

This patch does not add new restrictions (like "do not permit to use
str in encoding=None mode because result may be confusing") to preserve
current behavior (for example, using space name as str in schema
get_space).

  1. Introduce VARBINARY field type tarantool#4201

Closes #105

@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-105-unpack-binary branch 2 times, most recently from 6efb9de to 7da60f3 Compare March 16, 2022 09:38
@DifferentialOrange DifferentialOrange changed the title Differential orange/gh 105 unpack binary types: support working with binary for Python 3 Mar 16, 2022
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-105-unpack-binary branch from 7da60f3 to 797d33d Compare March 16, 2022 10:00
@DifferentialOrange DifferentialOrange marked this pull request as ready for review March 16, 2022 11:24
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-105-unpack-binary branch from 797d33d to 6aa89a6 Compare March 16, 2022 13:13
tarantool/utils.py Outdated Show resolved Hide resolved
test/suites/test_dml.py Outdated Show resolved Hide resolved
test/suites/test_dml.py Outdated Show resolved Hide resolved
tarantool/request.py Show resolved Hide resolved
test/suites/test_dml.py Outdated Show resolved Hide resolved
@DifferentialOrange
Copy link
Member Author

Note: test this PR rebased on #213 to ensure different msgpack package version compatiility

@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-105-unpack-binary branch 4 times, most recently from 8772bc7 to 155b1d7 Compare March 23, 2022 08:12
@DifferentialOrange DifferentialOrange changed the base branch from master to DifferentialOrange/gh-182-github-actions March 23, 2022 08:12
Copy link
Member

@Totktonada Totktonada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no objections, just several suggestions.

LGTM.

test/suites/test_dml.py Outdated Show resolved Hide resolved
test/suites/test_dml.py Outdated Show resolved Hide resolved
test/suites/test_dml.py Outdated Show resolved Hide resolved
tarantool/request.py Show resolved Hide resolved
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-182-github-actions branch 5 times, most recently from c46c513 to 44fc747 Compare March 31, 2022 08:29
Base automatically changed from DifferentialOrange/gh-182-github-actions to master March 31, 2022 08:37
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-105-unpack-binary branch 3 times, most recently from 4ec6748 to 88fa990 Compare March 31, 2022 16:00
This patch add CHANGELOG.md file. Current CHANGELOG contents covers
version 0.7.0, 0.7.1 changes and currently unreleased changes. For
changes in versions before 0.7.0, see releases GitHub page [1].

1. https://github.com/tarantool/tarantool-python/releases
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-105-unpack-binary branch from 2a4194e to 4b8ae94 Compare April 1, 2022 09:24
CHANGELOG.md Outdated Show resolved Hide resolved
@Totktonada
Copy link
Member

I glanced over the changes and I'm completely okay here. Please, finish the review with Sergey and proceed.

@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-105-unpack-binary branch from 4b8ae94 to 3d71614 Compare April 2, 2022 07:29
Copy link
Member

@ligurio ligurio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch!

LGTM

CHANGELOG.md Outdated Show resolved Hide resolved
This is a breaking change.

Before this patch, both bytes and str were encoded as mp_str. It was
possible to work with utf and non-utf strings, but not with
varbinary (mp_bin) [1]. This patch adds varbinary support for Python 3
by default. With Python 2 the behavior of the connector remains
the same.

Before this patch:

* encoding="utf-8" (default)

    Python 3 -> Tarantool          -> Python 3
    str      -> mp_str (string)    -> str
    bytes    -> mp_str (string)    -> str
                mp_bin (varbinary) -> bytes

* encoding=None

    Python 3 -> Tarantool          -> Python 3
    bytes    -> mp_str (string)    -> bytes
    str      -> mp_str (string)    -> bytes
                mp_bin (varbinary) -> bytes

Using bytes as key was not supported by several methods (delete,
update, select).

After this patch:

* encoding="utf-8" (default)

    Python 3 -> Tarantool          -> Python 3
    str      -> mp_str (string)    -> str
    bytes    -> mp_bin (varbinary) -> bytes

* encoding=None

    Python 3 -> Tarantool          -> Python 3
    bytes    -> mp_str (string)    -> bytes
    str      -> mp_str (string)    -> bytes
                mp_bin (varbinary) -> bytes

Using bytes as key are now supported by all methods.

Thus, encoding="utf-8" connection may be used to work with
utf-8 strings and varbinary and encodine=None connection
may be used to work with non-utf-8 strings.

This patch does not add new restrictions (like "do not permit to use
str in encoding=None mode because result may be confusing") to preserve
current behavior (for example, using space name as str in schema
get_space).

1. tarantool/tarantool#4201

Closes #105
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-105-unpack-binary branch from 3d71614 to dd6ef5b Compare April 4, 2022 12:08
@DifferentialOrange DifferentialOrange merged commit 9343e83 into master Apr 4, 2022
@DifferentialOrange DifferentialOrange deleted the DifferentialOrange/gh-105-unpack-binary branch April 4, 2022 16:21
DifferentialOrange added a commit to tarantool/doc that referenced this pull request May 11, 2022
Since the release of tarantool/python 0.8.0 [1] several things has
changed.

* Issue #105 has been fixed [2].
* CI has been migrated to GitHub Actions [3].
* New connection pool (ConnectionPool) with master discovery
  was introduced [4].
* old connection pool (MeshConnection) with round-robin failover was
  deprecated [4].

These changes together with GitHub stars update are introduced with this
patch.

1. https://github.com/tarantool/tarantool-python/releases/tag/0.8.0
2. tarantool/tarantool-python#211
3. tarantool/tarantool-python#213
4. tarantool/tarantool-python#207
DifferentialOrange added a commit to tarantool/doc that referenced this pull request May 11, 2022
Since the release of tarantool-python 0.8.0 [1] several things has
changed.

* Issue tarantool/tarantool-python#105 has been fixed [2].
* CI has been migrated to GitHub Actions [3].
* New connection pool (ConnectionPool) with master discovery
  was introduced [4].
* old connection pool (MeshConnection) with round-robin failover was
  deprecated [4].

These changes together with GitHub stars update are introduced with this
patch.

1. https://github.com/tarantool/tarantool-python/releases/tag/0.8.0
2. tarantool/tarantool-python#211
3. tarantool/tarantool-python#213
4. tarantool/tarantool-python#207
DifferentialOrange added a commit to tarantool/doc that referenced this pull request May 11, 2022
Since the release of tarantool-python 0.8.0 [1] several things has
changed.

* Issue tarantool/tarantool-python#105 has been fixed [2].
* CI has been migrated to GitHub Actions [3].
* New connection pool (ConnectionPool) with master discovery
  was introduced [4].
* old connection pool (MeshConnection) with round-robin failover was
  deprecated [4].

These changes together with GitHub stars update are introduced with this
patch.

1. https://github.com/tarantool/tarantool-python/releases/tag/0.8.0
2. tarantool/tarantool-python#211
3. tarantool/tarantool-python#213
4. tarantool/tarantool-python#207
patiencedaur pushed a commit to tarantool/doc that referenced this pull request May 16, 2022
Since the release of tarantool-python 0.8.0 [1] several things has
changed.

* Issue tarantool/tarantool-python#105 has been fixed [2].
* CI has been migrated to GitHub Actions [3].
* New connection pool (ConnectionPool) with master discovery
  was introduced [4].
* old connection pool (MeshConnection) with round-robin failover was
  deprecated [4].

These changes together with GitHub stars update are introduced with this
patch.

1. https://github.com/tarantool/tarantool-python/releases/tag/0.8.0
2. tarantool/tarantool-python#211
3. tarantool/tarantool-python#213
4. tarantool/tarantool-python#207
patiencedaur added a commit to tarantool/doc that referenced this pull request May 16, 2022
* Update python connector comparison table

Since the release of tarantool-python 0.8.0 [1] several things has
changed.

* Issue tarantool/tarantool-python#105 has been fixed [2].
* CI has been migrated to GitHub Actions [3].
* New connection pool (ConnectionPool) with master discovery
  was introduced [4].
* old connection pool (MeshConnection) with round-robin failover was
  deprecated [4].

These changes together with GitHub stars update are introduced with this
patch.

1. https://github.com/tarantool/tarantool-python/releases/tag/0.8.0
2. tarantool/tarantool-python#211
3. tarantool/tarantool-python#213
4. tarantool/tarantool-python#207

* Update translation

Co-authored-by: Patience Daur <[email protected]>
DifferentialOrange added a commit that referenced this pull request Sep 21, 2022
DifferentialOrange added a commit that referenced this pull request Sep 21, 2022
DifferentialOrange added a commit that referenced this pull request Sep 22, 2022
DifferentialOrange added a commit that referenced this pull request Sep 26, 2022
DifferentialOrange added a commit that referenced this pull request Sep 26, 2022
DifferentialOrange added a commit that referenced this pull request Oct 5, 2022
DifferentialOrange added a commit that referenced this pull request Oct 5, 2022
DifferentialOrange added a commit that referenced this pull request Oct 6, 2022
DifferentialOrange added a commit that referenced this pull request Oct 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unpack for binary data
3 participants