Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ZEBRA-1139] Merge OG sqlalchemy-continuum updates into our version #19

Open
wants to merge 106 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
106 commits
Select commit Hold shift + click to select a range
8ac917a
Failing tests for entities living in other schema
vault Mar 21, 2016
e2c7c5e
transaction uses pkey column for user_id foreign key
vault Feb 27, 2016
4f5ed91
table_builder copies schema of versioned table
vault Mar 1, 2016
4b656c6
Fix many-to-many versioning in separate schemas
vault Mar 18, 2016
95a7a65
Fixed `changeset` method for multi-flush transactions
oinopion Jun 20, 2016
b8fa575
little grammar fix
fpruitt Aug 24, 2017
bef7999
Merge pull request #168 from fpruitt/patch-1
kvesteri Aug 24, 2017
48f2491
improve memory usage on large tables
fuhrysteve Sep 4, 2017
8720552
Merge pull request #170 from fuhrysteve/vacuum_memory_enhancement
kvesteri Sep 7, 2017
7089b81
Merge pull request #132 from vault/master
kvesteri Oct 12, 2017
9cf6ebe
Bump version
kvesteri Oct 12, 2017
cdb2398
Merge branch 'fix-changeset' of https://github.com/oinopion/sqlalchem…
kvesteri Nov 5, 2017
712fa57
Merge branch 'oinopion-fix-changeset'
kvesteri Nov 5, 2017
486b6c8
Bump version
kvesteri Nov 5, 2017
8399829
don't include excluded properties
fuhrysteve Sep 4, 2017
4e222f2
add mypy & .cache to gitignore
fuhrysteve Sep 4, 2017
d540483
add test for excluded relationship
fuhrysteve Oct 11, 2017
a9ac2fb
add python 3.6, remove 3.3 to travis
fuhrysteve Nov 8, 2017
9027b5d
Update README.rst
tsantanaDH Mar 5, 2018
7181ffa
Update intro.rst
tsantanaDH Mar 5, 2018
f036ffa
Merge pull request #183 from tsantanaDH/patch-1
kvesteri Mar 7, 2018
5e00c0d
Merge pull request #169 from fuhrysteve/dont_include_excluded_relatio…
kvesteri Mar 7, 2018
c4aad39
Bump version
kvesteri Mar 7, 2018
f98469d
Adding a unittest, reproducing #166
May 16, 2018
acef01b
track cloned connections
Aug 15, 2017
8658a89
Merge pull request #167 from netcriptus/hotfix/track-cloned-connections
kvesteri Jun 3, 2018
d4ed900
Bump version
kvesteri Jun 3, 2018
79b9876
Reraise original exception instead of a new one
fuhrysteve Jun 7, 2018
5a3fbcf
Merge pull request #189 from fuhrysteve/reraise_original_exception
kvesteri Jun 12, 2018
8496cdd
Test for leaked connections on external rollback
vault Jul 26, 2018
3a5b967
Check connection.closed before connection.connection
vault Jul 26, 2018
4ded9b3
Invalidate units_of_work on Engine rollback event
vault Jul 26, 2018
345be03
Merge pull request #196 from vault/fix-connections-leaking
kvesteri Jul 30, 2018
cedad54
Bump version
kvesteri Jul 30, 2018
c7d4580
Working example in Readme
itdependsnetworks Sep 29, 2018
472513f
Assume Python 3 and write string without u prefix
EdwardBetts Nov 17, 2018
eb65894
Merge pull request #207 from EdwardBetts/patch-1
kvesteri Nov 18, 2018
1291a9a
Fix typo in docs
lyndsysimon Jan 8, 2019
a49ddcc
Merge pull request #208 from lyndsysimon/patch-1
kvesteri Jan 9, 2019
fb6b1e9
Fix trigger creation during alembic migrations
lyndsysimon Jan 11, 2019
3315913
Merge pull request #209 from lyndsysimon/alembic
kvesteri Jan 13, 2019
77c1719
Bump version
kvesteri Jan 13, 2019
007a0cc
Removed non-table columns from the restore operation
mauler Feb 26, 2019
0cb3e6b
Added is_table_column function
mauler Feb 26, 2019
c6b7fda
Merge pull request #213 from mauler/fix/issue-197
kvesteri Feb 27, 2019
f549c40
Bump version
kvesteri Feb 27, 2019
0cd9584
Add SA 1.3 support
kvesteri Mar 18, 2019
e2a6718
Merge pull request #220 from kvesteri/feature/sa-1.3
kvesteri Mar 19, 2019
b49936a
Revert fb6b1e970e
kvesteri Feb 27, 2019
978b694
Merge pull request #214 from kvesteri/hotfix/pg-native-test
kvesteri Mar 19, 2019
239a341
Update changes
kvesteri Mar 19, 2019
fcb1829
Issue #226
yetem Aug 7, 2019
591f9f0
test against python3.7
rooterkyberian Oct 16, 2018
fe8b040
add mysql to travis explicitly
rooterkyberian Sep 9, 2019
06ce8bc
added postgresql as service to travis.yml
rooterkyberian Sep 9, 2019
17eb104
Merge pull request #203 from rooterkyberian/py37
kvesteri Sep 19, 2019
8902be7
Add a link to the SQLAlchemy history_meta example
killthekitten Nov 7, 2019
1909e32
Merge pull request #230 from killthekitten/patch-1
kvesteri Nov 8, 2019
5ab6fbb
Fix a link formatting from markdown to rst
killthekitten Nov 8, 2019
2d5d235
Merge pull request #231 from killthekitten/patch-2
kvesteri Nov 8, 2019
36cd474
#128 Update docs regarding `alembic-migrations`
Dec 12, 2019
ce18c2e
Merge pull request #232 from shnela/fix/update-alembic-migrations-doc
kvesteri Dec 12, 2019
2c4d795
Add test to expose bug on unrelated change with m2m
Andrew-Dickinson Apr 17, 2020
f7e38b1
Add fix to m2m unrelated change bug
Andrew-Dickinson Apr 17, 2020
8c0c306
Fix order of old-new values in changeset example for create
vhermecz Apr 28, 2020
db73463
Merge pull request #243 from vhermecz/patch-1
kvesteri Apr 28, 2020
197470f
Fix the login/logout helpers in the FlaskPlugin tests
killthekitten May 2, 2020
26e23e5
Merge pull request #244 from killthekitten/fix-flask-login-tests
kvesteri May 3, 2020
f2ee5b6
Fix #245: create column aliases in the version model
killthekitten May 6, 2020
c2df540
Add explicit "pseudo-backref" relationships for version/parent
lgedgar Mar 14, 2020
af10632
Merge pull request #240 from lgedgar/fix-viewonly-backref-warning
kvesteri May 10, 2020
645d019
Bump version
kvesteri May 10, 2020
6f993bf
Merge pull request #202 from itdependsnetworks/patch-1
kvesteri May 10, 2020
80aeffc
Merge pull request #242 from Andrew-Dickinson/fix-unrelated-update
kvesteri May 10, 2020
69b6aa2
Merge pull request #227 from yetem/patch-1
kvesteri May 10, 2020
54af8b2
Update changes
kvesteri May 10, 2020
b5c732f
Merge pull request #246 from killthekitten/fix-245-create-column-aliases
kvesteri May 24, 2020
8a9b357
bump version
kvesteri May 24, 2020
aa42365
Fix class registry lookup for SQLAlchemy >= 1.4
marksteward Jan 1, 2022
f12a7c3
Prevent re-entry of after_configured
marksteward Jan 1, 2022
693cbca
Only pass classes to aliased()
marksteward Jan 1, 2022
1d1251b
Fix deprecated useexisting
marksteward Jan 1, 2022
5e3edc9
Skip tests for deprecated order_by on sqlalchemy>=1.4
marksteward Jan 16, 2022
3f5f95f
An entity with an empty polymorphic_identity can't be loaded
marksteward Jan 16, 2022
816fe45
Add GitHub test action
marksteward Jan 16, 2022
3f69373
Avoid sqlalchemy-i18n==1.1.0
marksteward Jan 16, 2022
ac5d180
Bump PyMySQL in test to support UTF-8
marksteward Jan 17, 2022
e03481c
Add DB engines to GitHub action
marksteward Jan 17, 2022
3da1c13
Update build badge
marksteward Jan 18, 2022
85f09a4
Bump version
marksteward Jan 19, 2022
4651698
Correct wording of LICENSE (fixes #224)
marksteward Jan 26, 2022
2e53b5f
Remove unused anyjson (fixes #261)
marksteward Jan 26, 2022
8629591
add test that reproduces issue 85 "TransactionBase.changed_entities f…
TomGoBravo Feb 7, 2022
6bfcf3d
use hasattr instead of __table__.columns, fixes tests/plugins/test_tr…
TomGoBravo Feb 7, 2022
ec08cf4
fix for unit_of_work.py:263: SAWarning: implicitly coercing SELECT ob…
TomGoBravo Feb 8, 2022
f96c9a1
fallback to as_scalar to continue working in SQLAlchemy < 1.4
TomGoBravo Feb 8, 2022
6c16fcd
Merge pull request #269 from TomGoBravo/master
marksteward Feb 10, 2022
7eda527
FlaskPlugin: use `get_id()` instead of `id` attr
jirikuncar Nov 8, 2016
66b99eb
t Test that recreates bug.
Jan 28, 2019
46886b0
B Activity plugin works with PKs named other than id. closes #210
Jan 28, 2019
f3353a7
t Test adjusted for python 2.7
Jan 28, 2019
745dc9e
NoChangesColumn -> NoChangesAttribute
TomGoBravo Feb 11, 2022
f1b0faa
Merge pull request #268 from TomGoBravo/fix-issue-85
marksteward Feb 11, 2022
92eeb1e
Allow to disable mod plugin in sync_triggers (#273)
nanvel Apr 18, 2022
e877d2c
Merge remote-tracking branch 'upstream/master' into emily__ZEBRA-1139…
emilyw26 Apr 18, 2022
ffc3eec
Bump version to 1.5.0
emilyw26 Apr 18, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
name: "Test"

on:
push:
paths-ignore:
- "docs/**"
pull_request:
paths-ignore:
- "docs/**"
schedule:
- cron: '40 1 * * 3'


jobs:
test:
name: test-python${{ matrix.python-version }}-sa${{ matrix.sqlalchemy-version }}-${{ matrix.db-engine }}
strategy:
matrix:
python-version:
# - "2.7"
# - "3.4"
# - "3.5"
# - "3.6"
# - "3.7"
- "3.8"
# - "3.9"
# - "3.10"
# - "pypy-3.7"
sqlalchemy-version:
- "<1.4"
- ">=1.4"
db-engine:
- sqlite
- postgres
- postgres-native
- mysql
runs-on: ubuntu-latest
services:
mysql:
image: mysql
ports:
- 3306:3306
env:
MYSQL_DATABASE: sqlalchemy_continuum_test
MYSQL_ALLOW_EMPTY_PASSWORD: yes
options: >-
--health-cmd "mysqladmin ping"
--health-interval 5s
--health-timeout 2s
--health-retries 3
postgres:
image: postgres
ports:
- 5432:5432
env:
POSTGRES_PASSWORD: postgres
POSTGRES_DB: sqlalchemy_continuum_test
options: >-
--health-cmd pg_isready
--health-interval 5s
--health-timeout 2s
--health-retries 3
steps:
- uses: actions/checkout@v1
- name: Install sqlalchemy
run: pip3 install 'sqlalchemy${{ matrix.sqlalchemy-version }}'
- name: Build
run: pip3 install -e '.[test]'
- name: Run tests
run: pytest
env:
DB: ${{ matrix.db-engine }}

6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,9 @@ nosetests.xml
.mr.developer.cfg
.project
.pydevproject

# mypy
.mypy_cache/

# Unit test / coverage reports
.cache
11 changes: 8 additions & 3 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
addons:
postgresql: 9.3
services:
- mysql
- postgresql

dist: xenial
sudo: true

env:
- DB=mysql
Expand All @@ -15,9 +19,10 @@ before_script:
language: python
python:
- 2.7
- 3.3
- 3.4
- 3.5
- 3.6
- 3.7
install:
- pip install -e ".[test]"
script:
Expand Down
63 changes: 62 additions & 1 deletion CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,71 @@ Changelog
Here you can see the full list of changes between each SQLAlchemy-Continuum release.


1.3.12 (2019-03-18)
1.3.12 (2022-01-18)
^^^^^^^^^^^^^^^^^^^

- Support SA 1.4

1.3.11 (2020-05-24)
^^^^^^^^^^^^^^^^^^^

- Made ModelBuilder create column aliases in version models (#246, courtesy of killthekitten)


1.3.10 (2020-05-10)
^^^^^^^^^^^^^^^^^^^

- Added explicit "pseudo-backref" relationships for version/parent (#240, courtesy of lgedgar)
- Fixed m2m Bug when an unrelated change is made to a model (#242, courtesy of Andrew-Dickinson)


1.3.9 (2019-03-19)
^^^^^^^^^^^^^^^^^^

- Added SA 1.3 support
- Reverted trigger creation from 1.3.7


1.3.8 (2019-02-27)
^^^^^^^^^^^^^^^^^^

- Fixed revert to ignore non-columns (#197, courtesy of mauler)


1.3.7 (2019-01-13)
^^^^^^^^^^^^^^^^^^

- Fix trigger creation during alembic migrations (#209, courtesy of lyndsysimon)


1.3.6 (2018-07-30)
^^^^^^^^^^^^^^^^^^

- Fixed ResourceClosedErrors from connections leaking when using an external transaction (#196, courtesy of vault)


1.3.5 (2018-06-03)
^^^^^^^^^^^^^^^^^^

- Track cloned connections (#167, courtesy of netcriptus)


1.3.4 (2018-03-07)
^^^^^^^^^^^^^^^^^^

- Exclude many-to-many properties from versioning if they are added in exclude parameter (#169, courtesy of fuhrysteve)


1.3.3 (2017-11-05)
^^^^^^^^^^^^^^^^^^

- Fixed changeset when updating object in same transaction as inserting it (#141, courtesy of oinopion)


1.3.2 (2017-10-12)
^^^^^^^^^^^^^^^^^^

- Fixed multiple schema handling (#132, courtesy of vault)


1.3.1 (2017-06-28)
Expand Down
5 changes: 3 additions & 2 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@ modification, are permitted provided that the following conditions are met:
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

* The names of the contributors may not be used to endorse or promote products
derived from this software without specific prior written permission.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
Expand Down
2 changes: 1 addition & 1 deletion benchmark.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ def test_versioning(

make_versioned(options=options)

dns = 'postgres://postgres@localhost/sqlalchemy_continuum_test'
dns = 'postgresql://postgres:postgres@localhost/sqlalchemy_continuum_test'
versioning_manager.plugins = plugins
versioning_manager.transaction_cls = transaction_cls
versioning_manager.user_cls = user_cls
Expand Down
7 changes: 6 additions & 1 deletion docs/alembic.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
Alembic migrations
==================

Each time you make changes to database structure you should also change the associated history tables. When you make changes to your models SQLAlchemy-Continuum automatically alters the history model definitions, hence you can use `alembic revision --autogenerate` just like before. You just need to make sure `make_versioned` function gets called before alembic gathers all your models.
Each time you make changes to database structure you should also change the associated history tables. When you make changes to your models SQLAlchemy-Continuum automatically alters the history model definitions, hence you can use `alembic revision --autogenerate` just like before. You just need to make sure `make_versioned` function gets called before alembic gathers all your models and `configure_mappers` is called afterwards.

Pay close attention when dropping or moving data from parent tables and reflecting these changes to history tables.

Troubleshooting
###############

If alembic didn't detect any changes or generates reversed migration (tries to remove `*_version` tables from database instead of creating), make sure that `configure_mappers` was called by alembic command.
4 changes: 2 additions & 2 deletions docs/intro.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Introduction
Why?
^^^^

SQLAlchemy already has versioning extension. This extension however is very limited. It does not support versioning entire transactions.
SQLAlchemy `already has a versioning extension <https://docs.sqlalchemy.org/en/13/orm/examples.html#module-examples.versioned_history>`_. This extension however is very limited. It does not support versioning entire transactions.

Hibernate for Java has Envers, which had nice features but lacks a nice API. Ruby on Rails has papertrail_, which has very nice API but lacks the efficiency and feature set of Envers.

Expand Down Expand Up @@ -54,7 +54,7 @@ In order to make your models versioned you need two things:
from sqlalchemy_continuum import make_versioned


make_versioned()
make_versioned(user_cls=None)


class Article(Base):
Expand Down
12 changes: 8 additions & 4 deletions docs/native_versioning.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,13 @@ Schema migrations
When making schema migrations (for example adding new columns to version tables) you need to remember to call sync_trigger in order to keep the version trigger up-to-date.
::

from sqlalchemy_continuum import versioning_manager # or import your custom one, if you have one
sync_trigger(conn,
'article_version',
versioning_manager=versioning_manager)
from sqlalchemy_continuum.dialects.postgresql import sync_trigger


sync_trigger(conn, 'article_version')

If you don't use `PropertyModTrackerPlugin`, then you have to disable it:

::

sync_trigger(conn, 'article_version', use_property_mod_tracking=False)
2 changes: 1 addition & 1 deletion docs/plugins.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Using plugins

::

from sqlalchemy.continuum.plugins import PropertyModTrackerPlugin
from sqlalchemy_continuum.plugins import PropertyModTrackerPlugin


versioning_manager.plugins.append(PropertyModTrackerPlugin())
Expand Down
2 changes: 1 addition & 1 deletion docs/version_objects.rst
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ you can easily check the changeset of given object in current transaction.

article = Article(name=u'Some article')
changeset(article)
# {'name': [u'Some article', None]}
# {'name': [None, u'Some article']}


Version relationships
Expand Down
8 changes: 4 additions & 4 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,14 @@ def get_version():
'pytest>=2.3.5',
'flexmock>=0.9.7',
'psycopg2>=2.4.6',
'PyMySQL==0.6.1',
'PyMySQL>=0.8.0',
'six>=1.4.0'
],
'anyjson': ['anyjson>=0.3.3'],
'flask': ['Flask>=0.9'],
'flask-login': ['Flask-Login>=0.2.9'],
'flask-sqlalchemy': ['Flask-SQLAlchemy>=1.0'],
'flexmock': ['flexmock>=0.9.7'],
'i18n': ['SQLAlchemy-i18n>=0.8.4'],
'i18n': ['SQLAlchemy-i18n>=0.8.4,!=1.1.0'],
}


Expand Down Expand Up @@ -77,9 +76,10 @@ def get_version():
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Topic :: Internet :: WWW/HTTP :: Dynamic Content',
'Topic :: Software Development :: Libraries :: Python Modules'
]
Expand Down
26 changes: 25 additions & 1 deletion sqlalchemy_continuum/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
)


__version__ = '1.4.3'
__version__ = '1.5.0'


versioning_manager = VersioningManager()
Expand Down Expand Up @@ -72,6 +72,18 @@ def make_versioned(
manager.track_association_operations
)

sa.event.listen(
sa.engine.Engine,
'rollback',
manager.clear_connection
)

sa.event.listen(
sa.engine.Engine,
'set_connection_execution_options',
manager.track_cloned_connections
)


def remove_versioning(
mapper=sa.orm.mapper,
Expand All @@ -98,3 +110,15 @@ def remove_versioning(
'before_cursor_execute',
manager.track_association_operations
)

sa.event.remove(
sa.engine.Engine,
'rollback',
manager.clear_connection
)

sa.event.remove(
sa.engine.Engine,
'set_connection_execution_options',
manager.track_cloned_connections
)
Loading