Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Generic primary key implementation #122

Merged
merged 12 commits into from
Jun 11, 2019
Merged

[WIP] Generic primary key implementation #122

merged 12 commits into from
Jun 11, 2019

Conversation

abondar
Copy link
Member

@abondar abondar commented Apr 3, 2019

Closes #36

Hey @grigi
I wrote down this implementation and, surprisingly, it's even works (on sqlite).
I'll try to do cleanup on it in following days and write some more tests, but could you review it by then?
This PR carries many changes and I am afraid there could be some places which I could have done better, but missed them in all this work.

I hope you with fresh view will be able to propose some changes for the better here

@abondar abondar requested a review from grigi April 3, 2019 15:16
@grigi
Copy link
Member

grigi commented Apr 3, 2019

Wow, this PR does a lot. Adds new datatypes even, and pg-specific optimisations, and makes uuid PK-able :-)
Can you also make CharField PK-able too?

@abondar
Copy link
Member Author

abondar commented Apr 3, 2019

Well, it's already should be pk-able, like any hashable value field. It's just that I was already beaten on last line of code and decided to write tests for other field types later

Copy link
Member

@grigi grigi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking really good 👍
Some minor comments, and then please add tests for CharField at least, and update documentation re pk handling.

tortoise/backends/sqlite/client.py Outdated Show resolved Hide resolved
"""

def __init__(self, *args, **kwargs):
super().__init__(type=UUID, *args, **kwargs)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should generated not also be auto-set when pk=True?

@grigi
Copy link
Member

grigi commented May 1, 2019

@abondar Do you need help in completing this? It has been sitting around for four weeks now?

@abondar
Copy link
Member Author

abondar commented May 1, 2019

Hi, sorry for dragging this for so long, couldn't find time to finish it and now I am on vacation until middle of may.
If you have courage and time I would be happy if you will finish it, otherwise I am going to finish it after coming back

@grigi
Copy link
Member

grigi commented May 1, 2019

Cool, enjoy your vacation 😄
I'll have a crack at some of it, partly as I want to understand everything here 👍

@grigi grigi force-pushed the generic_primary_key branch from 99685a4 to 1c838db Compare May 2, 2019 20:11
@grigi
Copy link
Member

grigi commented May 2, 2019

@abondar I rebased on latest master, fixed the linting/typing related issues, fixed the M2M table creation to use the right FK key type, and fixed postgres to work again.

I see there is a new attribute fetch_inserted that seems like it should control wether we fetch the inserted values, but I wonder why this is an attribute, as we should do the most optimal variety of SQL for insert and getting generated value?

I also see you are trying to replace the field classes for postgres, so we can use the native types easier, I didn't try and implement an auto-replace yet.

And, we need quite a few more tests and lots of documentation changes.

@grigi
Copy link
Member

grigi commented May 12, 2019

Could we just finish the test cases for primary keys, and then merge this without the replacing field classes on a per-db basis? This PR does enough and I feel the field refactoring should be a separate issue? Because it is quite large as well.

@grigi
Copy link
Member

grigi commented May 19, 2019

Turns out both MySQL and SQLite handles ROWID primary key tables and non-ROWID tables very differently.
Also the current implementation will force a ROWID on those tables, and then always collect it afterwards.
This is OK, but causes problems for PostreSQL because it can generate other types on the DB, e.g. UUID. What makes it better right now is that we don't support those cases.

This is one of the many things that is going to complicate the field refactoring #97

…update()/delete() to convert types to db type
@grigi
Copy link
Member

grigi commented May 19, 2019

@abondar Please review my changes.

I removed the custom PostreSQL uuid field type, as until we have parametrized queries for everything, we are likely to run into escaping issues when the to_db_value doesn't treat it as a string. PostgreSQL seems smart enough to parse UUIDs so for now it is a working limitation.

Also, have not updated the documentation.

From a code POV I'm happy with what we have here now.

Also, can we discuss what is to happen for v0.12.0 ?

@abondar
Copy link
Member Author

abondar commented May 21, 2019

Thank you!
Changes looks all good to me, so I think the only concern left is documentation updates.
If you don't have opportunity to write it, I think I'll try to manage some time on next week for it.

Regarding 0.12.0 I think this PR is big enough to be sole member of it.

@grigi
Copy link
Member

grigi commented May 26, 2019

?? That last commit broke stuff spectacularly in Travis, but I don't know why. as it doesn't happen locally.
In the MySQL test runs it is emitting PG sql, so I suspect some sort of resource leak?

@grigi
Copy link
Member

grigi commented Jun 7, 2019

Ok, that looks much better.
Time has been very short on my side, but it appears to be getting a bit better.

@grigi
Copy link
Member

grigi commented Jun 9, 2019

@abondar Please vet the docs

Copy link
Member Author

@abondar abondar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from comment, it all seems good to go for me


checksum = fields.CharField(pk=True)

guid = fields.UUIDField(pk=True)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can also mention, that filtering by .filter(pk__in=...) or whatever other filters is also supported

@grigi grigi merged commit 97b96cb into master Jun 11, 2019
@grigi grigi deleted the generic_primary_key branch August 24, 2019 05:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Generic PK field that references real PK field
2 participants