Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Snowflake IDs for nodes and edges #202

Merged
merged 6 commits into from
Nov 8, 2024
Merged

Use Snowflake IDs for nodes and edges #202

merged 6 commits into from
Nov 8, 2024

Conversation

cmutel
Copy link
Member

@cmutel cmutel commented Oct 31, 2024

Needed for distributed object creation without conflicting id values.

@cmutel cmutel marked this pull request as draft October 31, 2024 10:40
@cmutel cmutel requested a review from jsvgoncalves October 31, 2024 10:40
@cmutel
Copy link
Member Author

cmutel commented Oct 31, 2024

Merging blocked by brightway-lca/matrix_utils#22

This is now fixed.

Copy link
Member

@jsvgoncalves jsvgoncalves left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hold until #174 is merged.

bw2data/backends/base.py Outdated Show resolved Hide resolved
Comment on lines 8 to 21
class SnowflakeIDBaseClass(Model):
id = IntegerField(primary_key=True)

def save(self, **kwargs):
if self.id is None:
# If the primary key column data is already present (even if the object doesn't exist in
# the database), peewee will make an `UPDATE` query. This will have no effect if there
# isn't a matching row. Need for force an `INSERT` query instead as we generate the ids
# ourselves.
# https://docs.peewee-orm.com/en/latest/peewee/models.html#id4
self.id = next(snowflake_id_generator)
kwargs['force_insert'] = True
super().save(**kwargs)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also conflicting with #174; will need to be rebased after that one is merged. We can then make SignaledDataset inherit from this one and up goes the super.

Copy link
Member Author

@cmutel cmutel Oct 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can then make SignaledDataset

exatamente

bw2data/backends/schema.py Outdated Show resolved Hide resolved
bw2data/backends/utils.py Outdated Show resolved Hide resolved
bw2data/backends/proxies.py Outdated Show resolved Hide resolved
bw2data/backends/proxies.py Outdated Show resolved Hide resolved
@jsvgoncalves
Copy link
Member

jsvgoncalves commented Oct 31, 2024

After #174 is merged:

  • Fix schemas inheritance
  • Adjust _dict_as_…
  • Add snowflake implementation to revision ids

@cmutel
Copy link
Member Author

cmutel commented Oct 31, 2024

Adjust _dict_as_

This will go away as we will explicitly map functions instead of using string interpolation, see more-event-types branch.

cmutel and others added 2 commits November 8, 2024 10:38
Update compatibility.py

Remove debugging print statements

Clarify some comments

Improve error message

Cleanup error messages

Co-Authored-By: João Gonçalves <[email protected]>
@cmutel cmutel requested a review from jsvgoncalves November 8, 2024 10:43
@jsvgoncalves jsvgoncalves marked this pull request as ready for review November 8, 2024 12:25
@jsvgoncalves jsvgoncalves merged commit 9ddc291 into main Nov 8, 2024
@jsvgoncalves jsvgoncalves deleted the snowflake-ids branch November 8, 2024 12:26
jsvgoncalves added a commit that referenced this pull request Nov 8, 2024
Use Snowflake IDs for nodes and edges
jsvgoncalves added a commit that referenced this pull request Nov 8, 2024
Use Snowflake IDs for nodes and edges
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants