Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speeding up the matching process #25

Closed
BZ-BowenZhang opened this issue May 3, 2024 · 1 comment
Closed

Speeding up the matching process #25

BZ-BowenZhang opened this issue May 3, 2024 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@BZ-BowenZhang
Copy link
Collaborator

The current matching process runs slowly because Python's for loop is inefficient. Thus, we need to consider how to speed up the matching process to deal with the bigger dataset.

Tools

  1. Using the multithreaded processing package to speed up (e.g. multiprocess)
  2. Partly rewrite the functions to reduce the use of for loops.
@sgreenbury
Copy link
Collaborator

As discussed, we'll revisit optimisation after updating to an alternative matching approach in #13 and continue to work with a sample of 15000 of the SPC population for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

3 participants