Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow custom cleaners #23

Closed
wants to merge 19 commits into from

Conversation

nealio82
Copy link
Contributor

Our use-case is that we need to register custom cleaners into the migration. I've refactored (as much as is necessary to achieve my goal, although there is a lot more that could / should be done) to decouple sampling from cleaning and writing, and now use the migrator as a controller / manager to orchestrate the process. Decoupling has allowed me to remove the CleanAll and CleanMatched classes - you can now just use All and Matched and pass through a cleanfields directive in your stanza.

I've removed a lot of the public methods from classes' APIs, and preferred injection over setters. I've loosely encapsulated some of the config into objects, but there is more that can be done there. I think that if all the config is encapsulated and typed throughout, then there will be some pretty obvious low-hanging fruit to make some immediate architectural improvements.

I think that with a bit more work either the App or Migrator class will become redundant. That's beyond the scope of this PR - i might do that in a future PR. I've added a bunch of tests. The Migrator and App classes are pretty hard to test right now, so this suggests that there is still a lot of work to be done in there...

This PR depends heavily on the changes I've introduced in #19, #20, and #21 (as i've branched from my local master which already incorporates that work).

$this->migrateTableTriggers($setName, $tableCollection);
}

public function registerCustomCleaner(FieldCleaner $cleaner, string $alias): void
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ideally we'd be able to remove this in a future PR. i have a suspicion that some extra work on refactoring the Migrator and App classes (and adding a well-designed Config object) will show us a better path

$destConnection->exec('DROP TABLE IF EXISTS ' . $sourceConnection->quoteIdentifier($table));

$driverName = $sourceConnection->getDriver()->getName();
if ($driverName === 'pdo_mysql') {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in a future PR i think i'd like to add a DatabaseDriver interface, with separate concrete classes for MySQL and SQLite. This would make supporting other db drivers (such as Postgres) far easier

$this->destination->insert($tableName, $row);
}

public function postWrite(): void
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't like this method being here, but until the Migrator is refactored a bit more I think it's a necessary evil

@nealio82
Copy link
Contributor Author

nealio82 commented Dec 31, 2020

i think the unit test failure in https://travis-ci.org/github/MapleSyrupGroup/dbsampler/builds/752264598 will go away once we're building on current versions of PHP (certainly doesn't throw on my local 7.4)

@nealio82
Copy link
Contributor Author

nealio82 commented Jan 8, 2021

Once #20 is closed I'll update this PR with the up-to-date master

@michaeljoseph
Copy link
Contributor

🔔 @nealio82

@nealio82 nealio82 closed this Jan 13, 2023
@nealio82 nealio82 deleted the allow-custom-cleaners branch January 13, 2023 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants