Rationale for our recommended unsupervised training approach

[RobinL]

[EDIT: This content is now available in the main docs here]

[RobinL]

Correct!

[medwar99]

Thanks Robin, this really helps to explain the approach Splink presents to parameter estimations.

I get that the m values aren't the be-all of the model as a whole, but would it be worthwhile extrapolating further on section 3 regarding the choices we make for blocking rules used in training the m values? Their values relative to other comparisons can be impactful, and estimating m is a sticking point in other discussions.

At the moment, in the linked page, we have:

To speed this up, we can use a trick. By blocking on some columns (e.g. first name and surname), we now restrict our record comparisons to a subset with a far higher proportion of matches than random comparisons.

In another portion of the documentation on estimation of m, it's stated:

Each estimation pass requires the user to configure an estimation blocking rule to reduce the number of record comparisons generated to a manageable level.
...
Recall we are trying to estimate the m values of the model, i.e. proportion of records falling into each ComparisonLevel amongst truly matching records.

Both of these suggest that the main driver is to keep the number of comparisons low for performance, but should there be primary consideration for the proportion of positive matches in that block (and then secondary in reducing the block size)?

Likewise, in the docstring for estimate_u_using_random_sampling:

The blocking rule provided is used to generate pairwise record comparisons. Usually, this should be a blocking rule that results in a dataframe where matches are between about 1% and 99% of the blocked comparisons.

This suggests more that the (relative) proportion of matches in the sample wants to be higher, but doesn't really indicate which end (if any) of that range is preferable.

I feel the new page could be expanded to detail the blocking approach a bit more (but it might be the ML side of my brain driving this feeling). Given we are trying to estimate m as a measure of data quality and more specifically as probability of scenario given records are a match, are we looking for:

• Rules which are so tight that they provide a very small sample size of almost entirely positive matches, save for those that differ by fields that we haven't included in the rule (and can then estimate m for). To achieve this, could we start with deterministic rules on which we relax a single field during each pass?
• Rules which are reasonably tight, which raise the proportion of matches in a sample slightly but not to the point above. This may estimate m values for more fields, but likely introduces relatively more non-matches.
• Rules which are loose, but have a low number of samples, under the hope that the smaller sample size mean the relative proportions are okay.
• Doesn't really matter as long as matches proportion goes up, as the u parameter is so dominant the relative match-weights are already locked in.

I feel that the choice can have an impact on the model, especially when we view the parameter_estimation_comparisons_chart and see variance across the different training runs. Although I guess this may also be poor estimation of u starting the model off from a bad point.