Bug: `optimize_sa` outputs wrong starting layout #64

Glitchy-Tozier · 2023-07-09T09:46:31Z

Not sure what to do about this, but layout.resolve_modifiers(&k) produces (from time to time) different results.

I discovered this when I noticed that layouts weren't always evaluated the same way. I dug deeper and discovered that even the total combined weight after n-gram-expansion differed each time OnDemandBigramMapper (and the other n-gram mappers) are used.

There may be other issues at play as well, but what certainly is happening is that layout.resolve_modifiers(&k) produces inconsistent results. You can test it this way:

Anywhere in the Bigram-mapper, for example at the start of process_hold_layers, paste this code:

for i in 0..50 {
    let (_, m) = layout.resolve_modifiers(&i);
    println!("{}:  {:?}", i, layout.resolve_modifiers(&i));
    let good_count = m.layerkey_indices().len();

    for _ in 0..10000 {
        let (k, mods) = layout.resolve_modifiers(&i);

        if mods.layerkey_indices().len() != good_count {
            println!("Wrong: {:?}", (k, mods));
        }
    }
}

Then run the optimizer.
You'll see that within each call of BigramMapper, the function behaves consistently. However, between different calls of BigramMapper, the printed list looks differently.

Here's my test-results. You can use tools like https://text-compare.com/ to compare the different calls. At the end of each call, I've attached the total no. of modifiers found while looping through all ngrams in process_hold_layers().

0:  (0, Hold([]))
1:  (0, Hold([211]))
2:  (0, Hold([212]))
3:  (3, Hold([]))
4:  (3, Hold([211]))
5:  (3, Hold([212]))
6:  (6, Hold([]))
7:  (6, Hold([211]))
8:  (6, Hold([212]))
9:  (9, Hold([]))
10:  (9, Hold([211]))
11:  (9, Hold([212]))
12:  (12, Hold([]))
13:  (12, Hold([211]))
14:  (12, Hold([212]))
15:  (15, Hold([]))
16:  (15, Hold([211]))
17:  (15, Hold([212]))
18:  (18, Hold([]))
19:  (18, Hold([210]))
20:  (18, Hold([213]))
21:  (21, Hold([]))
22:  (21, Hold([210]))
23:  (21, Hold([213]))
24:  (21, Lock([214, 215]))
25:  (25, Hold([]))
26:  (25, Hold([210]))
27:  (25, Hold([213]))
28:  (25, Lock([214, 215]))
29:  (29, Hold([]))
30:  (29, Hold([210]))
31:  (29, Hold([213]))
32:  (29, Lock([214, 215]))
33:  (33, Hold([]))
34:  (33, Hold([210]))
35:  (35, Hold([]))
36:  (35, Hold([210]))
37:  (37, Hold([]))
38:  (38, Hold([]))
39:  (39, Hold([]))
40:  (39, Hold([211]))
41:  (41, Hold([]))
42:  (41, Hold([211]))
43:  (41, Hold([212]))
44:  (41, Lock([216, 216]))
45:  (41, Hold([217]))
46:  (46, Hold([]))
47:  (46, Hold([211]))
48:  (46, Hold([212]))
49:  (46, Lock([216, 216]))
16327

0:  (0, Hold([]))
1:  (0, Hold([210]))
2:  (0, Hold([211]))
3:  (3, Hold([]))
4:  (3, Hold([210]))
5:  (3, Hold([211]))
6:  (6, Hold([]))
7:  (6, Hold([210]))
8:  (6, Hold([211]))
9:  (9, Hold([]))
10:  (9, Hold([210]))
11:  (9, Hold([211]))
12:  (12, Hold([]))
13:  (12, Hold([210]))
14:  (12, Hold([211]))
15:  (15, Hold([]))
16:  (15, Hold([210]))
17:  (15, Hold([211]))
18:  (18, Hold([]))
19:  (18, Hold([209]))
20:  (18, Hold([212]))
21:  (21, Hold([]))
22:  (21, Hold([209]))
23:  (21, Hold([212]))
24:  (21, Lock([213, 214]))
25:  (25, Hold([]))
26:  (25, Hold([209]))
27:  (25, Hold([212]))
28:  (25, Lock([213, 214]))
29:  (29, Hold([]))
30:  (29, Hold([209]))
31:  (29, Hold([212]))
32:  (29, Lock([213, 214]))
33:  (33, Hold([]))
34:  (33, Hold([209]))
35:  (35, Hold([]))
36:  (35, Hold([209]))
37:  (35, Hold([212]))
38:  (35, Lock([213, 214]))
39:  (39, Hold([]))
40:  (40, Hold([]))
41:  (41, Hold([]))
42:  (41, Hold([210]))
43:  (43, Hold([]))
44:  (43, Hold([210]))
45:  (43, Hold([211]))
46:  (43, Lock([215, 215]))
47:  (43, Hold([216]))
48:  (48, Hold([]))
49:  (48, Hold([210]))
16312

0:  (0, Hold([]))
1:  (0, Hold([211]))
2:  (0, Hold([212]))
3:  (3, Hold([]))
4:  (3, Hold([211]))
5:  (3, Hold([212]))
6:  (6, Hold([]))
7:  (6, Hold([211]))
8:  (6, Hold([212]))
9:  (9, Hold([]))
10:  (9, Hold([211]))
11:  (9, Hold([212]))
12:  (12, Hold([]))
13:  (12, Hold([211]))
14:  (12, Hold([212]))
15:  (15, Hold([]))
16:  (15, Hold([211]))
17:  (15, Hold([212]))
18:  (18, Hold([]))
19:  (18, Hold([210]))
20:  (18, Hold([213]))
21:  (21, Hold([]))
22:  (21, Hold([210]))
23:  (21, Hold([213]))
24:  (21, Lock([214, 215]))
25:  (25, Hold([]))
26:  (25, Hold([210]))
27:  (25, Hold([213]))
28:  (25, Lock([214, 215]))
29:  (29, Hold([]))
30:  (29, Hold([210]))
31:  (29, Hold([213]))
32:  (29, Lock([214, 215]))
33:  (33, Hold([]))
34:  (33, Hold([210]))
35:  (35, Hold([]))
36:  (35, Hold([210]))
37:  (37, Hold([]))
38:  (38, Hold([]))
39:  (39, Hold([]))
40:  (39, Hold([211]))
41:  (41, Hold([]))
42:  (41, Hold([211]))
43:  (41, Hold([212]))
44:  (41, Lock([216, 216]))
45:  (41, Hold([217]))
46:  (46, Hold([]))
47:  (46, Hold([211]))
48:  (46, Hold([212]))
49:  (46, Lock([216, 216]))
16327

0:  (0, Hold([]))
1:  (0, Hold([211]))
2:  (0, Hold([212]))
3:  (3, Hold([]))
4:  (3, Hold([211]))
5:  (3, Hold([212]))
6:  (6, Hold([]))
7:  (6, Hold([211]))
8:  (6, Hold([212]))
9:  (9, Hold([]))
10:  (9, Hold([211]))
11:  (9, Hold([212]))
12:  (12, Hold([]))
13:  (12, Hold([211]))
14:  (12, Hold([212]))
15:  (15, Hold([]))
16:  (15, Hold([211]))
17:  (15, Hold([212]))
18:  (18, Hold([]))
19:  (18, Hold([210]))
20:  (18, Hold([213]))
21:  (21, Hold([]))
22:  (21, Hold([210]))
23:  (21, Hold([213]))
24:  (21, Lock([214, 215]))
25:  (25, Hold([]))
26:  (25, Hold([210]))
27:  (25, Hold([213]))
28:  (25, Lock([214, 215]))
29:  (29, Hold([]))
30:  (29, Hold([210]))
31:  (29, Hold([213]))
32:  (29, Lock([214, 215]))
33:  (33, Hold([]))
34:  (33, Hold([210]))
35:  (33, Hold([213]))
36:  (33, Lock([214, 215]))
37:  (37, Hold([]))
38:  (37, Hold([210]))
39:  (39, Hold([]))
40:  (40, Hold([]))
41:  (41, Hold([]))
42:  (41, Hold([211]))
43:  (43, Hold([]))
44:  (43, Hold([211]))
45:  (43, Hold([212]))
46:  (43, Lock([216, 216]))
47:  (43, Hold([217]))
48:  (48, Hold([]))
49:  (48, Hold([211]))
16345

0:  (0, Hold([]))
1:  (0, Hold([211]))
2:  (0, Hold([212]))
3:  (3, Hold([]))
4:  (3, Hold([211]))
5:  (3, Hold([212]))
6:  (6, Hold([]))
7:  (6, Hold([211]))
8:  (6, Hold([212]))
9:  (9, Hold([]))
10:  (9, Hold([211]))
11:  (9, Hold([212]))
12:  (12, Hold([]))
13:  (12, Hold([211]))
14:  (12, Hold([212]))
15:  (15, Hold([]))
16:  (15, Hold([211]))
17:  (15, Hold([212]))
18:  (18, Hold([]))
19:  (18, Hold([210]))
20:  (18, Hold([213]))
21:  (21, Hold([]))
22:  (21, Hold([210]))
23:  (21, Hold([213]))
24:  (21, Lock([214, 215]))
25:  (25, Hold([]))
26:  (25, Hold([210]))
27:  (25, Hold([213]))
28:  (25, Lock([214, 215]))
29:  (29, Hold([]))
30:  (29, Hold([210]))
31:  (29, Hold([213]))
32:  (29, Lock([214, 215]))
33:  (33, Hold([]))
34:  (33, Hold([210]))
35:  (35, Hold([]))
36:  (35, Hold([210]))
37:  (37, Hold([]))
38:  (38, Hold([]))
39:  (39, Hold([]))
40:  (39, Hold([211]))
41:  (41, Hold([]))
42:  (41, Hold([211]))
43:  (41, Hold([212]))
44:  (41, Lock([216, 216]))
45:  (41, Hold([217]))
46:  (46, Hold([]))
47:  (46, Hold([211]))
48:  (46, Hold([212]))
49:  (46, Lock([216, 216]))
16327

0:  (0, Hold([]))
1:  (0, Hold([211]))
2:  (0, Hold([212]))
3:  (3, Hold([]))
4:  (3, Hold([211]))
5:  (3, Hold([212]))
6:  (6, Hold([]))
7:  (6, Hold([211]))
8:  (6, Hold([212]))
9:  (9, Hold([]))
10:  (9, Hold([211]))
11:  (9, Hold([212]))
12:  (12, Hold([]))
13:  (12, Hold([211]))
14:  (12, Hold([212]))
15:  (15, Hold([]))
16:  (15, Hold([211]))
17:  (15, Hold([212]))
18:  (18, Hold([]))
19:  (18, Hold([210]))
20:  (18, Hold([213]))
21:  (21, Hold([]))
22:  (21, Hold([210]))
23:  (21, Hold([213]))
24:  (21, Lock([214, 215]))
25:  (25, Hold([]))
26:  (25, Hold([210]))
27:  (25, Hold([213]))
28:  (25, Lock([214, 215]))
29:  (29, Hold([]))
30:  (29, Hold([210]))
31:  (29, Hold([213]))
32:  (29, Lock([214, 215]))
33:  (33, Hold([]))
34:  (33, Hold([210]))
35:  (35, Hold([]))
36:  (35, Hold([210]))
37:  (37, Hold([]))
38:  (38, Hold([]))
39:  (39, Hold([]))
40:  (39, Hold([211]))
41:  (41, Hold([]))
42:  (41, Hold([211]))
43:  (41, Hold([212]))
44:  (41, Lock([216, 216]))
45:  (41, Hold([217]))
46:  (46, Hold([]))
47:  (46, Hold([211]))
48:  (46, Hold([212]))
49:  (46, Lock([216, 216]))
16327

0:  (0, Hold([]))
1:  (0, Hold([211]))
2:  (0, Hold([212]))
3:  (3, Hold([]))
4:  (3, Hold([211]))
5:  (3, Hold([212]))
6:  (6, Hold([]))
7:  (6, Hold([211]))
8:  (6, Hold([212]))
9:  (9, Hold([]))
10:  (9, Hold([211]))
11:  (9, Hold([212]))
12:  (12, Hold([]))
13:  (12, Hold([211]))
14:  (12, Hold([212]))
15:  (15, Hold([]))
16:  (15, Hold([211]))
17:  (15, Hold([212]))
18:  (18, Hold([]))
19:  (18, Hold([210]))
20:  (18, Hold([213]))
21:  (21, Hold([]))
22:  (21, Hold([210]))
23:  (21, Hold([213]))
24:  (21, Lock([214, 215]))
25:  (25, Hold([]))
26:  (25, Hold([210]))
27:  (25, Hold([213]))
28:  (25, Lock([214, 215]))
29:  (29, Hold([]))
30:  (29, Hold([210]))
31:  (29, Hold([213]))
32:  (29, Lock([214, 215]))
33:  (33, Hold([]))
34:  (33, Hold([210]))
35:  (35, Hold([]))
36:  (35, Hold([210]))
37:  (37, Hold([]))
38:  (38, Hold([]))
39:  (39, Hold([]))
40:  (39, Hold([211]))
41:  (41, Hold([]))
42:  (41, Hold([211]))
43:  (41, Hold([212]))
44:  (41, Lock([216, 216]))
45:  (41, Hold([217]))
46:  (46, Hold([]))
47:  (46, Hold([211]))
48:  (46, Hold([212]))
49:  (46, Lock([216, 216]))
16327

0:  (0, Hold([]))
1:  (0, Hold([211]))
2:  (0, Hold([212]))
3:  (3, Hold([]))
4:  (3, Hold([211]))
5:  (3, Hold([212]))
6:  (6, Hold([]))
7:  (6, Hold([211]))
8:  (6, Hold([212]))
9:  (9, Hold([]))
10:  (9, Hold([211]))
11:  (9, Hold([212]))
12:  (12, Hold([]))
13:  (12, Hold([211]))
14:  (12, Hold([212]))
15:  (15, Hold([]))
16:  (15, Hold([211]))
17:  (15, Hold([212]))
18:  (18, Hold([]))
19:  (18, Hold([210]))
20:  (18, Hold([213]))
21:  (21, Hold([]))
22:  (21, Hold([210]))
23:  (21, Hold([213]))
24:  (21, Lock([214, 215]))
25:  (25, Hold([]))
26:  (25, Hold([210]))
27:  (25, Hold([213]))
28:  (25, Lock([214, 215]))
29:  (29, Hold([]))
30:  (29, Hold([210]))
31:  (29, Hold([213]))
32:  (29, Lock([214, 215]))
33:  (33, Hold([]))
34:  (33, Hold([210]))
35:  (35, Hold([]))
36:  (35, Hold([210]))
37:  (37, Hold([]))
38:  (38, Hold([]))
39:  (39, Hold([]))
40:  (39, Hold([211]))
41:  (41, Hold([]))
42:  (41, Hold([211]))
43:  (41, Hold([212]))
44:  (41, Lock([216, 216]))
45:  (41, Hold([217]))
46:  (46, Hold([]))
47:  (46, Hold([211]))
48:  (46, Hold([212]))
49:  (46, Lock([216, 216]))
16327

The text was updated successfully, but these errors were encountered:

dariogoetz · 2023-07-09T11:15:25Z

I currently don't have the time to run the tests myself. I may be able to find some time in a couple of weeks.

I'm not sure that I understand what you are saying. When you say "then run the optimizer", do you mean that the various calls to bigram mapper happen for different layouts? In that case, it would not be surprising that the modifier expansion in the bigram mapper ends up with different total numbers of modifiers.
Example: If the two symbols in a bigram come from the same upper layer, it makes a difference if they belong to the same hand. On the same hand, the modifiers are only "held" for both symbols. On different hands, multiple key presses for the modifiers are involved. Maybe this is what you are observing.

Glitchy-Tozier · 2023-07-09T15:02:20Z

You're absolutely correct, my testing was flawed.
Evaluation might work just fine. The thing that confused me initially was that when starting a --run-forever simmulated-annealing optimization with a starting layout, that the starting-layout had different evaluations. However, now that I'm looking more closely, I can see that the optimizer sometimes shows mutated starting-layouts (see below). I'll have to check that out.

cargo build --release  && RUST_LOG=INFO ./target/release/optimize_sa -s k.o,yvgclßzhaeiudtrnsfxqäüöbpwmj --run-forever

Output (the last 8 lines are the important part):

    Finished release [optimized] target(s) in 0.16s
[2023-07-09T14:57:20Z INFO  keyboard_layout_optimizer::common] A-priori estimations from key_costs:
[2023-07-09T14:57:20Z INFO  keyboard_layout_optimizer::common] Finger loads (thumbs set to 0.00): 0.13 0.10 0.10 0.14 0.00 - 0.00 0.15 0.12 0.10 0.17
[2023-07-09T14:57:20Z INFO  keyboard_layout_optimizer::common] Row loads: Row 0: 0.05 Row 1: 0.21 Row 2: 0.48 Row 3: 0.14 Row 4: 0.12
[2023-07-09T14:57:20Z INFO  keyboard_layout_optimizer::common] Reading unigram file: '"ngrams/deu_mixed_wiki_web_0.6_eng_news_typical_wiki_web_0.4/1-grams.txt"'
[2023-07-09T14:57:20Z INFO  keyboard_layout_optimizer::common] Reading bigram file: '"ngrams/deu_mixed_wiki_web_0.6_eng_news_typical_wiki_web_0.4/2-grams.txt"'
[2023-07-09T14:57:20Z INFO  keyboard_layout_optimizer::common] Reading trigram file: '"ngrams/deu_mixed_wiki_web_0.6_eng_news_typical_wiki_web_0.4/3-grams.txt"'
[2023-07-09T14:57:20Z INFO  optimize_sa] Process   4: Starting optimization from k.o,yvgclßzhaeiudtrnsfxqäüöbpwmj
[2023-07-09T14:57:20Z INFO  optimize_sa] Process   2: Starting optimization from k.o,yvgclßzhaeiudtrnsfxqäüöbpwmj
[2023-07-09T14:57:20Z INFO  optimize_sa] Process   1: Starting optimization from k.o,yvgclßzhaeiudtrnsfxqäüöbpwmj
[2023-07-09T14:57:20Z INFO  optimize_sa] Process   5: Starting optimization from k.o,yvgclßzhaeiudtrnsfxqäüöbpwmj
[2023-07-09T14:57:20Z INFO  optimize_sa] Process   3: Starting optimization from k.o,yvgclßzhaeiudtrnsfxqäüöbpwmj
[2023-07-09T14:57:20Z INFO  optimize_sa] Process   6: Starting optimization from k.o,yvgclßzhaeiudtrnsfxqäüöbpwmj
[2023-07-09T14:57:20Z INFO  optimize_sa] Process   0: Starting optimization from k.o,yvgclßzhaeiudtrnsfxqäüöbpwmj
[2023-07-09T14:57:20Z INFO  optimize_sa] Process   7: Starting optimization from k.o,yvgclßzhaeiudtrnsfxqäüöbpwmj
[2023-07-09T14:57:20Z INFO  layout_optimization_sa::optimization] Process   0: Starting optimization with: initial_temperature: 150.00°, Parameters { init_temp: Some(150.0), key_switches: 1, stall_accepted: 5000, max_iters: 100000 }
[2023-07-09T14:57:20Z INFO  layout_optimization_sa::optimization] Process   1: Starting optimization with: initial_temperature: 150.00°, Parameters { init_temp: Some(150.0), key_switches: 1, stall_accepted: 5000, max_iters: 100000 }
[2023-07-09T14:57:20Z INFO  layout_optimization_sa::optimization] Process   4: Starting optimization with: initial_temperature: 150.00°, Parameters { init_temp: Some(150.0), key_switches: 1, stall_accepted: 5000, max_iters: 100000 }
[2023-07-09T14:57:20Z INFO  layout_optimization_sa::optimization] Process   7: Starting optimization with: initial_temperature: 150.00°, Parameters { init_temp: Some(150.0), key_switches: 1, stall_accepted: 5000, max_iters: 100000 }
[2023-07-09T14:57:20Z INFO  layout_optimization_sa::optimization] Process   5: Starting optimization with: initial_temperature: 150.00°, Parameters { init_temp: Some(150.0), key_switches: 1, stall_accepted: 5000, max_iters: 100000 }
[2023-07-09T14:57:20Z INFO  layout_optimization_sa::optimization] Process   6: Starting optimization with: initial_temperature: 150.00°, Parameters { init_temp: Some(150.0), key_switches: 1, stall_accepted: 5000, max_iters: 100000 }
[2023-07-09T14:57:20Z INFO  layout_optimization_sa::optimization] Process   2: Starting optimization with: initial_temperature: 150.00°, Parameters { init_temp: Some(150.0), key_switches: 1, stall_accepted: 5000, max_iters: 100000 }
[2023-07-09T14:57:20Z INFO  layout_optimization_sa::optimization] Process   3: Starting optimization with: initial_temperature: 150.00°, Parameters { init_temp: Some(150.0), key_switches: 1, stall_accepted: 5000, max_iters: 100000 }
[2023-07-09T14:57:21Z INFO  layout_optimization_sa::optimization] Process   5: Starting layout: k.o,yvgclßzhueiadtrnsfxqäüöbpwmj ( 398.9)
[2023-07-09T14:57:21Z INFO  layout_optimization_sa::optimization] Process   1: Starting layout: k.o,yvgclßzhaeiudtrnsfxqäüwbpömj ( 409.9)
[2023-07-09T14:57:21Z INFO  layout_optimization_sa::optimization] Process   3: Starting layout: k.o,yvgclßzhaeiudtrnsmxqäüöbpwfj ( 414.6)
[2023-07-09T14:57:21Z INFO  layout_optimization_sa::optimization] Process   6: Starting layout: k.o,yvgclßzhaeiudtrnsfxqäüöbpwmj ( 417.6)
[2023-07-09T14:57:21Z INFO  layout_optimization_sa::optimization] Process   0: Starting layout: k.o,yvgclßzhaeiudtrnsfxqäüöbpwmj ( 417.6)
[2023-07-09T14:57:21Z INFO  layout_optimization_sa::optimization] Process   2: Starting layout: k.o,yvgclßzhaeiudtrnsfxqäüöbpwmj ( 417.6)
[2023-07-09T14:57:21Z INFO  layout_optimization_sa::optimization] Process   4: Starting layout: k.omyvgclßzhaeiudtrnsfxqäüöbpw,j ( 406.5)
[2023-07-09T14:57:21Z INFO  layout_optimization_sa::optimization] Process   7: Starting layout: k.o,yvgclßzhaeiudtrnsfxqäüöbpwmj ( 417.6)

Glitchy-Tozier · 2023-07-09T17:29:39Z

It seems this probably was a weird quirk in the way Argmin-observers work.
Once we're done with #62, I can create a PR that clears up the output. It's really simple.

Glitchy-Tozier changed the title ~~Bug: layout.resolve_modifiers() is inconsistent.~~ Bug: optimize_sa outputs wrong starting layout Jul 22, 2023

Glitchy-Tozier mentioned this issue Oct 26, 2023

optimize_sa: Change output to better indicate functionality #70

Merged

dariogoetz closed this as completed in #70 Oct 26, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug: `optimize_sa` outputs wrong starting layout #64

Bug: `optimize_sa` outputs wrong starting layout #64

Glitchy-Tozier commented Jul 9, 2023

dariogoetz commented Jul 9, 2023

Glitchy-Tozier commented Jul 9, 2023

Glitchy-Tozier commented Jul 9, 2023 •

edited

Loading

Bug: optimize_sa outputs wrong starting layout #64

Bug: optimize_sa outputs wrong starting layout #64

Comments

Glitchy-Tozier commented Jul 9, 2023

dariogoetz commented Jul 9, 2023

Glitchy-Tozier commented Jul 9, 2023

Glitchy-Tozier commented Jul 9, 2023 • edited Loading

Bug: `optimize_sa` outputs wrong starting layout #64

Bug: `optimize_sa` outputs wrong starting layout #64

Glitchy-Tozier commented Jul 9, 2023 •

edited

Loading