You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If the difference between desired monitor resolution and the file's resolution is too large the I16 that stores the "min error" in "Find Best XML File In Folder" rolls over (sometimes to a negative number) and in some cases results in returning the wrong file - not the one containing the minimum error.
For example, I have two files corresponding to two screen resolutions - one for 2560x1440 and the other for 1596x1127. If I am currently using a monitor that has 2560x1440 I expect it to return the file for 2560x1440.
The difference between the 1592x1127 file and my 2560x1440 resolution is 964x313 and when it is squared it becomes 11792x32433. Those numbers are added and compared to the minimum error to determine whether it is the new minimum or not. Since it is an I16 and 11792+32433=44225 which is > 32767, it rolls over and returns -21311. Since this is (incorrectly) smaller than the current error, it is storing this value as the minimum and passing out the file index corresponding to this incorrect minimum instead of the one with the actual smaller number.
When I change the number to a 32-bit integer this problem goes away.
The text was updated successfully, but these errors were encountered:
If the difference between desired monitor resolution and the file's resolution is too large the I16 that stores the "min error" in "Find Best XML File In Folder" rolls over (sometimes to a negative number) and in some cases results in returning the wrong file - not the one containing the minimum error.
For example, I have two files corresponding to two screen resolutions - one for 2560x1440 and the other for 1596x1127. If I am currently using a monitor that has 2560x1440 I expect it to return the file for 2560x1440.

The difference between the 1592x1127 file and my 2560x1440 resolution is 964x313 and when it is squared it becomes 11792x32433. Those numbers are added and compared to the minimum error to determine whether it is the new minimum or not. Since it is an I16 and 11792+32433=44225 which is > 32767, it rolls over and returns -21311. Since this is (incorrectly) smaller than the current error, it is storing this value as the minimum and passing out the file index corresponding to this incorrect minimum instead of the one with the actual smaller number.
When I change the number to a 32-bit integer this problem goes away.
The text was updated successfully, but these errors were encountered: