[QUESTION/BUG] Wrong point found with specific needle on haystack

[henriquelino]

Hello! I've been stuck with this for the last couple of days, I have a PDF with some fields I need to click, the fields are light blue, so I search these using pyautogui.locateCenterOnScreen, but as I tracked down to pyscreeze._locateAll_opencv, I will use this on demo code

What happens is:

• When using cv.matchTemplate, it finds the correct position: Box(left=30, top=243, width=8, height=9)
• When using pyscreeze it finds many boxes at top of image but none is correct

I'm glad to help if needed, but atm I don't know what to do anymore besides using matchTemplate for this exact set of needle/haystack, this might be a bug, so I guess I should open this issue

---

Some info:

• OS: Windows 10 Pro Version 21H2
• Python: 3.9.13
• PyAutogui: 0.9.53
• pyscreeze: 0.1.28

---

The code:

import pyscreeze
import cv2

def template_match(haystack, needle):
    """from: https://stackoverflow.com/questions/7853628/how-do-i-find-an-image-contained-within-an-image"""

    method = cv2.TM_SQDIFF_NORMED

    haystack = cv2.imread(haystack)
    needle = cv2.imread(needle)

    h, w = needle.shape[:2]

    needle_w, needle_h = needle.shape[:2]

    result = cv2.matchTemplate(needle, haystack, method)

    minVal, maxVal, minLoc, maxLoc = cv2.minMaxLoc(result)
    x, y = minLoc

    return pyscreeze.Box(x, y, needle_w, needle_h)


if __name__ == '__main__':

    haystack = r".\haystack.png"
    needle = r".\needle.png"

    correct_box = template_match(haystack, needle)

    boxes: list[pyscreeze.Box] = list(pyscreeze._locateAll_opencv(needle, haystack))
    for box in boxes:
        print(f"Pyscreeze matched {box}")

    print(f"{correct_box = }")

---

The images:

• Needle:
  

• haystack: