Skip to content

PixelSearch and ImageSearch

Jump to bottom
Edison Hua edited this page Jan 14, 2025 · 147 revisions

Buffers

Use ImagePutBuffer to create a pixel buffer. See details here.

pic := ImagePutBuffer(0)   ; 0 = All, Monitor: 1,2,3...    ; Screen Capture
pic := ImagePutBuffer([200, 200, 30, 50])                  ; Screenshot [x, y, w, h]
pic := ImagePutBuffer("cats.jpg")                          ; Open a file

pic := ImagePutBuffer("A") ; Active Window                 ; Slow Window Capture
pic := ImagePutBuffer({window: "A", crop: [0, 0, 50, 50]}) ; Slow Window Capture + Crop
pic := ImagePutBuffer({screenshot: "A"})                   ; Fast Window Capture
pic := ImagePutBuffer([0, 0, 50, 50, "A"])                 ; Fast Window Capture + Faster Crop

For all 20+ input types, see here. Use of fast window capture is recommended. Only slow window capture can work with windows that are behind another.

Delete

  • Auto deletes as it loses scope. To delete a buffer manually, such as when stored inside an object, use pic := ""

Getter and Setter

  • Get pixel: color := pic[x, y] Returns the ARGB value of the pixel.
  • Set pixel: pic[x, y] := 0x00FF00 (RGB values are assumed to have an alpha of 255)

Properties

  • ptr - The pointer to the byte array holding the image.
  • size - The size of the byte array.
  • width, height - Get the width and height of the image.
  • stride, pitch - Calculates the scanline length using size // height
  • pBitmap - A GDI+ Bitmap reference to the byte array.

Basic Methods

  • crop(x, y, w, h) - Copies the cropped image into a new buffer.
  • clone() - Copies the image into a new buffer.
  • save(filepath?, quality?) - Defaults to writing a 32-bit ARGB uncompressed .bmp file. Other extensions use ImagePutFile.
  • show(window_border?, title?, pos?, style?, styleEx?, parent?) - Use show(1) for window borders. See ImageShow for details.

Advanced Methods

  • PixelSearch(color, variation?)
  • ImageSearch(image)
  • ColorKey(color?, replacement?) - Replaces one ARGB color with another ARGB color. Default color is the top-left pixel.
  • SetAlpha(alpha) - Sets the alpha channel of the entire image. Range: 0-255.
  • TransColor(color?, alpha?) - Sets the alpha channel of a single RGB color. For example, if two RGB colors have different alpha values, they can be set the same. Default color is the top-left pixel.

PixelGetColor / PixelSetColor

Get a color at [x, y]. Returns an ARGB value.

pic := ImagePutBuffer(1) ; Capture Primary Screen
color := pic[100, 200]
MsgBox % color

Compare color

Note: For comparisons use ARGB like 0xFFD2B318 ✅ because it starts with 0xFF, not RGB like 0xD2B318 ❌

; Gets the pixel at [100, 100].
color := pic[100, 100]

; Check if the pixel is red.
if (color = 0xFFFF0000) ; See how this uses ARGB, and not 0xFF0000?
    MsgBox

Set a color at [x, y].

Note: RGB values are assumed to have an alpha of 255.

; Sets the pixel at [100, 100] to red.
pic[100, 100] := 0xFF0000

Set multiple pixels by chaining the assignment operator.

; Example: Shows a small red square inside the image.
pic := ImagePutBuffer("https://picsum.photos/100")
pic[0, 0] := pic[0, 1] := pic[0, 2] := 0xFF0000
pic[1, 0] := pic[1, 1] := pic[1, 2] := 0xFF0000
pic[2, 0] := pic[2, 1] := pic[2, 2] := 0xFF0000
pic.show() ; or ImageShow(pic)

2

PixelSearch

Gets [x, y] position of a color.

pic := ImagePutBuffer("https://picsum.photos/300")     ; Load image
pic.show() ; or ImageShow(pic)                         ; Show image
if xy := pic.PixelSearch(0xFFFFFF) {                   ; Get [x, y] of 0xFFFFFF
    MouseMove xy[1], xy[2]                             ; Move cursor
    Send "{MButton}" ; MsgBox pic[xy*]                 ; Display color
} else Reload                                          ; Restart

Brief overview of options below

Option 1: PixelSearch, single color with no variation.
Option 2: PixelSearch, single color with single variation.
Option 3: PixelSearch, single color with multiple variation.
Option 4: PixelSearch, range of colors.
Option 5: PixelSearch, multiple colors with no variation.
Option 6: PixelSearch, multiple colors with single variation.
Option 7: PixelSearch, multiple colors with multiple variation.

PixelSearch(color, variation) (Option #2)

Specify a number between 0 and 255 (inclusive) to indicate the allowed number of shades of variation in either direction for the intensity of the red, green, and blue components of each pixel's color.

pic := ImagePutBuffer("https://picsum.photos/300")     ; Load image
if xy := pic.PixelSearch(0xFFFFFF, 3) {                ; Get [x, y] of 0xFFFFFF with variation of 3
    pic.show() ; or ImageShow(pic)                     ; Show image
    MouseMove xy[1], xy[2]                             ; Move cursor
    Send "{MButton}" ; MsgBox pic[xy*]                 ; Display color
} else Reload                                          ; Restart

Note: The syntax pic[xy*] is shorthand for pic[xy[1], xy[2]] by using the unpacking operator * to unpack arrays.

PixelSearch(color, [v1, v2, v3]) (Option #3)

Search for colors using the red, green, blue color channel where each color channel has its own variation.

; Red (0x11 ± 3) Green (0x22 ± 15), Blue (0x33 ± 100)
xy := pic.PixelSearch(0x112233, [3, 15, 100])

PixelSearch(0, [r1, r2, g1, g2, b1, b2]) (Option #4)

Search for colors (inclusive) between the red high and low, green high and low, and blue high and low. The color parameter is ignored. Since this specifies a range, it does not matter if it is increasing or decreasing.

; Red (245, 255) Green (137, 155), Blue (142, 99)
xy := pic.PixelSearch(0, [245, 255, 137, 155, 142, 99])

PixelSearch([c1, c2, ...]) (Option #5)

Search for multiple colors.

pic := ImagePutBuffer("https://picsum.photos/300")     ; Load image
pic.show() ; or ImageShow(pic)                         ; Show image
if xy := pic.PixelSearch([0xFFFFFF, 0x000000]) {       ; Get [x, y] of white or black
    MouseMove xy[1], xy[2]                             ; Move cursor
    Send "{MButton}" ; MsgBox pic[xy*]                 ; Display color
} else Reload                                          ; Restart

PixelSearch([c1, c2, ...], variation) (Option #6)

Search for multiple colors using a constant variation.

pic := ImagePutBuffer("https://picsum.photos/300")     ; Load image
if xy := pic.PixelSearch([0xFFFFFF, 0x000000], 3) {    ; Get [x, y] of white or black with variation of 3
    pic.show() ; or ImageShow(pic)                     ; Show image
    MouseMove xy[1], xy[2]                             ; Move cursor
    Send "{MButton}" ; MsgBox pic[xy*]                 ; Display color
} else Reload                                          ; Restart

PixelSearch([c1, c2, ...], [v1, v2, v3]) (Option #7)

Search for multiple colors where each color channel has its own variation.

; Searches for:
; Red (0xFF) Green (0xFF), Blue (0xFF ± 3)
; Red (0x00) Green (0x00), Blue (0x00 ± 3)

xy := pic.PixelSearch([0xFFFFFF, 0x000000], [0, 0, 3])

; List of matching colors:
; 0xFFFFFFFF, 0xFFFFFFFE, 0xFFFFFFFD, 0xFFFFFFFC
; 0xFF000000, 0xFF000001, 0xFF000002, 0xFF000003

PixelSearchAll

Returns a nested array [[x, y], [x2, y2], [x3, y3]...]. Options are identical to PixelSearch, see above ⬆️.

pic := ImagePutBuffer("https://picsum.photos/800")     ; Load image
pic.show() ; or ImageShow(pic)                         ; Show image
if xys := pic.PixelSearchAll([0xFFFFFF, 0x000000]) {   ; Search for white or black
    if xy := xys[1] {                                  ; Get first [x, y]
        MouseMove xy[1], xy[2]                         ; Move cursor
        Send "{MButton}" ; MsgBox pic[xy*]             ; Display color
    }

    ; TrayTip,, % xys.length()                         ; Search count (AutoHotkey v1)
    ; TrayTip xys.length                               ; Search count (AutoHotkey v2)

} else Reload                                          ; Restart

ImageSearch

Search for a matching image. Returns top-left [x, y] of the first match.

pic := ImagePutBuffer(0)                               ; Screen capture
pic.show() ; or ImageShow(pic)                         ; Show image
if xy := pic.ImageSearch("test_image.png") {           ; Search image
    MouseMove xy[1], xy[2]                             ; Move cursor
    Send "{MButton}"                                   ; MsgBox pic[xy*]
}

Cache the search image for faster speed.

pic := ImagePutBuffer(0)                               ; Screen capture
search := ImagePutBuffer("test_image.png")             ; Convert File -> Buffer
pic.show() ; or ImageShow(pic)                         ; Show image
if xy := pic.ImageSearch(search)                       ; Search image
    MouseMove xy[1], xy[2]                             ; Move cursor

If the search image contains transparent pixels, they will be ignored and only fully opaque pixels in the search image will be checked.

; Add the Transcolor line to the above example.
search := ImagePutBuffer("test_image.png")             ; Convert File -> Buffer
search.TransColor(0xFFFFFF)                            ; Sets all white pixels to be transparent

Experimental: To add variation to the ImageSearch function, add it as the 2nd parameter: ImageSearch(target_image, 3)

ImageSearchAll

Returns an array of all matches in the form of [[x, y], [x2, y2], [x3, y3]...]. To get the number of matches, use the .length property of the returned array.

pic := ImagePutBuffer(0)                               ; Screen capture
search := ImagePutBuffer("test_image.png")             ; Convert File -> Buffer
xys := pic.ImageSearchAll(search)                      ; Array of [x, y] arrays
if (xys) {
    count := xys.length                                ; Number of matches
    xy := xys[1]                                       ; First match
    ; xy2 := xys[2]                                    ; Second match
    ; xylast := xys[-1]                                ; Last match
    if (xy)                                            ; Check first [x, y]
        MouseMove xy[1], xy[2]                         ; Move cursor
}
Clone this wiki locally