Bug: "No available threads to lock"

[danpalmer]

Describe the bug

The following assert regularly fails:

usearch_assert_m(available_threads_.size(), "No available threads to lock");

Steps to reproduce

This is flaky and does not always happen, but appears to happen when USearch is being used many times in quick succession (i.e. high traffic, or a number of queue jobs running).

My USearch implementation is contained within a Swift Actor, so all access should be occurring on the same thread, making the use inherently single threaded. Checking the state of all threads at the point that the assert fails, this does appear to be the case – nothing else is using USearch.

Expected behavior

I don't expect asserts to fail in production code.

I did read #488, but it's unclear if/how this applies to the Swift bindings, and/or to single threaded access.

USearch version

2.16.9

Operating System

macOS 15.0

Hardware architecture

Arm

Which interface are you using?

Other bindings

Contact Details

See profile.

Are you open to being tagged as a contributor?

• I am open to being mentioned in the project .git history as a contributor

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

[ashvardanian]

Hi @danpalmer! Can you please provide a wider code snippet of how you initialize the Index and how do you spawn the threads?

[danpalmer]

Sure, this is the entire interaction with USearch. Apologies for the poor code, this hasn't been reviewed by anyone yet.

import Config
import Foundation
import PathKit
import USearch

actor USearchEmbeddingStore: EmbeddingStore {
    let index: USearchIndex
    let path: Path
    var isReady: Bool = false

    init(model: LLModel, path: Path) {
        index = USearchIndex.make(
            metric: .l2sq,
            dimensions: UInt32(model.embeddingSize),
            connectivity: 24,
            quantization: .F16
        )
        self.path = path
    }

    func load() throws {
        if isReady {
            return
        }
        if path.exists {
            index.load(path: path.string)
        } else {
            index.save(path: path.string)
        }
        isReady = true
    }

    func save() throws {
        if !isReady {
            throw EmbeddingStoreError.notReady
        }
        index.save(path: path.string)
    }

    func upsert(id: USearchKey, embedding: [Double]) throws {
        if !isReady {
            throw EmbeddingStoreError.notReady
        }
        index.reserve(1)
        index.addDouble(key: id, vector: embedding)
    }

    func remove(id: USearchKey) throws {
        if !isReady {
            throw EmbeddingStoreError.notReady
        }
        index.remove(key: id)
    }

    func query(_ embedding: [Double], nearest count: Int) async throws -> [UInt64] {
        if !isReady {
            throw EmbeddingStoreError.notReady
        }
        let (keys, _) = index.search(vector: embedding, count: count)
        return keys
    }
}

The Swift bindings don't provide any threading control, and being an actor this all operates on one thread. As the USearch functions are all synchronous I'd assume no work is happening concurrently, so I'm not sure why threading is an issue at all so far with this solution.

As I scale things it would be nice to have some concurrency in the bindings, but I'm not there yet.

[danpalmer]

Hey, is there anything I can provide to help debug this? Is there anything I should look into? Anything that might work as a possible workaround?

I've tried digging through, but the Swift interface differs somewhat from the Obj-C layer, which then differs again from the C++ layer, and I don't think I'm going to make much progress debugging threading issues two languages and two concurrency models away from what I'm writing. Would appreciate a starting place!

[ashvardanian]

It's a very bad idea to reserve space one-by-one. The solution would be to generalize the reserve function to also accept a thread count and pass it down to ObjC and C++.

[danpalmer]

I must admit I really don't understand the reservation. Very open to changing this, I just can't find any documentation on it other than bugs that suggest that you always need at least 1 reserved. What exactly are the semantics of it? How does space reservation play into threading?

What is the threading model, and how does that play into Swift Actors, where all access would be guaranteed to be on a single thread?

[danpalmer]

Hey @ashvardanian, any update here? I have some time to try fixes here, and if there's a need to expose threading in the Objective-C or Swift API I may be able to help with that.