Estimate indexing progress better #337
Labels
enhancement
New feature or request
Comments
|
Maybe we could accept the downside of counting all indexable files, and just count all up front. Currently it counts on the encountered chunks in the indexing stream, so it takes a while to get a total count. It probably takes milliseconds anyway to count thousands (if not more). The only trouble I see there is that we don't have an off-the-bat file is done, as the chunks from those files are processed unordered. |
timonv
added a commit
to bosun-ai/swiftide
that referenced
this issue
Mar 3, 2025
Allows playing with the iterator outside of the stream. Relates to bosun-ai/kwaak#337
timonv
pushed a commit
to bosun-ai/swiftide
that referenced
this issue
Mar 5, 2025
## 🤖 New release
* `swiftide-core`: 0.21.1 -> 0.22.0 (✓ API compatible changes)
* `swiftide-macros`: 0.21.1 -> 0.22.0
* `swiftide-agents`: 0.21.1 -> 0.22.0 (✓ API compatible changes)
* `swiftide-indexing`: 0.21.1 -> 0.22.0 (✓ API compatible changes)
* `swiftide-integrations`: 0.21.1 -> 0.22.0 (⚠ API breaking changes)
* `swiftide-query`: 0.21.1 -> 0.22.0 (✓ API compatible changes)
* `swiftide`: 0.21.1 -> 0.22.0 (✓ API compatible changes)
### ⚠ `swiftide-integrations` breaking changes
```text
--- failure enum_missing: pub enum removed or renamed ---
Description:
A publicly-visible enum cannot be imported by its prior path. A `pub use` may have been removed, or the enum itself may have been renamed or removed entirely.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.39.0/src/lints/enum_missing.ron
Failed in:
enum swiftide_integrations::groq::GroqBuilderError, previously in file /tmp/.tmp04qAim/swiftide-integrations/src/groq/mod.rs:23
enum swiftide_integrations::openai::OpenAIBuilderError, previously in file /tmp/.tmp04qAim/swiftide-integrations/src/openai/mod.rs:38
enum swiftide_integrations::groq::OptionsBuilderError, previously in file /tmp/.tmp04qAim/swiftide-integrations/src/groq/mod.rs:46
--- failure struct_missing: pub struct removed or renamed ---
Description:
A publicly-visible struct cannot be imported by its prior path. A `pub use` may have been removed, or the struct itself may have been renamed or removed entirely.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.39.0/src/lints/struct_missing.ron
Failed in:
struct swiftide_integrations::groq::GroqBuilder, previously in file /tmp/.tmp04qAim/swiftide-integrations/src/groq/mod.rs:23
struct swiftide_integrations::openai::OpenAIBuilder, previously in file /tmp/.tmp04qAim/swiftide-integrations/src/openai/mod.rs:38
struct swiftide_integrations::groq::OptionsBuilder, previously in file /tmp/.tmp04qAim/swiftide-integrations/src/groq/mod.rs:46
struct swiftide_integrations::groq::Groq, previously in file /tmp/.tmp04qAim/swiftide-integrations/src/groq/mod.rs:25
struct swiftide_integrations::openai::OpenAI, previously in file /tmp/.tmp04qAim/swiftide-integrations/src/openai/mod.rs:40
struct swiftide_integrations::groq::Options, previously in file /tmp/.tmp04qAim/swiftide-integrations/src/groq/mod.rs:48
```
<details><summary><i><b>Changelog</b></i></summary><p>
## `swiftide`
<blockquote>
##
[0.22.0](v0.21.1...v0.22.0)
- 2025-03-03
### New features
-
[a754846](a754846)
*(agents)* Implement Serialize and Deserialize for chat messages
````text
Persist, retry later, evaluate it completions in a script, you name it.
````
-
[0a592c6](0a592c6)
*(indexing)* Add `iter()` for file loader
([#655](#655))
````text
Allows playing with the iterator outside of the stream.
Relates to bosun-ai/kwaak#337
````
-
[57116e9](57116e9)
Groq ChatCompletion
([#650](#650))
````text
Use the new generics to _just-make-it-work_.
````
-
[4fd3259](4fd3259)
Continue Agent on Tool Failure
([#628](#628))
````text
Ensure tool calls and responses are always balanced, even when the tool retry limit is reached
bosun-ai/kwaak#313
````
### Miscellaneous
- [0000000](0000000) Update
Cargo.toml dependencies
**Full Changelog**:
0.21.1...0.22.0
</blockquote>
</p></details>
---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Using kwaak for the first time, I noticed that the indexing total count kept moving upward, while the completed count stayed at zero for a long while (using Ollama with the config from the readme (llama 3.3 / llama 3.2 / bge-m3).
It would be nice to come to a total estimate as soon as possible so that the timing can be estimated.
The message could be more explicit to be helpful to users about what's actually happening. Currently it's "Indexing a bit of code 0/123".
The text was updated successfully, but these errors were encountered: