Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(lancedb): Should not error if table exists #349

Merged
merged 5 commits into from
Oct 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,4 +74,4 @@ jobs:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- uses: r7kamura/rust-problem-matchers@v1
- name: clippy
run: cargo clippy --workspace --all-features
run: cargo clippy --all-targets --all-features
12 changes: 6 additions & 6 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion examples/index_codebase_reduced_context.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
//! The pipeline will:
//! - Load all `.rs` files from the current directory
//! - Skip any nodes previously processed; hashes are based on the path and chunk (not the
//! metadata!)
//! metadata!)
//! - Generate an outline of the symbols defined in each file to be used as context in a later step and store it in the metadata
//! - Chunk the code into pieces of 10 to 2048 bytes
//! - For each chunk, generate a condensed subset of the symbols outline tailored for that specific chunk and store that in the metadata
Expand Down
51 changes: 46 additions & 5 deletions swiftide-integrations/src/lancedb/persist.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,19 @@ impl Persist for LanceDB {
let conn = self.get_connection().await?;
let schema = self.schema.clone();

conn.create_empty_table(&self.table_name, schema)
.execute()
.await
.map(|_| ())
.map_err(Into::into)
if let Err(err) = conn.open_table(&self.table_name).execute().await {
if matches!(err, lancedb::Error::TableNotFound { .. }) {
conn.create_empty_table(&self.table_name, schema)
.execute()
.await
.map(|_| ())
.map_err(anyhow::Error::from)?;
} else {
return Err(err.into());
}
}

Ok(())
}

#[tracing::instrument(skip_all)]
Expand Down Expand Up @@ -151,3 +159,36 @@ impl LanceDB {
Ok(batches)
}
}

#[cfg(test)]
mod test {
use swiftide_core::{indexing::EmbeddedField, Persist as _};
use temp_dir::TempDir;

use super::*;

async fn setup() -> (TempDir, LanceDB) {
let tempdir = TempDir::new().unwrap();
let lancedb = LanceDB::builder()
.uri(tempdir.child("lancedb").to_str().unwrap())
.vector_size(384)
.with_metadata("filter")
.with_vector(EmbeddedField::Combined)
.table_name("swiftide_test")
.build()
.unwrap();
lancedb.setup().await.unwrap();

(tempdir, lancedb)
}

#[tokio::test]
async fn test_no_error_when_table_exists() {
let (_guard, lancedb) = setup().await;

lancedb
.setup()
.await
.expect("Should not error if table exists");
}
}
4 changes: 2 additions & 2 deletions swiftide-integrations/src/treesitter/code_tree.rs
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,7 @@ mod tests {
#[test]
fn test_parsing_on_java_enum() {
let parser = CodeParser::from_language(SupportedLanguages::Java);
let code = r#"
let code = r"
enum Material {
DENIM,
CANVAS,
Expand All @@ -274,7 +274,7 @@ mod tests {
c = Material.DENIM;
}
}
"#;
";
let tree = parser.parse(code).unwrap();
let result = tree.references_and_definitions().unwrap();
assert_eq!(result.definitions, vec!["Material", "Person", "getName"]);
Expand Down
2 changes: 1 addition & 1 deletion swiftide-integrations/src/treesitter/outliner.rs
Original file line number Diff line number Diff line change
Expand Up @@ -316,7 +316,7 @@ public class HelloWorld {
"#;
let outliner = CodeOutliner::new(SupportedLanguages::Java);
let summary = outliner.outline(code).unwrap();
println!("{}", summary);
println!("{summary}");
assert_eq!(
summary,
"\nimport java.io.PrintStream;\nimport java.util.Scanner;\n\npublic class HelloWorld {\n // This is a comment\n public static void main(String[] args) \n}"
Expand Down
4 changes: 2 additions & 2 deletions swiftide/tests/query_pipeline.rs
Original file line number Diff line number Diff line change
Expand Up @@ -89,8 +89,8 @@ async fn test_hybrid_search_qdrant() {
.build()
.unwrap();

let fastembed_sparse = FastEmbed::try_default_sparse().unwrap().to_owned();
let fastembed = FastEmbed::try_default().unwrap().to_owned();
let fastembed_sparse = FastEmbed::try_default_sparse().unwrap().clone();
let fastembed = FastEmbed::try_default().unwrap().clone();

println!("Qdrant URL: {qdrant_url}");

Expand Down