Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement test cases for search #10193

Merged
merged 74 commits into from
Jan 8, 2024
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
c93a22b
Move search folder to logic
Luggas4you Aug 20, 2023
d789dd3
Add testEmptyLibrarySearch
Luggas4you Aug 20, 2023
2688d17
Add initializeDatabaseFromPath
Luggas4you Aug 20, 2023
ab36944
Add TestLibraryA & testUpperAndLowerWordSearch
Luggas4you Aug 20, 2023
2383a15
Add testSimpleSingleFieldSearch
Luggas4you Aug 20, 2023
032ea25
Add testSimpleMultipleFieldSearch
Luggas4you Aug 20, 2023
97d2b82
Add testSensitiveWordSearch
Luggas4you Aug 20, 2023
0b2cc9b
Add testSensitiveMutipleFieldSearch
Luggas4you Aug 20, 2023
5ac3848
Add BibEntries for test-library-B
Luggas4you Aug 20, 2023
7a3012e
Add testSimpleRegularExpression
Luggas4you Aug 20, 2023
461b1d3
Add testSensitiveRegularExpression
Luggas4you Aug 20, 2023
e448d65
SimplePDFFulltextSearch WIP
Luggas4you Aug 25, 2023
7e73fcb
Change set.Files to add.File
Luggas4you Aug 28, 2023
7c28794
Add testSimplePDFNoteFulltextSearch WIP
Luggas4you Aug 30, 2023
d1a9c84
Move indexer
Luggas4you Aug 30, 2023
d9d84a2
Add missing @Tests to SearchFunctionalityTest.java WIP
Luggas4you Sep 30, 2023
dbcf01d
WIP
Luggas4you Oct 1, 2023
c34d308
Exclude non-functional @Tests
Luggas4you Oct 6, 2023
ec9d961
Merge branch 'main' into searchtestcases
koppor Jan 2, 2024
0612d31
Add withFiles
koppor Jan 2, 2024
2ea65a6
Remove empty lines
koppor Jan 2, 2024
f38e39d
@NullMarked for LinkedFile
koppor Jan 2, 2024
6e42ece
Fix Logger parameter
koppor Jan 2, 2024
e1844fd
Streamline tests
koppor Jan 3, 2024
df9c069
Fix checkstyle
koppor Jan 3, 2024
ebaaf4a
Refine tests
koppor Jan 6, 2024
1d8b17c
Get rid of missing "identity" formatter
koppor Jan 6, 2024
06cebdf
Minimize test files
koppor Jan 6, 2024
40d8022
Remove obsolete method (and make indexer class variable to speedup)
koppor Jan 6, 2024
c475550
Fix filenames
koppor Jan 7, 2024
d92b425
Some refactorings
koppor Jan 7, 2024
e081044
Revert global indexer mapping (does not work)
koppor Jan 7, 2024
3a0dde6
Revert "Revert global indexer mapping (does not work)"
koppor Jan 7, 2024
9e81a9a
Small code updates
koppor Jan 7, 2024
7e49053
Rename test files
koppor Jan 7, 2024
d888e6f
Some more logging
koppor Jan 7, 2024
f9f368b
Refine transaction boundaries (and some minor tweaks)
koppor Jan 7, 2024
4cbcf9b
Add TODO
koppor Jan 7, 2024
660f31c
Fix off-by-one error
koppor Jan 7, 2024
06e49ba
Remove non-used .bib file
koppor Jan 7, 2024
5e6213b
Refine comments
koppor Jan 7, 2024
b682b90
Fix variable assignment
koppor Jan 7, 2024
dd065e3
test-library-A -> test-library-title-casing
koppor Jan 7, 2024
407308e
Merge test cases of library-B.bib into title-casing.bib
koppor Jan 7, 2024
825e219
Merge branch 'main' into searchtestcases
koppor Jan 7, 2024
ac0ae7d
Increase transaction boundary for index addition/removal
koppor Jan 7, 2024
5ab3b37
Merge branch 'searchtestcases' of https://github.com/Luggas4you/jabre…
koppor Jan 7, 2024
dce3b9e
More readable directory names for index directory
koppor Jan 7, 2024
603ec36
WIP: Introduce PdfIndexerManager
koppor Jan 7, 2024
4497943
Preparation for "fulltext search not checking all attached files upon…
koppor Jan 8, 2024
c2b4f16
Use "right" factory (and rename factor getter)
koppor Jan 8, 2024
1c1a1b3
Fix FullTextSeachRule (refactoring introduced a bug)
koppor Jan 8, 2024
f102b21
Revert property for recheck of attached files
koppor Jan 8, 2024
0cce22c
Fix linting issues
koppor Jan 8, 2024
9672c48
Fix Formatters optimization
koppor Jan 8, 2024
4455dc4
Merge branch 'main' into searchtestcases
koppor Jan 8, 2024
5374efd
Fix test
koppor Jan 8, 2024
ff24c3d
Exception for architecture
koppor Jan 8, 2024
7e2e738
Refine .gitignore
koppor Jan 8, 2024
2b2bc64
Add missing }
koppor Jan 8, 2024
2386718
Merge remote-tracking branch 'upstream/main' into searchtestcases
koppor Jan 8, 2024
a1ef7fd
Do not search for PDF files in case of an exception of a search
koppor Jan 8, 2024
be63cdc
Remove duplicate code (and unneccsary pre-fetch of search results)
koppor Jan 8, 2024
ad9fb68
Add exception for test
koppor Jan 8, 2024
4a325c0
Merge branch 'main' into searchtestcases
koppor Jan 8, 2024
8da2b33
Remove ".getMessage()"
koppor Jan 8, 2024
2cd8184
Move comment and remove obsolete variable
koppor Jan 8, 2024
1bd6402
Fix typo
koppor Jan 8, 2024
f4cbbf7
Fix name
koppor Jan 8, 2024
f15f2be
Add dot
koppor Jan 8, 2024
807ce2f
Add comment
koppor Jan 8, 2024
e597a17
Call splitting method
koppor Jan 8, 2024
751c4b4
Add JavaDocComment
koppor Jan 8, 2024
af10895
Revert lambdas
koppor Jan 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
227 changes: 227 additions & 0 deletions src/test/java/org/jabref/logic/search/SearchFunctionalityTest.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,227 @@
package org.jabref.logic.search;

import java.io.IOException;
import java.net.URISyntaxException;
import java.nio.file.Path;
import java.util.Collections;
import java.util.EnumSet;
import java.util.List;

import org.jabref.logic.importer.ImportFormatPreferences;
import org.jabref.logic.importer.ParserResult;
import org.jabref.logic.importer.fileformat.BibtexImporter;
import org.jabref.logic.pdf.search.indexing.PdfIndexer;
import org.jabref.logic.pdf.search.retrieval.PdfSearcher;
import org.jabref.logic.util.StandardFileType;
import org.jabref.model.database.BibDatabase;
import org.jabref.model.database.BibDatabaseContext;
import org.jabref.model.entry.BibEntry;
import org.jabref.model.entry.LinkedFile;
import org.jabref.model.entry.field.StandardField;
import org.jabref.model.entry.types.StandardEntryType;
import org.jabref.model.pdf.search.PdfSearchResults;
import org.jabref.model.search.rules.SearchRules;
import org.jabref.model.util.DummyFileUpdateMonitor;
import org.jabref.preferences.FilePreferences;

import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.mockito.Answers;
import org.mockito.Mockito;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

public class SearchFunctionalityTest {

private BibDatabase database;
private PdfSearcher search;
private BibDatabaseContext context;
private FilePreferences filePreferences;
BibEntry entry1A = new BibEntry(StandardEntryType.Misc)
.withCitationKey("entry1")
.withField(StandardField.AUTHOR, "Test")
.withField(StandardField.TITLE, "cASe");
BibEntry entry2A = new BibEntry(StandardEntryType.Misc)
.withCitationKey("entry2")
.withField(StandardField.AUTHOR, "test")
.withField(StandardField.TITLE, "casE");
BibEntry entry3A = new BibEntry(StandardEntryType.Misc)
.withCitationKey("entry3")
.withField(StandardField.AUTHOR, "tESt")
.withField(StandardField.TITLE, "Case");
BibEntry entry4A = new BibEntry(StandardEntryType.Misc)
.withCitationKey("entry4")
.withField(StandardField.AUTHOR, "tesT")
.withField(StandardField.TITLE, "CASE");
BibEntry entry5A = new BibEntry(StandardEntryType.Misc)
.withCitationKey("entry5")
.withField(StandardField.AUTHOR, "TEST")
.withField(StandardField.TITLE, "case");
BibEntry entry1B = new BibEntry(StandardEntryType.Misc)
.withCitationKey("entry1")
.withField(StandardField.AUTHOR, "Test")
.withField(StandardField.TITLE, "Case");
BibEntry entry2B = new BibEntry(StandardEntryType.Misc)
.withCitationKey("entry2")
.withField(StandardField.AUTHOR, "User")
.withField(StandardField.TITLE, "case");
BibEntry entry3B = new BibEntry(StandardEntryType.Misc)
.withCitationKey("entry3")
.withField(StandardField.AUTHOR, "test")
.withField(StandardField.TITLE, "text");
BibEntry entry4B = new BibEntry(StandardEntryType.Misc)
.withCitationKey("entry4")
.withField(StandardField.AUTHOR, "Special")
.withField(StandardField.TITLE, "192? title.");
BibEntry entry1C = new BibEntry(StandardEntryType.Misc)
.withCitationKey("minimal1");
BibEntry entry2C = new BibEntry(StandardEntryType.Misc)
.withCitationKey("minimal2");

BibEntry entry3C = new BibEntry(StandardEntryType.Misc)
.withCitationKey("minimal3");
BibEntry entry4C = new BibEntry(StandardEntryType.Misc)
.withCitationKey("minimal-note1");
BibEntry entry5C = new BibEntry(StandardEntryType.Misc)
.withCitationKey("minimal-note2");
BibEntry entry6C = new BibEntry(StandardEntryType.Misc)
.withCitationKey("minimal-note3");

@BeforeEach
public void setUp(@TempDir Path indexDir) throws IOException {
filePreferences = mock(FilePreferences.class);
database = new BibDatabase();
context = mock(BibDatabaseContext.class);
entry1C.setFiles(Collections.singletonList(new LinkedFile("Minimal", "minimal.pdf", StandardFileType.PDF.getName())));
koppor marked this conversation as resolved.
Show resolved Hide resolved
entry1C.setFiles(Collections.singletonList(new LinkedFile("Minimal 1", "minimal1.pdf", StandardFileType.PDF.getName())));
entry1C.setFiles(Collections.singletonList(new LinkedFile("Minimal 2", "minimal2.pdf", StandardFileType.PDF.getName())));
entry1C.setFiles(Collections.singletonList(new LinkedFile("Minimalnote", "minimal-note.pdf", StandardFileType.PDF.getName())));
entry1C.setFiles(Collections.singletonList(new LinkedFile("Minimalnote 1", "minimal-note1.pdf", StandardFileType.PDF.getName())));
entry1C.setFiles(Collections.singletonList(new LinkedFile("Minimalnote 2", "minimal-note2.pdf", StandardFileType.PDF.getName())));
when(context.getFileDirectories(Mockito.any())).thenReturn(Collections.singletonList(Path.of("src/test/resources/org/jabref/logic/search")));
when(context.getFulltextIndexPath()).thenReturn(indexDir);
when(context.getDatabase()).thenReturn(database);
when(context.getEntries()).thenReturn(database.getEntries());

database.insertEntry(entry1C);

PdfIndexer indexer = PdfIndexer.of(context, filePreferences);
search = PdfSearcher.of(context);

indexer.createIndex();
indexer.addToIndex(context);
}

private void initializeDatabaseFromPath(Path testFile) throws IOException {
ParserResult result = new BibtexImporter(mock(ImportFormatPreferences.class, Answers.RETURNS_DEEP_STUBS), new DummyFileUpdateMonitor()).importDatabase(testFile);
context = new BibDatabaseContext(result.getDatabase(), result.getMetaData());

database = context.getDatabase();
search = PdfSearcher.of(context);
}

@Test
public void testEmptyLibrarySearch() throws IOException, URISyntaxException {
initializeDatabaseFromPath(Path.of(SearchFunctionalityTest.class.getResource("empty.bib").toURI()));

List<BibEntry> matches = new DatabaseSearcher(new SearchQuery("Test", EnumSet.noneOf(SearchRules.SearchFlags.class)), database).getMatches();
assertEquals(Collections.emptyList(), matches);
}

@Test
public void testUpperAndLowerWordSearch() throws IOException, URISyntaxException {
initializeDatabaseFromPath(Path.of(SearchFunctionalityTest.class.getResource("test-library-A.bib").toURI()));

//Positive search test
List<BibEntry> matches = new DatabaseSearcher(new SearchQuery("Test", EnumSet.noneOf(SearchRules.SearchFlags.class)), database).getMatches();
assertEquals(List.of(entry1A, entry2A, entry3A, entry4A, entry5A), matches);
//Negative search test
matches = new DatabaseSearcher(new SearchQuery("Best", EnumSet.noneOf(SearchRules.SearchFlags.class)), database).getMatches();
assertEquals(Collections.emptyList(), matches);
}

@Test
public void testSimpleSingleFieldSearch() throws IOException, URISyntaxException {
initializeDatabaseFromPath(Path.of(SearchFunctionalityTest.class.getResource("test-library-A.bib").toURI()));

//Positive search test
List<BibEntry> matches = new DatabaseSearcher(new SearchQuery("author=Test", EnumSet.noneOf(SearchRules.SearchFlags.class)), database).getMatches();
assertEquals(List.of(entry1A, entry2A, entry3A, entry4A, entry5A), matches);
//Negative search test
matches = new DatabaseSearcher(new SearchQuery("author=Case", EnumSet.noneOf(SearchRules.SearchFlags.class)), database).getMatches();
assertEquals(Collections.emptyList(), matches);
}

@Test
public void testSimpleMultipleFieldSearch() throws IOException, URISyntaxException {
initializeDatabaseFromPath(Path.of(SearchFunctionalityTest.class.getResource("test-library-A.bib").toURI()));

//Positive search test
List<BibEntry> matches = new DatabaseSearcher(new SearchQuery("author=Test and title=Case", EnumSet.noneOf(SearchRules.SearchFlags.class)), database).getMatches();
assertEquals(List.of(entry1A, entry2A, entry3A, entry4A, entry5A), matches);
//Negative search test
matches = new DatabaseSearcher(new SearchQuery("author=Test and title=Test", EnumSet.noneOf(SearchRules.SearchFlags.class)), database).getMatches();
assertEquals(Collections.emptyList(), matches);
}

@Test
public void testSensitiveWordSearch() throws IOException, URISyntaxException {
initializeDatabaseFromPath(Path.of(SearchFunctionalityTest.class.getResource("test-library-A.bib").toURI()));

//Positive search test
List<BibEntry> matches = new DatabaseSearcher(new SearchQuery("Test", EnumSet.of(SearchRules.SearchFlags.CASE_SENSITIVE)), database).getMatches();
assertEquals(List.of(entry1A), matches);
//Negative search test
matches = new DatabaseSearcher(new SearchQuery("TesT", EnumSet.of(SearchRules.SearchFlags.CASE_SENSITIVE)), database).getMatches();
assertEquals(Collections.emptyList(), matches);
}

@Test
public void testSensitiveMultipleFieldSearch() throws IOException, URISyntaxException {
initializeDatabaseFromPath(Path.of(SearchFunctionalityTest.class.getResource("test-library-A.bib").toURI()));

//Positive search test
List<BibEntry> matches = new DatabaseSearcher(new SearchQuery("author=Test and title=cASe", EnumSet.of(SearchRules.SearchFlags.CASE_SENSITIVE)), database).getMatches();
assertEquals(List.of(entry1A), matches);
//Negative search test
matches = new DatabaseSearcher(new SearchQuery("author=Test and title=case", EnumSet.of(SearchRules.SearchFlags.CASE_SENSITIVE)), database).getMatches();
assertEquals(Collections.emptyList(), matches);
}

@Test
public void testSimpleRegularExpression() throws IOException, URISyntaxException {
initializeDatabaseFromPath(Path.of(SearchFunctionalityTest.class.getResource("test-library-B.bib").toURI()));

//Positive search test
List<BibEntry> matches = new DatabaseSearcher(new SearchQuery("[/9]", EnumSet.of(SearchRules.SearchFlags.REGULAR_EXPRESSION)), database).getMatches();
assertEquals(List.of(entry4B), matches);
//Negative search test
matches = new DatabaseSearcher(new SearchQuery("[/8]", EnumSet.of(SearchRules.SearchFlags.REGULAR_EXPRESSION)), database).getMatches();
assertEquals(Collections.emptyList(), matches);
}

@Test
public void testSensitiveRegularExpression() throws IOException, URISyntaxException {
initializeDatabaseFromPath(Path.of(SearchFunctionalityTest.class.getResource("test-library-B.bib").toURI()));

//Positive search test
List<BibEntry> matches = new DatabaseSearcher(new SearchQuery("\\bCase\\b", EnumSet.of(SearchRules.SearchFlags.REGULAR_EXPRESSION, SearchRules.SearchFlags.CASE_SENSITIVE)), database).getMatches();
assertEquals(List.of(entry1B), matches);
//Negative search test
matches = new DatabaseSearcher(new SearchQuery("\\bCas\\b", EnumSet.of(SearchRules.SearchFlags.REGULAR_EXPRESSION, SearchRules.SearchFlags.CASE_SENSITIVE)), database).getMatches();
assertEquals(Collections.emptyList(), matches);
}

@Test
public void testSimplePDFFulltextSearch() throws IOException, URISyntaxException {
initializeDatabaseFromPath(Path.of(SearchFunctionalityTest.class.getResource("test-library-C.bib").toURI()));

PdfSearchResults result = search.search("This is a short sentence, comma included.", 10);
assertEquals(6, result.numSearchResults());
}
}


File renamed without changes.