Skip to content
This repository has been archived by the owner on Nov 21, 2018. It is now read-only.

Merging in Claudia's LM Retrieval code #2

Open
wants to merge 96 commits into
base: master
Choose a base branch
from
Open

Merging in Claudia's LM Retrieval code #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
b596009
added basic JM language model
chauff Jul 16, 2013
f0c887b
added basic JM language model
chauff Jul 16, 2013
76bf3e8
added selective term statistics reading
chauff Jul 16, 2013
58fa597
removed settings folder
chauff Jul 16, 2013
c05593e
retrieval functionality
chauff Jul 17, 2013
9fa704a
retrieval functionality
chauff Jul 17, 2013
0b2f22c
updated retrieval functionality
chauff Jul 17, 2013
c8356a7
updated retrieval functionality
chauff Jul 17, 2013
ed36a3e
Updated retrieval functionality
chauff Jul 17, 2013
e38af82
updated retrieval code
chauff Jul 18, 2013
f4e18e4
LM retrieval model in semi-working form
chauff Jul 18, 2013
5d092f9
Updated retrieval model; baseline run result added
chauff Jul 19, 2013
864fd89
Updated retrieval model; baseline run result added
chauff Jul 19, 2013
bd866fa
Updated retrieval model; baseline run result added
chauff Jul 19, 2013
d4ff2ce
Added overlap Perl script
chauff Jul 19, 2013
2814ea9
Added overlap Perl script
chauff Jul 19, 2013
ee73c71
updated README
chauff Jul 19, 2013
057ed13
updated README
chauff Jul 19, 2013
47efb6d
updated README
chauff Jul 19, 2013
6aad52c
updated README
chauff Jul 19, 2013
8b9d164
updated README
chauff Jul 19, 2013
94368b3
updated README
chauff Jul 19, 2013
020a10c
updated README
chauff Jul 19, 2013
f5505dd
updated README
chauff Jul 19, 2013
6c5ab08
Non-working version of cosine based document filtering added
chauff Jul 19, 2013
b89df4d
Refined filtering
chauff Jul 22, 2013
a8fa533
Refined filtering
chauff Jul 22, 2013
42e8749
erging changes from lintool clueweb
chauff Jul 23, 2013
d7c0b03
fixing merge/push/pull issue
chauff Jul 23, 2013
93c50a4
updated for with working branch of lintool/clueweb
chauff Jul 23, 2013
c63c320
Updated README
chauff Jul 23, 2013
4fd0b07
Updated filtering for near-duplicates and spam scores
chauff Jul 23, 2013
7a3c70c
Added switch for Lucene Analyzer
chauff Jul 23, 2013
f57aaca
moved Analyzer code to new class
chauff Jul 24, 2013
425e341
Fixed Build*DocVectors (missaligned Lucene Analyzer)
chauff Jul 24, 2013
2722800
Updated README, new baseline run added
chauff Jul 25, 2013
d5869ea
Updated README, new baseline run added
chauff Jul 25, 2013
9516464
Updated README, new baseline run added
chauff Jul 25, 2013
dd0abbc
Updated README, new baseline run added
chauff Jul 25, 2013
5ece04f
Updated README, new baseline run added
chauff Jul 25, 2013
d7ac619
Updated README, new baseline run added
chauff Jul 25, 2013
f0367c4
Updated README, new baseline run added
chauff Jul 25, 2013
b2fc146
Updated README, new baseline run added
chauff Jul 25, 2013
2aa5cf9
Updated Spam scoring
chauff Jul 25, 2013
2fda847
Spam filtering revised, working version
chauff Jul 25, 2013
d9dc41b
Updated README with spam filtering instructions
chauff Jul 25, 2013
8b7d392
safety commit
chauff Jul 25, 2013
db38e53
Document extraction app added
chauff Jul 26, 2013
b7c327d
Document extraction app added
chauff Jul 26, 2013
fca438b
Fixed stream closing issues
chauff Jul 27, 2013
ae40796
Added Krovetz stemmer.
jimmy0017 Jul 27, 2013
dbfd070
Started relevance model implementation
chauff Jul 29, 2013
850bda6
Krovetz stemmer code from lintool-clueweb added
chauff Jul 29, 2013
f4b00ef
small fixes
chauff Jul 30, 2013
31cb4c5
eclipse settings added
chauff Jul 30, 2013
c8854c3
Added sanity check to TRECResultFileParser
chauff Jul 30, 2013
fa285c6
small fixes
chauff Jul 30, 2013
3d1016b
Tika boilerpipe HTML parser added
chauff Jul 31, 2013
72d33e0
updated HTML parser options
chauff Aug 3, 2013
b3892f5
RM3 adaptation
chauff Aug 3, 2013
a2a8525
updated HTML parser
chauff Aug 4, 2013
59fd49a
updated html parser
chauff Aug 4, 2013
55694c9
Merge pull request #1 from chauff/htmlParser
chauff Aug 4, 2013
c96abbf
Updated README
chauff Aug 4, 2013
9903285
Updated README
chauff Aug 4, 2013
dd991cf
RMModel working, started RMRetrieval
chauff Aug 6, 2013
9c0ce63
RMModel working, started RMRetrieval
chauff Aug 6, 2013
3748880
deleted unused settings files
chauff Aug 6, 2013
2a7a31c
RM retrieval model
chauff Aug 6, 2013
013e1bb
RM retrieval model
chauff Aug 6, 2013
b3939fe
updates on RMRetrieval
chauff Aug 7, 2013
a229ff8
Updated RMRetrieval app
chauff Aug 7, 2013
986047b
Fixed RMRetrieval
chauff Aug 8, 2013
aa26f1b
Fixed RMRetrieval
chauff Aug 8, 2013
f9181ca
manually resolving conflicts
chauff Aug 8, 2013
1218718
Merge pull request #2 from chauff/RMModel
chauff Aug 8, 2013
9bcf992
Manually fixing remaining conflicts
chauff Aug 8, 2013
65bb9fd
Manually fixing conflicts
chauff Aug 8, 2013
146cea5
Added description for RM1/RM3 retrieval runs
chauff Aug 9, 2013
c124190
Add a Bitdeli badge to README
bitdeli-chef Feb 13, 2014
4e99015
Merge pull request #3 from bitdeli-chef/master
chauff Feb 13, 2014
846d24e
Added additional helper progs, additional work on relevance model
chauff Jun 3, 2014
cb965a4
Updated RM
chauff Aug 13, 2014
4ce1a25
Exception handling improved
chauff Aug 15, 2014
d287827
Added contenteditable material
chauff Aug 21, 2014
a884243
Added comparison to Indri runs
chauff Aug 29, 2014
bcbd3c3
Move to Hadoop 2.X
chauff Jul 11, 2016
c2a934f
Separate parsing of <title> and <body> tags added.
chauff Jul 15, 2016
9e64452
Unit testing HTML parsers
chauff Jul 15, 2016
b184d7f
-
chauff Jul 15, 2016
650f3d0
-
chauff Jul 15, 2016
fa896af
-
chauff Jul 15, 2016
168aa2f
Extraction of all documents in the specified path; duplicate check
chauff Jul 15, 2016
7da0362
Document extraction for Solr
chauff Jul 15, 2016
6fbe9f9
clueweb09.app.DocumentExtractor documentation updated
chauff Jul 15, 2016
5c0e91c
Version nudge
chauff Jul 15, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
.classpath
.project
target/
.*.swp
5 changes: 5 additions & 0 deletions .settings/org.eclipse.core.resources.prefs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
eclipse.preferences.version=1
encoding//src/main/java=UTF-8
encoding//src/main/resources=UTF-8
encoding//src/test/java=UTF-8
encoding/<project>=UTF-8
1 change: 1 addition & 0 deletions .settings/org.eclipse.jdt.core.prefs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ org.eclipse.jdt.core.compiler.debug.localVariable=generate
org.eclipse.jdt.core.compiler.debug.sourceFile=generate
org.eclipse.jdt.core.compiler.problem.assertIdentifier=error
org.eclipse.jdt.core.compiler.problem.enumIdentifier=error
org.eclipse.jdt.core.compiler.problem.forbiddenReference=warning
org.eclipse.jdt.core.compiler.source=1.6
org.eclipse.jdt.core.formatter.align_type_members_on_columns=false
org.eclipse.jdt.core.formatter.alignment_for_arguments_in_allocation_expression=16
Expand Down
4 changes: 4 additions & 0 deletions .settings/org.eclipse.m2e.core.prefs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
activeProfiles=
eclipse.preferences.version=1
resolveWorkspaceProjects=true
version=1
6 changes: 6 additions & 0 deletions .settings/org.sonar.ide.eclipse.core.prefs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
analyseLocally=false
eclipse.preferences.version=1
extraProperties=
projectKey=org.clueweb\:clueweb-tools
serverUrl=http\://localhost\:9999
version=2
246 changes: 152 additions & 94 deletions README.md
View file
Open in desktop

Large diffs are not rendered by default.

Loading