Prepare Release (#174)

* [CCP-218] Changes for displaying merged ids and merged concepts. (#163)

When user searches with the word net id that has been merged, the merged
concept will be displayed, Previously the wrapper of the word net was displayed but from
now the merged concept itself will be displayed.

Also the RESTFul apis will display the merged ids of the concepts.

* [CCP-210] Concept search requires atleast one search parameter (#166)

* [CCP-210] Concept search requires atleast one search parameter

From now on concept search rest api requires atleast one search parameter. If no
parameter is provided, the system returns the error message "No search condition
present" and return 400 http status code.

Previously the system went into 500 error when no parameter is provided.

* [CCP-210] Review comments fix.

* Story/ccp 202 (#168)

* [CCP-174] Made session timeout as 4 hours (#138)

* [CCP-174] hotfix to master

Updated session timeout to 4 hours

* [CCP-174] Changed the comments to four hours

* Story/ccp 202 master (#158)

* [CCP-202] Integration test for master branch

* [CCP-202-master] Changes for test case for master branch

Added all the applicable test cases for master branch. Few test cases
such as json and validations are temporarily removed. Will be added
once the develop integration test branch is merged to master.

* [CCP-202-master] Removed unwanted files.

Removed unwanted files. Will be added when their corresponding
test cases are added in integration test.

* [CCP-202-master] Added security context for luceneDAO

* [CCP-202-master] Review comments fix

* [CCP-202] Added correct expected output.

These test cases will be fixed when develop branch is getting merged.

* [CCP-202-master] Removed word net database.

* [CCP-202-master] Added logger statements for downloads manager and integration test

* [CCP-202-master] Added log4j config file for
integration test

* Caught all exceptions and logged errors.

* [CCP-202-master] Created new file before loading from dropbox.

* Logger level changed to error for exceptions
Changed the way fileoutputstream is instantiated
Closing resources are handled in same try block.

* [CCP-202-master] Made sure security context is setup only
when required.

* [CCP-202-master] Changed the correct file name

* [CCP-202] Fixes for test case in master branch

* [CCP-202] Reverted to master branch code base.

* [CCP-202-master] Loggers are logged as error in exception catch blocks

* Fix XML

* [CCP-202] Changes for integration tests.

1) Most of the files had trailing spaces. Removed those spaces.
2) Made sure correct error message is returned when there are no entries in concept look up
3) For multiple entries returned in concept look up, alternative ids were missing in the output. Made
sure the json and xml files contains the alternative ids.

* [CCP-202] Removed duplicated files due to conflicts

* [CCP-202] unit test changes for correct error message when concept is not found

* [CCP-202] Changes for Concept look up IT test cases.

* [CCP-202] Changes for not considering ordering.

Ordering in output will not be tested.

* [CCP-202] Dependancy changes

* Bug/CCP-211 (#167)

* [CCP-211] Removed the backslash "/" in concept entry

Back slash has been removed in equalsTo and similarTo in concept entry. This has been
removed in setter because lucene gets the values from the object using reflection and not
the getters so changes are made while loading the values. Also from now on any insertion of
equalsTo and simialrTo will not be having "/".

Also any search with "/" at the end needs to be removed. This will be done using spring conversion
service.

* [CCP-211] Changes for concept search of equals and similar to

Before searching for equals and similar to, we can check if equals
and similar to contains a back slash in the end of the url. If so we
remove the backslash and search on lucene. Also while inserting the
values on to lucene we check for the backslash.

* [CCP-211] Formatting changes

* [CCP-211] Review comments changes

Handled multiple equalsTo and similarTo string values.
used PropertyDescriptor for getting and setting values.
Included exceptions in unit test cases.

* [CCP-211] Review comments changes.

Moved operators, similarTo and equalsTo to bean class. pre processing of
typeUri and numberOfPages in concept search class. This is because
typeUri and numberOfpAges needs dependency that needs to be
injected which is not possible in ConceptsearchParameters.

* [CCP-211] Review comments fix.

* [CCP-211] Changes for review comments.

Made sure correct operator is used while querying lucene.
Number of pages is loaded correctly.

* [CCP-211] Test case fix.

Removed iterating the transient fields using Property descriptor.

* [CCP-218] Changes for displaying merged ids and merged concepts. (#163)

When user searches with the word net id that has been merged, the merged
concept will be displayed, Previously the wrapper of the word net was displayed but from
now the merged concept itself will be displayed.

Also the RESTFul apis will display the merged ids of the concepts.

* [CCP-211] Review comments fixes.

Changed the if conditions to ternary operator.
Fixed a bug in getSimilarTo() in conceptEntry

* [CCP-211] Review comments changes

Added comments on why transient field modifier is used.
Used ternary operator to get the number of records per page.

* [CCP-211] Review comments changes

* [CCP-221] Changes for selecting the rows in datable. (#170)

* [CCP-221] Changes for selecting the rows in datable.

* [CCP-221] Review comments.

changed the wordnetId to wordnetid in jsp.
Added word field while creating concept wrapper.
Word details can be entered by user when concept wrapper is being
created.

* [CCP-221] Changes for setting the correct word.

Word field is set with the maximum word length.
When a selection is removed the word field is
updated accordingly.

* [CCP-221] Formatting changes

* [CCP-222] Deletion is done before merging (#172)

* [CCP-222] Deletion is done before merging

Deletion is done before merging, because once the concepts are merged, the change made
in the getConceptEntry in ConceptManager always fetches the merged concept. Because of
this instead of fetching the original concept and setting it to deleted, we set the delete
flag on the merged concept.

* [CCP-222] Pushed deletion after creating the concept.

As per best practice, deletion is performed after creating the merged
concepts. deletion is performed on the database that existed before
creating the merged concepts. If this is not done, when the code searches
for the word net concept, instead of returning the word net concept, the
system returns the merged concept. The same applies to local CCP concepts
as well.

* [CCP-222] Passed the concept entry instead of the id

Instead of passing the id and fetching the entry from the id
in delete method, the delete method accepts the concept entry instead
of the id and deletes the corresponding concept entry that has been
passed.

* [CCP-222] Made sure one method is used for deletion of concept

* Bug/CCP-228 (#173)

* [CCP-228] Changes for populating the data on error

Added a backing bean and changes to jQuery to populate the
selected word net ids for the wrappers.

* [CCP-228] Added validator for concept add wrapper.

Spring validator is added for concept wrapper screen. Previously errors
are passed by checking on the controller layer.

* [CCP-228] Review comments fix.

Added test cases for ConceptWrapperService
Added javadoc for ConceptWrapperService
Removed request parameter from addConcept in ConceptWrapperController.

* [CCP-228] Review comments changes

Renamed the method name in test case files/
removed the logger statement in js
Updated javadoc

* Bug/CCP-209 (#169)

* [CCP-209] Introduced the new short words

* [CCP-209] Changes for short word searching.

* [CCP-209] Formatting changes

* [CCP-209] Used boolean queries instead of query parser

* [CCP-209] Review comments fixes

* [CCP-209] Changes for integration test

* [CCP-209] Variable name changes

* [CCP-209] Integration test changes.

Made sure all the xml responses are validated using xml() method in MockMvc
Changed the ordering in multipleEntry return for xml.

* Made xml as string due to encoding problems

* Changed concept list to text field.

This is because if there are any spaces in the concept list such as
"Vogon web" , the phrase query is not working if the spaced words
are used using a string field. So changed to text field.

* [CCP-209] Removed the spaces in search.

* [CCP-209] Analyze the short term field with standard analyzer

* [CCP-209] Changed concept list to string field

* [CCP-209] Changed the non tokenized fields to use boolean query.

* [CCP-209] Fixed review comments

Removed the additional field name and used the untokenized with the existing
luceneFieldName.

* [CCP-209] Review comments changes

Introduced keywordAnalyzer for searching shorter words.

* [CCP-209] Review comments changes

Rename constant field name.

* [CCP-209] Added wild card query.

Users can perform search using the wild card characters in query.

* [CCP-209] Changes for stemming search.

Previous to this change stemming search was not performed. From now on
search on word field will return stem words as well as results. Sorting is
done based on the relevance.

Updates to integration tests to support stem searching.

* [CCP-209] Changes for wildcard search.

Added integration tests for wild card search and made sure results
are sorted based on relevancy in wild card search.

* [CCP-209] Changes for unit test cases.

Output of unit test cases are changed because of using LinkedhashMap
instead of hash map in concept lookup to retain ordering of results
from lucene.

* [CCP-209] Added print statement in integration test

* [CCP-209] Ignore Attribute order is set to true in
XMLUnit

* [CCP-209] Comparison of XML is done in a recursive fashion.

* [CCP-209] Added recursive validation for xml

Previously test case were failing due to ordering issue in xml. but
that is fixed by using XmlUnit RecursiveElementNameTextQualifier.
Spring in built uses XmlUnit but does not set this ElementQualifier.

* [CCP-227] Added a field for supporting wildcardsearch

* [CCP-209] Review comments fix

Merged the if and else condition. Extracted a method for building the query.
Added javadoc for wild card search query method.

* [CCP-209] Changes for searching words as per comments

When user searches for "scott miller", the search results will contain
"scott miller" "miller scott" "scott D.miller".

Need to fix short term search.

* [CCP-209] Fix for short word search.

included short word search.

* [CCP-209] Review comments fix.

Included wild card search in short term field search.

* [CCP-209] Removed wildcard search from short term field

* [CCP-209] Fix for integration tests

* [CCP-209] Added the search for description.

Description test case was not working fine due to the stop word a. Readded
the test case.

* [CCP-209] Review comments changes

Changes to if else block and removed unused code.

* [CCP-209] Removed blank lines.

* [CCP-227] Changes for deleting word net concepts. (#171)

* [CCP-227] Changes for editing a concept.

When a concept is edited, its wordnet entries are updated
accordingly. Previously when a concept wrapper is edited
the wordnet entries were not updated.

* [CCP-227] Changes for deleting the word net concept

A new method has been introduced in concept manager to delete
the concept directly without fetching the concept and deleting that wrapper.

* [CCP-227] Updates to javadoc review comments

* [CCP-227] Added description for the fields

* [CCP-227] Review comments changes

Instead of passing the concept edit bean, the word net details are passed to the
edit method.

* [CCP-227] Fixed merge conflicts issue

* [CCP-227] Removed the logic of getting entry in client

Client should just retrieve the concept entry. Removed the logic of retrieving
the word net entries and added it to the   manager layers.

* [CCP-227] Review comments changes

* [CCP-227] Review comments fix

* [CCP-227] Review comments fix.

* [CCP-227] Deleted the concept in index instead of updating in client.

* [CCP-227] Renamed method from deleteIndex to deleteFromIndex

Conceptpower+Spring/src/integration-test/java/edu/asu/conceptpower/IntegrationTest.java

@@ -1,6 +1,8 @@
 package edu.asu.conceptpower;
 
 import java.security.Principal;
+
+import org.custommonkey.xmlunit.XMLUnit;
 import org.junit.Before;
 import org.junit.runner.RunWith;
 import org.springframework.beans.factory.annotation.Autowired;
@@ -40,6 +42,7 @@ public String getName() {
     @Before
     public void setup() throws Exception {
         this.mockMvc = MockMvcBuilders.webAppContextSetup(this.wac).build();
+        XMLUnit.setIgnoreAttributeOrder(true);
         if (!isSetupDone) {
             this.mockMvc.perform(MockMvcRequestBuilders.post("/auth/indexConcepts").principal(principal));
             MvcResult mr = null;

Conceptpower+Spring/src/integration-test/java/edu/asu/conceptpower/rest/ConceptIDLookupIT.java

@@ -4,8 +4,12 @@
 import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
 
 import org.apache.commons.io.IOUtil;
+import org.custommonkey.xmlunit.Diff;
+import org.custommonkey.xmlunit.XMLAssert;
+import org.custommonkey.xmlunit.examples.RecursiveElementNameAndTextQualifier;
 import org.junit.Test;
 import org.springframework.http.MediaType;
+import org.springframework.test.web.servlet.MvcResult;
 import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;
 
 import edu.asu.conceptpower.IntegrationTest;
@@ -65,10 +69,13 @@ public void test_getConceptById_successForConceptWrapperInJson() throws Exceptio
     public void test_getConceptById_successForWordNetIdInXml() throws Exception {
         final String output = IOUtil
                 .toString(this.getClass().getClassLoader().getResourceAsStream("output/wordNetConcept.xml"));
-        this.mockMvc
-                .perform(MockMvcRequestBuilders.get("/Concept").param("id", "WID-02380464-N-01-polo_pony")
-                        .accept(MediaType.APPLICATION_XML_VALUE))
-                .andExpect(content().string(output)).andExpect(status().isOk());
+        MvcResult mvcResult = this.mockMvc.perform(MockMvcRequestBuilders.get("/Concept")
+                .param("id", "WID-02380464-N-01-polo_pony").accept(MediaType.APPLICATION_XML_VALUE))
+                .andExpect(status().isOk()).andReturn();
+        Diff xmlDifference = new Diff(output, mvcResult.getResponse().getContentAsString());
+        xmlDifference.overrideElementQualifier(new RecursiveElementNameAndTextQualifier());
+        XMLAssert.assertXMLEqual("Similarlity failed in test_getConceptById_successForWordNetIdInXml", xmlDifference,
+                true);
 
     }
 
@@ -84,20 +91,27 @@ public void test_getConceptById_invalidWordNetIdInXml() throws Exception {
     public void test_getConceptById_successForLocalConceptIdInXml() throws Exception {
         final String output = IOUtil
                 .toString(this.getClass().getClassLoader().getResourceAsStream("output/conceptForLocalId.xml"));
-        this.mockMvc
-                .perform(MockMvcRequestBuilders.get("/Concept").param("id", "CONdf62c00c-f4a9-4564-9dd6-c9b955650f3a")
-                        .accept(MediaType.APPLICATION_XML_VALUE))
-                .andExpect(content().string(output)).andExpect(status().isOk());
+        MvcResult mvcResult = this.mockMvc.perform(MockMvcRequestBuilders.get("/Concept")
+                .param("id", "CONdf62c00c-f4a9-4564-9dd6-c9b955650f3a").accept(MediaType.APPLICATION_XML_VALUE))
+                .andExpect(status().isOk()).andReturn();
+        Diff xmlDifference = new Diff(output, mvcResult.getResponse().getContentAsString());
+        xmlDifference.overrideElementQualifier(new RecursiveElementNameAndTextQualifier());
+        XMLAssert.assertXMLEqual("Similarlity failed in test_getConceptById_successForLocalConceptIdInXml",
+                xmlDifference,
+                true);
     }
 
     @Test
     public void test_getConceptById_successForConceptWrapperInXml() throws Exception {
         final String output = IOUtil
                 .toString(this.getClass().getClassLoader().getResourceAsStream("output/conceptWrapper.xml"));
-        this.mockMvc
-                .perform(MockMvcRequestBuilders.get("/Concept").param("id", "CONe7fbf694-5609-4691-bca8-916526c2ba6a")
-                        .accept(MediaType.APPLICATION_XML_VALUE))
-                .andExpect(content().string(output)).andExpect(status().isOk());
+        MvcResult mvcResult = this.mockMvc.perform(MockMvcRequestBuilders.get("/Concept")
+                .param("id", "CONe7fbf694-5609-4691-bca8-916526c2ba6a").accept(MediaType.APPLICATION_XML_VALUE))
+                .andExpect(status().isOk()).andReturn();
+        Diff xmlDifference = new Diff(output, mvcResult.getResponse().getContentAsString());
+        xmlDifference.overrideElementQualifier(new RecursiveElementNameAndTextQualifier());
+        XMLAssert.assertXMLEqual("Similarlity failed in test_getConceptById_successForConceptWrapperInXml",
+                xmlDifference, true);
     }
 
 }

Conceptpower+Spring/src/integration-test/java/edu/asu/conceptpower/rest/ConceptLookupIT.java

@@ -6,8 +6,12 @@
 import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
 
 import org.apache.commons.io.IOUtil;
+import org.custommonkey.xmlunit.Diff;
+import org.custommonkey.xmlunit.XMLAssert;
+import org.custommonkey.xmlunit.examples.RecursiveElementNameAndTextQualifier;
 import org.junit.Test;
 import org.springframework.http.MediaType;
+import org.springframework.test.web.servlet.MvcResult;
 import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;
 import org.springframework.test.web.servlet.result.MockMvcResultMatchers;
 
@@ -48,30 +52,63 @@ public void test_getWordNetEntry_successForMultipleEntryInJson() throws Exceptio
     public void test_getWordNetEntry_successForSingleEntryInXml() throws Exception {
         final String output = IOUtil
                 .toString(this.getClass().getClassLoader().getResourceAsStream("output/conceptLookUpWordNetEntry.xml"));
-        this.mockMvc
-                .perform(MockMvcRequestBuilders.get("/ConceptLookup/Douglas Weiner/noun")
-                        .accept(MediaType.APPLICATION_XML_VALUE))
-                .andExpect(content().string(output)).andExpect(status().isOk());
+        MvcResult mvcResult = this.mockMvc.perform(MockMvcRequestBuilders.get("/ConceptLookup/Douglas Weiner/noun")
+                .accept(MediaType.APPLICATION_XML_VALUE)).andExpect(status().isOk()).andReturn();
+        Diff xmlDifference = new Diff(output, mvcResult.getResponse().getContentAsString());
+        xmlDifference.overrideElementQualifier(new RecursiveElementNameAndTextQualifier());
+        XMLAssert.assertXMLEqual("Similarlity failed in test_getWordNetEntry_successForSingleEntryInXml",
+                xmlDifference,
+                true);
     }
 
     @Test
     public void test_getWordNetEntry_noResultsInXml() throws Exception {
         final String output = IOUtil
                 .toString(this.getClass().getClassLoader().getResourceAsStream("output/conceptEntryNotFound.xml"));
-        this.mockMvc
-                .perform(MockMvcRequestBuilders.get("/ConceptLookup/Gustav Robert Kirchhoff/verb")
-                        .accept(MediaType.APPLICATION_XML_VALUE))
-                .andExpect(content().string(output)).andExpect(status().isOk());
+        MvcResult mvcResult = this.mockMvc.perform(MockMvcRequestBuilders
+                .get("/ConceptLookup/Gustav Robert Kirchhoff/verb").accept(MediaType.APPLICATION_XML_VALUE))
+                .andExpect(status().isOk()).andReturn();
+        Diff xmlDifference = new Diff(output, mvcResult.getResponse().getContentAsString());
+        xmlDifference.overrideElementQualifier(new RecursiveElementNameAndTextQualifier());
+        XMLAssert.assertXMLEqual("Similarlity failed in test_getWordNetEntry_noResultsInXml",
+                xmlDifference, true);
     }
 
     @Test
     public void test_getWordNetEntry_successForMultipleEntryInXml() throws Exception {
         final String output = IOUtil.toString(
                 this.getClass().getClassLoader().getResourceAsStream("output/conceptLookUpForMultipletEntry.xml"));
-        this.mockMvc
-                .perform(MockMvcRequestBuilders.get("/ConceptLookup/Douglas/noun")
-                        .accept(MediaType.APPLICATION_XML_VALUE))
-                .andExpect(content().xml(output)).andExpect(status().isOk());
+
+        MvcResult mvcResult = this.mockMvc.perform(
+                MockMvcRequestBuilders.get("/ConceptLookup/Douglas/noun").accept(MediaType.APPLICATION_XML_VALUE))
+                .andExpect(status().isOk()).andReturn();
+        Diff xmlDifference = new Diff(output, mvcResult.getResponse().getContentAsString());
+        xmlDifference.overrideElementQualifier(new RecursiveElementNameAndTextQualifier());
+        XMLAssert.assertXMLEqual("Similarlity failed in test_getWordNetEntry_successForMultipleEntryInXml",
+                xmlDifference, true);
     }
 
+    @Test
+    public void test_getWordNetEntry_searchWithWildCardOneOrMoreCharactersInXml() throws Exception {
+        final String output = IOUtil.toString(
+                this.getClass().getClassLoader().getResourceAsStream("output/wildCardSearchConceptLookUp1.xml"));
+        MvcResult mvcResult = this.mockMvc.perform(
+                MockMvcRequestBuilders.get("/ConceptLookup/Dougl*/noun").accept(MediaType.APPLICATION_XML_VALUE))
+                .andExpect(status().isOk()).andReturn();
+
+        Diff xmlDifference = new Diff(output, mvcResult.getResponse().getContentAsString());
+        xmlDifference.overrideElementQualifier(new RecursiveElementNameAndTextQualifier());
+        XMLAssert.assertXMLEqual("Similarlity failed in test_getWordNetEntry_searchWithWildCardOneOrMoreCharactersInXml",
+                xmlDifference, true);
+    }
+
+    @Test
+    public void test_getWordNetEntry_searchWithWildCardOneOrMoreCharactersInJson() throws Exception {
+        final String output = IOUtil.toString(
+                this.getClass().getClassLoader().getResourceAsStream("output/wildCardSearchConceptLookUp1.json"));
+        this.mockMvc
+                .perform(MockMvcRequestBuilders.get("/ConceptLookup/Dougl*/noun")
+                        .accept(MediaType.APPLICATION_JSON_VALUE))
+                .andExpect(content().json(output)).andExpect(status().isOk());
+    }
 }