diff --git a/README.md b/README.md index cfe0aa72..5ba9bea3 100644 --- a/README.md +++ b/README.md @@ -25,4 +25,4 @@ docker compose up ``` This will build and run containers for WebOCD service and arangodb, which is required for the service to run properly. -You can adjust the database container default password by modifying docker-compose.yml. Keep in mind to also change the config files [config_test.properties](ocd/arangoDB/config_test.properties) and [standard_config.properties](ocd/arangoDB/standard_config.properties) so that WebOCD tests as well as the service can use the database. \ No newline at end of file +You can adjust the database container default password by modifying docker-compose.yml. Keep in mind to also change the config file [config.properties](ocd/arangoDB/config.properties), so that WebOCD tests as well as the service can use the database. \ No newline at end of file diff --git a/docs/pages/index.md b/docs/pages/index.md index 555a8103..ce6c1e12 100644 --- a/docs/pages/index.md +++ b/docs/pages/index.md @@ -28,10 +28,10 @@ The OCD Service is publicly accessible via this [link](http://webocd.dbis.rwth-a Along with the service we provide a web client which also has its own [project](https://github.com/rwth-acis/OCD-Web-Client). The client can only be used with a valid Learning Layers account. We recommend to use the service along with the web client. -We additionally provide a brief [Integration Tutorial](/integration/) that shows you the usage of the most important requests so that you can get started with the OCD Service more quickly. +We additionally provide a brief [Integration Tutorial](/REST-OCD-Services/pages/wiki/integration/) that shows you the usage of the most important requests so that you can get started with the OCD Service more quickly. A more detailed usage examples of WebOCD Service, using the WebOCD webclient can be found here [link](/REST-OCD-Services/pages/tutorials/login). ## The OCD Service -The service is developed using the Eclipse IDE, so we recommend you to stick to that tool for any work on this project. +The service is developed using the IntelliJ IDEA, so we recommend you to stick to that tool for any work on this project. Please refer to the following pages in order to obtain more information about the OCD Service. @@ -41,9 +41,10 @@ Please refer to the following pages in order to obtain more information about th + [User Management and Authentication](/REST-OCD-Services/pages/wiki/user-management-and-authentication) + [Deploying the OCD Service](/REST-OCD-Services/pages/wiki/service-deployment) + [Database Configuration](/REST-OCD-Services/pages/wiki/database-configuration) ++ [Tutorials using WebOCD Service and Webclient](/REST-OCD-Services/pages/tutorials/login) ### Important -The OCD Service is largely based on the commercial **yFiles** library. You will only be able to run it yourself if you have access to the archive _y.jar_. The Chair i5 of RWTH Aachen University has a license for that library. If you are affiliated with that institute, please contact somebody to obtain the library. Also, keep in mind that the archive mentioned above must under no circumstances be publicly deployed or otherwise made publicly available. Please refer to [Deploying the OCD Service](/REST-OCD-Services/pages/wiki/service-deployment) for more information. +The WebOCD Service uses **ArangoDB** database, which has to be separately installed on the machine where the service runs. Alternatively, docker can be used to build the WebOCD Service together with the database. To build and run WebOCD Service with Docker, refer to [link](/REST-OCD-Services/pages/wiki/running#docker). -The service also currently requires **Java JDK 14** to be built and most likely also to run it. However, there also exists an older version for JDK 8 (tag [1.0.0](https://github.com/rwth-acis/REST-OCD-Services/tree/1.0.0)) which is still functional in case you need it. +The service also currently requires **Java JDK 17** to be built and most likely also to run it. However, there also exists an older version for JDK 8 (tag [1.0.0](https://github.com/rwth-acis/REST-OCD-Services/tree/1.0.0)) which is still functional in case you need it. diff --git a/docs/pages/wiki/build-process.md b/docs/pages/wiki/build-process.md index 5b031fc3..38733e95 100644 --- a/docs/pages/wiki/build-process.md +++ b/docs/pages/wiki/build-process.md @@ -4,59 +4,27 @@ title: Build Process --- # Introduction -The OCD Service is built via Gradle. We recommend a version similar to 6.8, yet others may also work. The project has mostly been developed with the Eclipse IDE using the gradle buildship plugin (most eclipse IDEs already include this), yet IDEs like IntelliJ are actively used as well. Finally note that you will likely need Java version 14 for the build to work without problems. - -There also exists an older version using Apache Ant which is described at the bottom. Here, there are two build files, _build.xml_, and _ocd_build.xml_. Please do only use the _ocd_build.xml_. Note that Eclipse might illustrate errors for the build file because apparently it is not able to resolve the external target dependencies to the second build file correctly. This should not cause an issue. It is highly recommended that you build the project with Eclipse (To use Java 8 for the older version you may have to go with the Eclipse IDE for Java EE Developers variant) as this is the environment used during development and therefore the best tested one. +The OCD Service is built via Gradle. We recommend a version similar to 7.3, yet newer versions may also work. The project has mostly been developed with the IntelliJ IDEA. You can import the project into IntelliJ as a Gradle project. Finally note that you will likely need Java version 17 for the build to work without problems. # Executing the Build Process ## First Build -When you are going to run the build process the very first time, you will need two manually obtain two libraries: -+ _y.jar_ forms part of the commercial yFiles library by yWorks which is not publicly available. Therefore you will have to add it manually to the project. The Chair i5 of RWTH Aachen University has a license for the library, so please contact somebody to obtain access. This will obviously only be possible if you are somehow affiliated with the Chair. -+ _ysvg.jar_ is an SVG writer for yFiles Graph2D objects that makes it possible to add useful features to generated SVG diagrams. The library is used for the visualization of graphs and covers. -Both libraries then have to be placed into the _yFiles_ folder. - -Now, you just have to import the project as a gradle project via eclipse or simply open in via IntelliJ. After the first build, eclipse might still give you errors saying the yFiles jars are missing. To fix this, simply refresh the gradle project via the option you get from right clicking the folder. +When you are going to run the build process the very first time, you just have to import the project as a gradle project IntelliJ and execute the gradle *build* task. Make sure to adapt the database configuration _ocd/arangoDB/config.properties_ to fit your running ArangoDB instance. After the build, you can then either start the _start_network.bat_ or _start_network.sh_ files in the bin folder to get the service running, see [Running the OCD Service](/REST-OCD-Services/pages/wiki/running) for more information. ## General Build -There are numerous targets which can be run. This summarizes the most important ones: +There are numerous gradle tasks which can be executed. This summarizes the most important ones: + _jar_: Generates the service jar in the _service_ and a copy in the _rest\_ocd\_services/export/jars_ directory. This archive is required for running the service. + _test_: Runs all jUnit tests . The test reports are stored in _rest\_ocd\_services/export/test_reports_. -+ _yguard_: Generates an obfuscated service jar in the _rest\_ocd\_services/export/obfuscated_ directory. This is required for public service deployment. Please refer to [Deploying the OCD Service](/REST-OCD-Services/pages/wiki/service-deployment) for more information. A log file of the obfuscation process is stored under _ocd/yGuard/log.xml_. + _build_: The target you'll likely be running most of the time. This executes the build process and generates a jar as well as the executables. ++ _startscripts:_ Generates startscripts into the bin folder that are used to start the service. ++ _generateAgents_: Generates the user and service agents as specified in _build.gradle_. The users will be loaded when the service (or more precisely the underlying LAS2peer node) is launched. ++ _clean_: Clean up files generated from the build. This should be executed before rebuilding. -# Potential Issues - -Currently, the build might fail if a version of Java not equal to 14 (or 8 for the older version of the service) is used, even with eclipse in compliance mode for that version, and running the service with a version other than this one is also likely to cause errors. - -## Linux Users -The system was mainly tested on Windows so far. For Linux, there is still a small issue with the old Ant Build. In case you receive an exception that the database (_db_) could not be created, please do manually remove the directory _db_ from the _ocd_ directory before the build. - - -# Executing the Old Ant Build Process - -## First Build -When you are going to run the build process the very first time, please make sure to run the target _get_deps_ in order to obtain all dependencies. - -Due to the yFiles library being commercial, there are three dependencies which will be missing and must be added manually to the _lib_ folder: -+ _y.jar_ forms part of the commercial yFiles library by yWorks which is not publicly available. Therefore you will have to add it manually to the project. The Chair i5 of RWTH Aachen University has a license for the library, so please contact somebody to obtain access. This will obviously only be possible if you are somehow affiliated with the Chair. -+ _yguard.jar_ is a free obfuscation tool by yWorks which is only required for public deployment of the service. Anybody can download it from this [link](https://www.yworks.com/downloads#yGuard). Refer to [Deploying the OCD Service](/REST-OCD-Services/pages/wiki/service-deployment) for more information. You can probably do without it. -+ _ysvg.jar_ is an SVG writer for yFiles Graph2D objects that makes it possible to add useful features to generated SVG diagrams. The library is used for the visualization of graphs and covers. -## Linux Users -Please make sure that the program _ocd/lfr/LfrBenchmarkLinux_ is executable (via `chmod +x `). +# Potential Issues +Currently, the build might fail if a version of Java not equal to 17 is used. -## General Build -There are numerous targets which can be run. This summarizes the most important ones: -+ _jar_: Generates the service jar in the _service_ and a copy in the _export/jars_ directory. This archive is required for running the service. -+ _generate_configs_: Generates the user and service agents as specified in _etc/ant_configuration_. The users will be loaded when the service (or more precisely the underlying LAS2peer node) is launched. -+ _junit_: Runs all jUnit tests (except _src/test/i5/las2peer/services/ocd/DatabaseInitializer.java_, take look at the target _setup_database_ for more information). The test reports are stored in _export/test_reports_. -+ _startscripts:_ Generates startscripts into the bin folder that are used to start the service. -+ _obfuscate_: Generates an obfuscated service jar in the _export/obfuscated_ directory. This is required for public service deployment. Please refer to [Deploying the OCD Service](/REST-OCD-Services/pages/wiki/service-deployment) for more information. A log file of the obfuscation process is stored under _ocd/yGuard/log.xml_. -+ _setup_database_: This is intended for automatically populating the database for presentation or debugging purposes. It is not required in any way to actually make the database work. The database is set up by running _src/test/i5/las2peer/services/ocd/DatabaseInitializer.java_ as a jUnit test. You can adapt the code in any way that suits your needs. -+ _all_: Runs all of the above except _setup_database_. +Running WebOCD Service might cause *AgentAccessDeniedException* if the *build* task was performed multiple times without having performed the *clean* task inbetween. If this occurs, simply do the *clean* task followed by the *build* task and the issue should be solved. -## Building with IntelliJ -For IntelliJ, you can follow the above guide as well. For correct indexing in the IDE you however need to add the lib folder as a library via right click. You might also want to double check that build.xml isn't selected as an ant build file instead of ocd_build.xml diff --git a/docs/pages/wiki/database-configuration.md b/docs/pages/wiki/database-configuration.md index 8607f2c5..8e16a67c 100644 --- a/docs/pages/wiki/database-configuration.md +++ b/docs/pages/wiki/database-configuration.md @@ -3,11 +3,8 @@ layout: page title: Database Configurations --- -# JPA and persistence.xml -The service makes use of the Java Persistence API (JPA) specification to simplify database access. The JPA implementation that is being used is EclipseLink. If you want to use the service with another database or change some aspects of the database configuration you must adapt the following files accordingly. -+ _rest\_ocd\_services/src/main/java/META-INF/persistence.xml_: This database configuration is used for the actual execution of the service through the service jar (e.g. when running _bin/start_network.bat_) -+ _rest\_ocd\_services/src/main/java/META-INF/testing/persistence.xml_: This database configuration is used during jUnit testing. +# ArangoDB +The service makes use of ArangoDB database. A database instance should be running before executing the build process. Otherwise, the database related tests will fail. +Database instance details that WebOCD Service uses can be configured by changing the _ocd/arangoDB/config.properties_ file. -The file locations are different in the old ant build: -+ _ocd/eclipselink/persistence.xml_ for the execution -+ _src/META-INF/persistence.xml_ for testing \ No newline at end of file +If the database is not available on the machine where WebOCD Service should run, it is possible to use Docker, without installing/running the database first. For this refer to [Running the OCD Service](/REST-OCD-Services/pages/wiki/running#docker). diff --git a/docs/pages/wiki/integration.md b/docs/pages/wiki/integration.md index f52dade5..8fd3f4e6 100644 --- a/docs/pages/wiki/integration.md +++ b/docs/pages/wiki/integration.md @@ -4,13 +4,15 @@ title: Integration --- # Introduction -This tutorial will give you a brief explanation about how to handle the OCD Service. It explains a few basic requests for e.g. uploading a graph, retrieving the saved graphs, creating a cover from a graph, ... +This tutorial will give you a brief explanation about how to handle the OCD Service. It explains a few basic requests for e.g. uploading a graph, retrieving the saved graphs, creating a cover from a graph etc. -However this is only a fraction of what the service actually allows. Other functionalities include for instance the execution of metrics to determine the quality of the calculated covers or the computation of benchmark graphs and covers for the generation of standardized testing data. +However, this is only a fraction of what the service actually allows. Other functionalities include for instance the execution of metrics to determine the quality of the calculated covers or the computation of benchmark graphs and covers for the generation of standardized testing data. At the bottom, an overview of all possible requests is given. For a more detailed description of the requests you should refer to the OCD Service's source code. All responses other than actual graph and cover data (which has the specified output format), as well as JSON visualization data are sent in XML format. +For more detailed information about using WebOCD Service, refer to the [tutorials](/REST-OCD-Services/pages/tutorials/login), which explains usage of WebOCD Service with the [WebClient](https://github.com/rwth-acis/OCD-Web-Client) + # Graphs ## List Available Graphs @@ -115,7 +117,7 @@ After that, make a post request to run the algorithm with the following paramete + _name_: Email_Cover + _algorithm_: _SPEAKER_LISTENER_LABEL_PROPAGATION_ALGORITHM_ + _body_: `` -`memorySize100probabilityThreshold0.15` + `memorySize100probabilityThreshold0.15` + _contentWeighting_: false + _componentNodeCountFilter_: 0 diff --git a/docs/pages/wiki/project-structure.md b/docs/pages/wiki/project-structure.md index 020df142..cc7096e9 100644 --- a/docs/pages/wiki/project-structure.md +++ b/docs/pages/wiki/project-structure.md @@ -7,19 +7,18 @@ title: Project Structure This page gives you an overview over the file structure of the OCD Service Project. The focus will be set on the _ocd_ directory where most service specific files are located (aside from the source code). Some other files will be discussed as well. For more information on the remaining files you may refer to the LAS2peer Template Project or LAS2peer documentation. # Source Code -The actual source code of the service is located under _rest\_ocd\_services/src/main/java_. The most important class here is probably the _i5/las2peer/services/ocd/ServiceClass.java_ which contains the entire Service API. You will also find the _META-INF_ directory with the persistence.xml files. For more information on this aspect take a look at [Database Configuration](/REST-OCD-Services/pages/wiki/database-configuration). +The actual source code of the service is located under _rest\_ocd\_services/src/main/java_. The most important class here is probably the _i5/las2peer/services/ocd/ServiceClass.java_ which contains the entire Service API. -The source code for the jUnit tests is in _rest\_ocd\_services/src/test_. This directory includes also the _src/test/i5/las2peer/services/ocd/DatabaseInitializer.java_ which can be used for database initialisation. For more information please refer to [Build Process](/REST-OCD-Services/pages/wiki/build-process). +The source code for the jUnit tests is in _rest\_ocd\_services/src/test_. + +ArangoDB database configuration file *config.properties* is located in *ocd/arangoDB* directory. This file can be modified to adjust database used for WebOCD Service as well as tests. For more information please refer to [Build Process](/REST-OCD-Services/pages/wiki/database-configuration). # OCD Directory The _ocd_ directory contains most files apart from the source code which are specific to the OCD Service. You will find the following subdirectories. -+ _db_: Contains the database of the JPA provider. ++ _arangoDB_: Contains the database configuration file. + _eclipselink_: Any log files for database creation or dropping created by the EclipseLink JPA persitence provider will be stored here. -+ _lfr_: Provides an application from Lancichinetti for calculating the LFR benchmarks he introduced (for directed and weighted graphs). It is compiled once for each Linux and Windows. + _test_: Contains a folder with data files for jUnit test input. An additional folder for test output files may be created during the build. -+ _yGuard_: Any log files of the obfuscation process will be stored here. Please refer to [Deploying the OCD Service](Deploying the OCD Service) for more information. -+ (_ivy_ if using the old ant build: Contains an _ocd_ivy.xml_ file to load the service's dependencies via Ivy. Note that there is an additional _ivy.xml_ and _ivy.settings_ file for dependencies required by LAS2peer under _etc/ivy_ from the root directory.) # Other Important Resources + _bin_: Comprises scripts for launching the service and generating Agents. @@ -27,6 +26,3 @@ The _ocd_ directory contains most files apart from the source code which are spe + _service_: This directory contains the service jar. Please refer to [Build Process](/REST-OCD-Services/pages/wiki/build-process) for more information. + _bin/start_network.bat_ and _bin/start_network.sh_: Skripts for launching the OCD Service. Please refer to [Running the OCD Service](/REST-OCD-Services/pages/wiki/running) for more information. + _etc/startup/passphrases.txt_: Password storage for preinstalled users. Please refer to [User Management and Authentication](/REST-OCD-Services/pages/wiki/user-management-and-authentication) for more information. -+ (_ocd_build.xml_ if using the old ant build: The build file for the OCD Service. Please do only make use of this build file. Please refer to [Build Process](/REST-OCD-Services/pages/wiki/build-process) for more information.) -+ (_ocd.conf_ if using the old ant build: Upstart script for the OCD Service. Please refer to [Deploying the OCD Service](/REST-OCD-Services/pages/wiki/service-deployment) for more information.) -+ (_etc/ant_configuration/user.properties_ if using the old ant build: Ant property file for user agent generation. Please refer to [User Management and Authentication](/REST-OCD-Services/pages/wiki/user-management-and-authentication) for more information.) \ No newline at end of file diff --git a/docs/pages/wiki/running.md b/docs/pages/wiki/running.md index b105fe31..6408fd35 100644 --- a/docs/pages/wiki/running.md +++ b/docs/pages/wiki/running.md @@ -11,12 +11,20 @@ Before you start the service you might want to configure the _start_network.bat_ Use `chmod +x ` to allow the execution of the scripts. # Running the Service -Once the database is accessible, you can launch the service on a single node LAS2peer network with the scripts _start_network.bat_ or _start_network.sh_ which are located in the _bin_ directory. The service should be running within a few seconds and be available as a REST service under the basepath _`http://127.0.0.1:8080/ocd/`_ (Note that your browser will display a 404 if you type in this adress, to see if the service is running check the address without the _`/ocd`_ part instead). You can terminate the process by either typing _exit_ in the interactive console or of course also by any other common means. +Once the arangoDB instance is accessible, you can launch the service on a single node LAS2peer network with the scripts _start_network.bat_ or _start_network.sh_ which are located in the _bin_ directory. The service should be running within a few seconds and be available as a REST service under the basepath _`http://127.0.0.1:8080/ocd/`_ (Note that your browser will display a 404 if you type in this address, to see if the service is running check the address without the _`/ocd`_ part instead). You can terminate the process by either typing _exit_ in the interactive console or of course also by any other common means. It is recommended that you use the OCD Service in combination with the web client at https://github.com/rwth-acis/OCD-Web-Client, which will take care of formulating the right requests and especially the login process for you as well as provide JSON visualization. +# Docker +To build and run WebOCD Service using Docker, you can navigate to the REST-OCD-Service directory and execute +``` +docker compose up +``` +This will build and run containers for WebOCD service and ArangoDB database, which is required for the service to run properly. + +You can adjust the database container default password by modifying *docker-compose.yml*. Keep in mind to also change the config file *ocd/arangoDB/config.properties*, so that WebOCD tests as well as the service can use the database. # Potential Issues -Your service might run into the _java.lang.outOfMemoryError_, this indicates that the heap size of the program is most likely to small. You can increase the maximum size by adding the -Xmx parameter in the _start_network_ files. For information on how to use it, please refer to the official Documentation [here](https://docs.oracle.com/cd/E13150_01/jrockit_jvm/jrockit/jrdocs/refman/optionX.html). +Your service might run into the _java.lang.outOfMemoryError_, this indicates that the heap size of the program is most likely to small. You can increase the maximum size by adding the -Xmx parameter in the _start_network_ files. For information on how to use it, please refer to the official Documentation (https://docs.oracle.com/cd/E13150_01/jrockit_jvm/jrockit/jrdocs/refman/optionX.html) If you are getting a warning about encryption problems then you most likely have to change the Security Policy Files of your JRE. This can also happen after a JRE update. Please refer to the Las2Peer Template Project for more information. \ No newline at end of file diff --git a/docs/pages/wiki/service-deployment.md b/docs/pages/wiki/service-deployment.md index 435cd0fb..6050e9bb 100644 --- a/docs/pages/wiki/service-deployment.md +++ b/docs/pages/wiki/service-deployment.md @@ -4,10 +4,7 @@ title: Service Deployment --- # Building -If you want to deploy the OCD Service to a server, the service must first be build. There are two basic alternatives. Either you directly build the project on the machine where it should be deployed, or you build it on some other machine and then move the files to their designated location. You will have to execute the _jar_ target in order to create the service jar. Please refer to [Build Process](Build Process) for more information. Make sure you also generate the user agents for any required user accounts through the target by altering the gradle build file accordingly and running the _jar_ target, if needed. - -# Public Deployment: Obfuscation -The OCD Service makes heavy use of the archive _y.jar_ which is part of the commercial yFiles framework by yWorks. The Chair i5 of RWTH Aachen University has a license allowing the usage of this software. However, it is not allowed to make this library publicly accessible to others. Therefore, if you intend to deploy the OCD Service on a public machine (accessible to people which are not affiliated with the chair) you must obfuscate that library. For this purpose you will require the archive _yguard.jar_ which is offered for free by yWorks. The _yguard_ target will help you with the obfuscation. Please refer to [Build Process](Build Process) for more information. For the obfuscation requirements concerning yFiles and more information on yGuard please refer to the corresponding documentation by yWorks. Also make sure that the yGuard log file under _ocd/yGuard/log.xml_ is not being deployed. +If you want to deploy the OCD Service to a server, the service must first be build. There are two basic alternatives. Either you directly build the project on the machine where it should be deployed, or you build it on some other machine and then move the files to their designated location. You will have to execute the _jar_ target in order to create the service jar. Please refer to [Build Process](/REST-OCD-Services/pages/wiki/build-process) for more information. Make sure you also generate the user agents for any required user accounts through the target by altering the gradle build file accordingly and running the _jar_ target, if needed. # Additional Aspects There are a few additional aspects that should be taken under consideration. First of all, there are several project folders which do not need to be deployed. After completing the build, only the following resources are relevant for the service execution, any others may be removed. @@ -23,8 +20,7 @@ There are a few additional aspects that should be taken under consideration. Fir + README.md + (ocd.conf in the old and build) -Please keep in mind again that in the case of a public deployment the yGuard log file must not be deployed. -Also, when deploying on a Linux system, make sure with `chmod +x ` that the execution of all relevant scripts and programs is allowed. Besides the scripts to launch the service and the database this includes also the LFR Benchmark Application. Please refer to [Project Structure](Project Structure) for more information. +Also, when deploying on a Linux system, make sure with `chmod +x ` that the execution of all relevant scripts and programs is allowed. Please refer to [Project Structure](/REST-OCD-Services/pages/wiki/project-structure/) for more information. Remember also that the Security Policy Files of the JRE being used on the server will in some cases have to be changed, please refer to the LAS2peer Template Project for more information. \ No newline at end of file diff --git a/docs/pages/wiki/user-management-and-authentication.md b/docs/pages/wiki/user-management-and-authentication.md index 69d497e9..1655b3a4 100644 --- a/docs/pages/wiki/user-management-and-authentication.md +++ b/docs/pages/wiki/user-management-and-authentication.md @@ -9,7 +9,7 @@ This part deals with manually adding users which should not be necessary if you # User Management -You can preinstall up to three users via the Ant build target _generate_configs_. For that purpose the users must be defined in the Ant property file _etc/ant_configuration/user.properties_ and the corresponding password in _etc/startup/passphrases.txt_. For preinstalling more users you will have to adapt the _build.xml_ in the root directory accordingly. +You can preinstall up to three users via the gradle task _generateAgents_. For that purpose the users must be defined in the *gradle.properties* file. It is also possible to manually add LAS2peer user agents to an existing network. Please refer to the LAS2peer Template Project and LAS2peer for more information.