Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

extending data extraction endpoints with /contributions #34

Merged
merged 48 commits into from
Nov 4, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
f8e00b1
adding of contributions controller
FabiKo117 Aug 25, 2020
8e67244
adding further endpoints under /contributions
FabiKo117 Aug 26, 2020
7a6c2c5
refactoring of response streams
FabiKo117 Aug 26, 2020
89d5391
adapting swagger nicknames of data extraction endpoints
FabiKo117 Aug 27, 2020
15f0ca6
adding of /latest endpoints
FabiKo117 Aug 27, 2020
325779c
javadoc fix
FabiKo117 Aug 27, 2020
923de6d
adding of processing for /contribution/latest
FabiKo117 Aug 27, 2020
b07248c
javadoc fix
FabiKo117 Aug 27, 2020
43acc24
adding of new request executor class
FabiKo117 Aug 27, 2020
b5bed76
variable naming changes and removing of unused class
FabiKo117 Aug 28, 2020
5d83671
adding output for different contribution types
FabiKo117 Aug 31, 2020
366e408
finishing a working /contributions implementation
FabiKo117 Sep 1, 2020
6106150
adding more descriptions and applying auto-formatting
FabiKo117 Sep 1, 2020
ec8fea3
fixing javadoc issues
FabiKo117 Sep 15, 2020
adc9edf
removing unused import and method
FabiKo117 Sep 17, 2020
b456cc3
fixing javadoc issues
FabiKo117 Sep 17, 2020
9818d08
fixing contribution processing
FabiKo117 Sep 18, 2020
fd3664a
adding and refactoring tests for data-extraction
FabiKo117 Sep 18, 2020
15cec05
adding support for deletions
FabiKo117 Sep 21, 2020
5b70f06
implementing deletion handling for /geometry endpoint
FabiKo117 Sep 22, 2020
1834fb0
changing name of streaming method
FabiKo117 Sep 22, 2020
15619fb
adding /contributions endpoint to logger
FabiKo117 Sep 22, 2020
14ba7c2
javadoc fixes
FabiKo117 Sep 22, 2020
315cbf0
refactoring of data-extraction tests
FabiKo117 Sep 22, 2020
e541bea
adding tests on deletions
FabiKo117 Sep 23, 2020
ad27bb7
changing geometry creation for deletions
FabiKo117 Sep 23, 2020
4560761
removing unnecessary code
FabiKo117 Sep 23, 2020
cba009b
minor code quality and comment changes
FabiKo117 Sep 23, 2020
f51512f
adding test on /contributions/latest
FabiKo117 Sep 29, 2020
a9e1179
refactoring of data-extraction
FabiKo117 Sep 29, 2020
6850c47
adding junit tests for /contributions get requests
Oct 1, 2020
1feb38e
adding junit test for /contributions post requests
Oct 2, 2020
84bed51
adding junit test for /contributions/latest post request
Oct 2, 2020
0f2de70
modifying spec names in swagger
FabiKo117 Oct 2, 2020
d818a68
modifying valid parameters for /contributions
FabiKo117 Oct 2, 2020
0989912
modifying description for contributions in swagger
FabiKo117 Oct 2, 2020
a10dce2
modifying swagger for /contributions
FabiKo117 Oct 2, 2020
2621afa
adding junit test
Oct 5, 2020
82ffeb4
adding base docs for /contributions
FabiKo117 Oct 5, 2020
e53251e
adding the /contributions/latest endpoint to docs
FabiKo117 Oct 5, 2020
2bae972
add examples (curl, python, R) for contributions and contributions/la…
Oct 8, 2020
89d0ea0
contributions endpoint corrections
Oct 8, 2020
5467cfc
swagger: make resource specs sortable
tyrasd Oct 19, 2020
7f1009f
Adding parameters descriptions, solves #58
Oct 27, 2020
b1e82dd
extending parameter descriptions in docs
FabiKo117 Oct 27, 2020
f50dce2
minor writing-style fixes in docs
FabiKo117 Nov 2, 2020
bb6be21
Revert "minor writing-style fixes in docs"
FabiKo117 Nov 2, 2020
d8f4c5f
minor writing-style fixes in docs
FabiKo117 Nov 2, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
587 changes: 587 additions & 0 deletions docs/endpoints.rst

Large diffs are not rendered by default.

25 changes: 25 additions & 0 deletions docs/http-response-status.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
HTTP Response Status
====================

List of HTTP status codes and respective descriptions.

2xx success
-----------

* ``200 OK`` - standard response for successful GET or POST requests

4xx client errors
-----------------

* ``400 Bad Request`` - the ohsome API cannot or will not process the request due to a client error (e.g. malformed request syntax)
* ``401 Unauthorized`` - the client does not have valid authentication credentials for the target resource
* ``404 Not Found``- the requested resource coud not be found
* ``405 Method not allowed`` - the requested method is not supported (e.g. a POST request for resources which accept only GET requests)
* ``413 Payload Too Large`` - the request is larger than the ohsome API is willing or able to process

5xx server errors
-----------------

* ``500 Internal Server Error``- generic error message, given when an unexpected condition in the ohsome API was encountered
* ``501 Not Implemented`` - the ohsome API either does not recognize the request method, or it lacks the ability to fulfil the request
* ``503 Service Unavailable`` - the ohsome API cannot handle the request (e.g. because it is overloaded or down for maintenance); temporary state
9 changes: 6 additions & 3 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,17 @@ Welcome to the documentation of the ohsome API!
.. autodocs

.. toctree::
:maxdepth: 2
:maxdepth: 1
:caption: Additional Information:
:hidden:


boundaries
group-by
time
filter
response-parameters
http-response-status

.. autodocs

Additional Links:
-----------------
Expand Down
36 changes: 36 additions & 0 deletions docs/response-parameters.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
Response Parameters
===================

Descriptions of the custom response parameters that are marked with a leading ``@`` for the
data extraction and contributions endpoints.

General Parameters
------------------

* ``@osmId`` - id of the OSM feature with its type (node/way/relation)
* ``@version`` - version of the feature
* ``@changesetId``- id assigned to a changeset
* ``@osmType``- type (node/way/relation) of OSM feature


Extraction Parameters
---------------------

**specific for /elements**

* ``@snapshotTimestamp`` - describes for which timestamp a snapshot of this feature was requested
* ``@lastEdit`` - describes the timestamp at which this feature was edited the last time

**specific for /elementsFullHistory**

* ``@validFrom`` - indicates when a creation or change of this feature with the provided attributes and geometry was made; has the same value as the fromTimestamp if the creation/change happened before the requested time interval
* ``@validTo`` - indicates until when this feature with the provided attributes and geometry stayed unchanged or undeleted; has the same value as the toTimestamp if the change/deletion happened after the requested time interval

Contribution Parameters
-----------------------

* ``@timestamp`` - indicates when this contribution occurred
* ``@creation`` - contribution type; indicates if this feature is newly created (true); cannot occur in combination with other contribution types
* ``@geometryChange`` - contribution type; indicates if the geometry of this feature has changed (true); can occur in combination with @tagChange
* ``@tagChange``- contribution type; indicates if the tag of this feature has changed (true); can occur in combination with @geometryChange
* ``@deletion`` - contribution type; indicates if the feature is deleted (true); cannot occur in combination with other contribution types
4 changes: 2 additions & 2 deletions docs/time.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ ISO-8601 conform timestring(s).

.. note:: The ohsome API only supports the UTC time zone (Z).

Supported time formats:
-----------------------
Supported time formats
----------------------

* timestamp: ``2014-01-01``
* list of timestamps: ``2014-01-01,2015-07-01,2018-10-10``
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ public class LoggingConfig implements WebMvcConfigurer {
@Override
public void addInterceptors(InterceptorRegistry registry) {
registry.addInterceptor(loggingRequestInterceptor).addPathPatterns("/elements/**/",
"/elementsFullHistory/**/", "/metadata/**/", "/users/**/");
"/elementsFullHistory/**/", "/metadata/**/", "/users/**/", "/contributions/**/");
}

}
104 changes: 84 additions & 20 deletions src/main/java/org/heigit/ohsome/ohsomeapi/config/SwaggerConfig.java
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,18 @@

import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import org.heigit.ohsome.ohsomeapi.Application;
import org.heigit.ohsome.ohsomeapi.controller.DefaultSwaggerParameters;
import org.heigit.ohsome.ohsomeapi.controller.ParameterDescriptions;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.context.annotation.PropertySource;
import org.springframework.web.bind.annotation.RequestMethod;
import springfox.documentation.builders.ParameterBuilder;
Expand All @@ -22,23 +28,62 @@
import springfox.documentation.service.Tag;
import springfox.documentation.spi.DocumentationType;
import springfox.documentation.spring.web.plugins.Docket;
import springfox.documentation.swagger.web.InMemorySwaggerResourcesProvider;
import springfox.documentation.swagger.web.SwaggerResource;
import springfox.documentation.swagger.web.SwaggerResourcesProvider;
import springfox.documentation.swagger2.annotations.EnableSwagger2;

/** Swagger configuration class. */
@Configuration
@EnableSwagger2
@PropertySource("classpath:application.properties")
public class SwaggerConfig {
@Primary
public class SwaggerConfig implements SwaggerResourcesProvider {
private enum OhsomeApiResourceSpec {
DATA_AGGREGATION("Data Aggregation", 1),
DATA_EXTRACTION("Data Extraction", 2),
CONTRIBUTIONS("Contributions", 3),
METADATA("Metadata", 9);

private final String name;
private final int sorting;
OhsomeApiResourceSpec(String name, int sorting) {
this.name = name;
this.sorting = sorting;
}
}

private final InMemorySwaggerResourcesProvider resourcesProvider;
private final Map<String, Integer> resourcesSorting = new HashMap<>();

/**
* Creates swagger configuration object, initializes internal specs sorting table.
*/
@Autowired
public SwaggerConfig(InMemorySwaggerResourcesProvider resourcesProvider) {
this.resourcesProvider = resourcesProvider;
for (OhsomeApiResourceSpec spec : OhsomeApiResourceSpec.values()) {
resourcesSorting.put(spec.name, spec.sorting);
}
}

@Override
public List<SwaggerResource> get() {
return resourcesProvider.get().stream()
.sorted(Comparator.comparing(r -> resourcesSorting.getOrDefault(r.getName(), 99)))
.collect(Collectors.toList());
}

/** Creates the Swagger2 documentation for the dataAggregation resources. */
@Bean
public Docket dataAggregationDocket() {
ArrayList<ResponseMessage> responseMessages = defineResponseMessages();
return new Docket(DocumentationType.SWAGGER_2).groupName("Data Aggregation").select()
return new Docket(DocumentationType.SWAGGER_2)
.groupName(OhsomeApiResourceSpec.DATA_AGGREGATION.name).select()
.apis(RequestHandlerSelectors
.basePackage("org.heigit.ohsome.ohsomeapi.controller.dataaggregation"))
.paths(PathSelectors.any()).build().apiInfo(apiInfo()).useDefaultResponseMessages(false)
.globalOperationParameters(defineGlobalOperationParams(false))
.globalOperationParameters(defineGlobalOperationParams(false, false))
.tags(new Tag("Users", "Compute data aggregation functions on users"),
new Tag("Area", "Compute the area of polygonal OSM elements"),
new Tag("Length", "Compute the length of linear OSM elements"),
Expand All @@ -52,7 +97,8 @@ public Docket dataAggregationDocket() {
@Bean
public Docket metadataDocket() {
ArrayList<ResponseMessage> responseMessages = defineResponseMessages();
return new Docket(DocumentationType.SWAGGER_2).groupName("Metadata").select()
return new Docket(DocumentationType.SWAGGER_2)
.groupName(OhsomeApiResourceSpec.METADATA.name).select()
.apis(
RequestHandlerSelectors.basePackage("org.heigit.ohsome.ohsomeapi.controller.metadata"))
.paths(PathSelectors.any()).build().apiInfo(apiInfo()).useDefaultResponseMessages(false)
Expand All @@ -64,16 +110,32 @@ public Docket metadataDocket() {
@Bean
public Docket rawDataDocket() {
ArrayList<ResponseMessage> responseMessages = defineResponseMessages();
return new Docket(DocumentationType.SWAGGER_2).groupName("Data Extraction").select()
return new Docket(DocumentationType.SWAGGER_2)
.groupName(OhsomeApiResourceSpec.DATA_EXTRACTION.name).select()
.apis(RequestHandlerSelectors.basePackage("org.heigit.ohsome.ohsomeapi.controller.rawdata"))
.paths(PathSelectors.any()).build().apiInfo(apiInfo()).useDefaultResponseMessages(false)
.globalOperationParameters(defineGlobalOperationParams(true))
.globalOperationParameters(defineGlobalOperationParams(true, false))
.tags(new Tag("Data Extraction", "Direct access to the OSM data"),
new Tag("Full-History Data Extraction",
"Direct access to the full-history of the OSM data"))
.forCodeGeneration(true).globalResponseMessage(RequestMethod.GET, responseMessages);
}

/** Creates the Swagger2 documentation for the contributions resources. */
tyrasd marked this conversation as resolved.
Show resolved Hide resolved
@Bean
public Docket contributionsDocket() {
ArrayList<ResponseMessage> responseMessages = defineResponseMessages();
return new Docket(DocumentationType.SWAGGER_2)
.groupName(OhsomeApiResourceSpec.CONTRIBUTIONS.name).select()
.apis(RequestHandlerSelectors
.basePackage("org.heigit.ohsome.ohsomeapi.controller.contributions"))
.paths(PathSelectors.any()).build().apiInfo(apiInfo()).useDefaultResponseMessages(false)
.globalOperationParameters(defineGlobalOperationParams(true, true))
.tags(
new Tag("Contributions", "Direct access to all contributions provided to the OSM data"))
.forCodeGeneration(true).globalResponseMessage(RequestMethod.GET, responseMessages);
}

/** Defines custom response messages for the used response codes. */
private ArrayList<ResponseMessage> defineResponseMessages() {
ArrayList<ResponseMessage> responseMessages = new ArrayList<>();
Expand Down Expand Up @@ -111,7 +173,8 @@ private ApiInfo apiInfo() {
* Defines the description of each parameter, which are used in all resources for the Swagger2
* documentation.
*/
private List<Parameter> defineGlobalOperationParams(boolean isDataExtraction) {
private List<Parameter> defineGlobalOperationParams(boolean isDataExtraction,
boolean isContributions) {
final String string = "string";
final String query = "query";
List<Parameter> globalOperationParams = new ArrayList<>();
Expand All @@ -124,15 +187,17 @@ private List<Parameter> defineGlobalOperationParams(boolean isDataExtraction) {
globalOperationParams.add(new ParameterBuilder().name("bpolys")
.description(ParameterDescriptions.BPOLYS).modelRef(new ModelRef(string))
.parameterType(query).defaultValue("").required(false).build());
globalOperationParams.add(new ParameterBuilder().name("types")
.description(ParameterDescriptions.DEPRECATED_USE_FILTER).modelRef(new ModelRef(string))
.allowMultiple(true).parameterType(query).defaultValue("").required(false).build());
globalOperationParams.add(new ParameterBuilder().name("keys")
.description(ParameterDescriptions.DEPRECATED_USE_FILTER).modelRef(new ModelRef(string))
.parameterType(query).defaultValue("").required(false).build());
globalOperationParams.add(new ParameterBuilder().name("values")
.description(ParameterDescriptions.DEPRECATED_USE_FILTER).modelRef(new ModelRef(string))
.parameterType(query).defaultValue("").required(false).build());
if (!isContributions) {
globalOperationParams.add(new ParameterBuilder().name("types")
.description(ParameterDescriptions.DEPRECATED_USE_FILTER).modelRef(new ModelRef(string))
.allowMultiple(true).parameterType(query).defaultValue("").required(false).build());
globalOperationParams.add(new ParameterBuilder().name("keys")
.description(ParameterDescriptions.DEPRECATED_USE_FILTER).modelRef(new ModelRef(string))
.parameterType(query).defaultValue("").required(false).build());
globalOperationParams.add(new ParameterBuilder().name("values")
.description(ParameterDescriptions.DEPRECATED_USE_FILTER).modelRef(new ModelRef(string))
.parameterType(query).defaultValue("").required(false).build());
}
globalOperationParams
.add(new ParameterBuilder().name("filter").description(ParameterDescriptions.FILTER)
.modelRef(new ModelRef(string)).parameterType(query)
Expand All @@ -159,10 +224,9 @@ private List<Parameter> defineGlobalOperationParams(boolean isDataExtraction) {
.description(ParameterDescriptions.CLIP_GEOMETRY).modelRef(new ModelRef(string))
.parameterType(query).defaultValue("true").required(false).build());
}
globalOperationParams.add(
new ParameterBuilder().name("showMetadata").description(ParameterDescriptions.SHOW_METADATA)
.modelRef(new ModelRef(string)).parameterType(query)
.defaultValue("").required(false).build());
globalOperationParams.add(new ParameterBuilder().name("showMetadata")
.description(ParameterDescriptions.SHOW_METADATA).modelRef(new ModelRef(string))
.parameterType(query).defaultValue("").required(false).build());
return globalOperationParams;
}
}
Loading