Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(impl):[TRI-1664] new batch api endpoints impl #561

Merged
merged 20 commits into from
Oct 9, 2023
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
34e2c2c
feat(impl):[TRI-1664] new batch api endpoints impl
ds-ext-kmassalski Oct 2, 2023
c55ea1b
feat(impl):[TRI-1664] new api endpoint
ds-ext-kmassalski Oct 3, 2023
233c856
feat(impl):[TRI-1664] update api impl
ds-ext-kmassalski Oct 3, 2023
026d296
feat(impl):[TRI-1664] change default bpn allowed api
ds-ext-kmassalski Oct 3, 2023
e65cc7a
Update irs-api/src/main/java/org/eclipse/tractusx/irs/controllers/Bat…
ds-ext-kmassalski Oct 4, 2023
0072e4e
Update irs-api/src/main/java/org/eclipse/tractusx/irs/controllers/Bat…
ds-ext-kmassalski Oct 4, 2023
595b792
Update irs-models/src/main/java/org/eclipse/tractusx/irs/component/Re…
ds-ext-kmassalski Oct 4, 2023
b84cf42
feat(impl):[TRI-1664] fix tests
ds-ext-kmassalski Oct 4, 2023
c204308
Merge branch 'main' into feature/TRI-1664-new-batch-api-endpoint
ds-ext-kmassalski Oct 4, 2023
d346572
feat(impl):[TRI-1664] merge main
ds-ext-kmassalski Oct 4, 2023
8b15cd9
feat(impl):[TRI-1664] cleanup - move code from IrsService responsibility
ds-ext-kmassalski Oct 4, 2023
a19c6fc
Merge branch 'feature/TRI-1678-fix-jsonschema-validation' into featur…
ds-ext-kmassalski Oct 5, 2023
7a5f7c3
feat(impl):[TRI-1664] fix code smells
ds-ext-kmassalski Oct 5, 2023
e597629
Merge branch 'main' into feature/TRI-1664-new-batch-api-endpoint
ds-ext-kmassalski Oct 5, 2023
39b4704
feat(impl):[TRI-1664] fix test
ds-ext-kmassalski Oct 5, 2023
4cdb21f
feat(impl):[TRI-1664] cover with more unit tests
ds-ext-kmassalski Oct 5, 2023
d6193cf
Merge branch 'main' into feature/TRI-1664-new-batch-api-endpoint
ds-ext-kmassalski Oct 5, 2023
150f628
Merge branch 'main' into feature/TRI-1664-new-batch-api-endpoint
ds-ext-kmassalski Oct 9, 2023
da4b773
feat(impl):[TRI-1664] update changelog, merge main
ds-ext-kmassalski Oct 9, 2023
c6e3bbe
feat(impl):[TRI-1664] fix test
ds-ext-kmassalski Oct 9, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions docs/src/api/irs-v1.0.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -158,9 +158,8 @@ paths:
- Environmental- and Social Standards
/irs/ess/orders:
post:
description: "Registers an order for an ess investigation with an array of\
\ {globalAssetIds}. Each globalAssetId will be processed in an separate job,\
\ grouped in batches."
description: "Registers an order for an ESS investigation with an array of
{globalAssetIds}. Each globalAssetId will be processed in an separate job, grouped in batches."
operationId: registerESSInvestigationOrder
requestBody:
content:
Expand Down Expand Up @@ -208,8 +207,8 @@ paths:
security:
- oAuth2:
- profile email
summary: "Registers an order for an ess investigation with an array of {globalAssetIds}.\
\ Each globalAssetId will be processed in an separate job, grouped in batches."
summary: "Registers an order for an ESS investigation with an array of {globalAssetIds}.
Each globalAssetId will be processed in an separate job, grouped in batches."
tags:
- Environmental- and Social Standards
/irs/jobs:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,10 @@ public JobParameter getJobParameter() {
return getJob().getParameter();
}

public boolean jobIsCompleted() {
return this.getJob().getState().equals(JobState.COMPLETED);
}

/**
* Builder for {@link MultiTransferJob}.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -123,10 +123,10 @@ public BatchOrderCreated registerBatchOrder(final @Valid @RequestBody RegisterBa
}

@Operation(operationId = "registerESSInvestigationOrder",
summary = "Registers an order for an ess investigation with an array of {globalAssetIds}. Each globalAssetId will be processed in an separate job, grouped in batches.",
summary = "Registers an order for an ESS investigation with an array of {globalAssetIds}. Each globalAssetId will be processed in an separate job, grouped in batches.",
security = @SecurityRequirement(name = "oAuth2", scopes = "profile email"),
tags = { "Environmental- and Social Standards" },
description = "Registers an order for an ess investigation with an array of {globalAssetIds}. Each globalAssetId will be processed in an separate job, grouped in batches.")
description = "Registers an order for an ESS investigation with an array of {globalAssetIds}. Each globalAssetId will be processed in an separate job, grouped in batches.")
@ApiResponses(value = { @ApiResponse(responseCode = "201", description = "Returns orderId of registered Batch order.",
content = { @Content(mediaType = APPLICATION_JSON_VALUE,
schema = @Schema(implementation = BatchOrderCreated.class),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -275,10 +275,8 @@ public PageResult getJobsByState(
explode = Explode.FALSE, array = @ArraySchema(schema = @Schema(implementation = JobState.class), maxItems = Integer.MAX_VALUE))
@RequestParam(value = "states", required = false, defaultValue = "") final List<JobState> states,
@Parameter(hidden = true)
@RequestParam(value = "jobStates", required = false, defaultValue = "") final List<JobState> jobStates,
@Parameter(hidden = true)
@ParameterObject final Pageable pageable) {
return itemJobService.getJobsByState(states, jobStates, pageable);
ds-ext-kmassalski marked this conversation as resolved.
Show resolved Hide resolved
return itemJobService.getJobsByState(states, pageable);
}

@Operation(operationId = "getAllAspectModels",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,6 @@ public interface IIrsItemGraphQueryService {
JobHandle registerItemJob(@NonNull RegisterJob request);

PageResult getJobsByState(@NonNull List<JobState> states, Pageable pageable);
PageResult getJobsByState(@NonNull List<JobState> states, @NonNull List<JobState> jobStates, Pageable pageable);

Job cancelJobById(@NonNull UUID jobId);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ public IrsItemGraphQueryService(final JobOrchestrator<ItemDataRequest, AASTransf
}

@Override
public PageResult getJobsByState(final @NonNull List<JobState> states, final Pageable pageable) {
public PageResult getJobsByState(@NonNull final List<JobState> states, final Pageable pageable) {
final List<MultiTransferJob> jobs = filterJobs(states);
final List<JobStatusResult> jobStatusResults = jobs.stream()
.map(job -> JobStatusResult.builder()
Expand All @@ -155,12 +155,6 @@ private List<MultiTransferJob> filterJobs(final @NotNull List<JobState> states)
}
}

@Override
public PageResult getJobsByState(@NonNull final List<JobState> states, @NonNull final List<JobState> jobStates,
final Pageable pageable) {
return getJobsByState(states.isEmpty() ? jobStates : states, pageable);
}

private PagedListHolder<JobStatusResult> paginateAndSortResults(final Pageable pageable,
final List<JobStatusResult> results) {
final PagedListHolder<JobStatusResult> pageListHolder = new PagedListHolder<>(new ArrayList<>(results));
Expand All @@ -184,7 +178,7 @@ public JobHandle registerItemJob(final @NonNull RegisterJob request) {
}

public JobHandle registerItemJob(final @NonNull RegisterJob request, final UUID batchId, final String owner) {
final var params = buildJobParameter(request);
final var params = JobParameter.create(request);
if (params.getDirection().equals(Direction.UPWARD) && !params.getBomLifecycle().equals(BomLifecycle.AS_BUILT)) {
// Currently not supported variant
throw new IllegalArgumentException("Upward direction is supported only for asBuilt bomLifecycle parameter!");
Expand All @@ -193,6 +187,7 @@ public JobHandle registerItemJob(final @NonNull RegisterJob request, final UUID
throw new ResponseStatusException(HttpStatus.INTERNAL_SERVER_ERROR,
"Can't start job with BPN lookup - configured bpdm endpoint is empty!");
}
validateAspectTypeValues(params.getAspects());

final JobInitiateResponse jobInitiateResponse = orchestrator.startJob(request.getKey().getGlobalAssetId(),
params, batchId, owner);
Expand All @@ -206,27 +201,6 @@ public JobHandle registerItemJob(final @NonNull RegisterJob request, final UUID
}
}

private JobParameter buildJobParameter(final @NonNull RegisterJob request) {
final BomLifecycle bomLifecycle = Optional.ofNullable(request.getBomLifecycle()).orElse(BomLifecycle.AS_BUILT);
final List<String> aspectTypeValues = Optional.ofNullable(request.getAspects())
.orElse(List.of(bomLifecycle.getDefaultAspect()));
validateAspectTypeValues(aspectTypeValues);
final Direction direction = Optional.ofNullable(request.getDirection()).orElse(Direction.DOWNWARD);

return JobParameter.builder()
.depth(request.getDepth())
.bomLifecycle(bomLifecycle)
.bpn(request.getKey().getBpn())
.direction(direction)
.aspects(aspectTypeValues.isEmpty()
? List.of(bomLifecycle.getDefaultAspect())
: aspectTypeValues)
.collectAspects(request.isCollectAspects())
.lookupBPNs(request.isLookupBPNs())
.callbackUrl(request.getCallbackUrl())
.build();
}

private void validateAspectTypeValues(final List<String> aspectTypeValues) {
try {
final HashSet<AspectModel> availableModels = new HashSet<>(
Expand Down Expand Up @@ -294,7 +268,7 @@ public Jobs getJobForJobId(final MultiTransferJob multiJob, final boolean includ
final var submodels = new ArrayList<Submodel>();
final var bpns = new ArrayList<Bpn>();

if (jobIsCompleted(multiJob)) {
if (multiJob.jobIsCompleted()) {
final var container = retrieveJobResultRelationships(multiJob.getJob().getId());
relationships.addAll(container.getRelationships());
tombstones.addAll(container.getTombstones());
Expand Down Expand Up @@ -428,8 +402,4 @@ private ItemContainer retrieveJobResultRelationships(final UUID jobId) {
}
}

private boolean jobIsCompleted(final MultiTransferJob multiJob) {
return multiJob.getJob().getState().equals(JobState.COMPLETED);
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ private void thereIsJwtAuthentication() {

Jwt jwt() {
return new Jwt("token", Instant.now(), Instant.now().plusSeconds(30), Map.of("alg", "none"),
Map.of(SUB, "sub", "clientId", "clientId", "bpn", "BPNL00000003CRHK"));
Map.of(SUB, "sub", "clientId", "clientId", "bpn", "BPNL00000001CRHK"));
}

@NotNull
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ void getJobsByState() throws Exception {
final String returnJobAsString = objectMapper.writeValueAsString(returnedJob);

when(authorizationService.verifyBpn()).thenReturn(Boolean.TRUE);
when(service.getJobsByState(any(), any(), any())).thenReturn(
when(service.getJobsByState(any(), any())).thenReturn(
new PageResult(new PagedListHolder<>(List.of(returnedJob))));

this.mockMvc.perform(get("/irs/jobs"))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,9 +95,8 @@ void shouldSuccessfullyStartJobAndReturnWithExtendedSubmodelList() {
.build();

when(irsItemGraphQueryService.registerItemJob(any(RegisterJob.class), any(), any())).thenReturn(JobHandle.builder().id(createdJobId).build());
when(jobStore.find(eq(createdJobId.toString()))).thenReturn(Optional.of(MultiTransferJob.builder().job(expectedResponse.getJob()).build()));
when(irsItemGraphQueryService.getJobForJobId(any(MultiTransferJob.class), eq(true))).thenReturn(expectedResponse);
when(securityHelperService.isAdmin()).thenReturn(true);
when(jobStore.find(createdJobId.toString())).thenReturn(Optional.of(MultiTransferJob.builder().job(expectedResponse.getJob()).build()));
when(irsItemGraphQueryService.getJobForJobId(any(MultiTransferJob.class), eq(true))).thenReturn(expectedResponse); when(securityHelperService.isAdmin()).thenReturn(true);
ds-ext-kmassalski marked this conversation as resolved.
Show resolved Hide resolved

final JobHandle jobHandle = essService.startIrsJob(request);
final Jobs jobs = essService.getIrsJob(jobHandle.getId().toString());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -352,7 +352,7 @@ private void createMockForJobIdAndShell(final UUID mockedJobId, final String moc
final BpnInvestigationJob bpnInvestigationJob = BpnInvestigationJob.create(jobs, incindentBPNSs);

when(bpnInvestigationJobCache.findByJobId(mockedJobId)).thenReturn(Optional.of(bpnInvestigationJob));
when(jobStore.find(eq(mockedJobId.toString()))).thenReturn(Optional.of(MultiTransferJob.builder().job(jobs.getJob()).build()));
when(jobStore.find(mockedJobId.toString())).thenReturn(Optional.of(MultiTransferJob.builder().job(jobs.getJob()).build()));
when(irsItemGraphQueryService.getJobForJobId(any(MultiTransferJob.class), eq(false))).thenReturn(jobs);
}

Expand All @@ -373,7 +373,7 @@ private void createMockForJobIdAndShells(final UUID mockedJobId, final List<Stri
List.of("BPNS000000000DDD"));

when(bpnInvestigationJobCache.findByJobId(mockedJobId)).thenReturn(Optional.of(bpnInvestigationJob));
when(jobStore.find(eq(mockedJobId.toString()))).thenReturn(Optional.of(MultiTransferJob.builder().job(jobs.getJob()).build()));
when(jobStore.find(mockedJobId.toString())).thenReturn(Optional.of(MultiTransferJob.builder().job(jobs.getJob()).build()));
when(irsItemGraphQueryService.getJobForJobId(any(MultiTransferJob.class), eq(false))).thenReturn(jobs);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ void shouldStartNextBatchWhenPreviousFinished() {
.jobTimeout(timeout)
.lookupBPNs(Boolean.TRUE)
.owner(owner)
.jobType(BatchOrder.JobType.REGULAR)
.jobType(BatchOrder.JobType.ESS)
.build();
final Batch firstBatch = Batch.builder()
.batchId(FIRST_BATCH_ID)
Expand All @@ -156,7 +156,7 @@ void shouldStartNextBatchWhenPreviousFinished() {
.owner(owner)
.build();

given(irsItemGraphQueryService.registerItemJob(any(), any(), eq(owner))).willReturn(
given(essService.startIrsJob(any(), any(), eq(owner))).willReturn(
JobHandle.builder().id(UUID.randomUUID()).build());

batchOrderStore.save(BATCH_ORDER_ID, batchOrder);
Expand All @@ -165,7 +165,7 @@ void shouldStartNextBatchWhenPreviousFinished() {
// when
eventListener.handleBatchProcessingFinishedEvent(new BatchProcessingFinishedEvent(BATCH_ORDER_ID, FIRST_BATCH_ID, ProcessingState.PARTIAL, ProcessingState.COMPLETED, 1, ""));
// then
verify(irsItemGraphQueryService, times(numberOfJobs)).registerItemJob(any(), eq(SECOND_BATCH_ID), eq(owner));
verify(essService, times(numberOfJobs)).startIrsJob(any(), eq(SECOND_BATCH_ID), eq(owner));
verify(timeoutScheduler, times(1)).registerBatchTimeout(SECOND_BATCH_ID, timeout);
verify(timeoutScheduler, times(1)).registerJobsTimeout(anyList(), eq(timeout));
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
import org.eclipse.tractusx.irs.common.auth.SecurityHelperService;
import org.eclipse.tractusx.irs.component.PartChainIdentificationKey;
import org.eclipse.tractusx.irs.component.RegisterBatchOrder;
import org.eclipse.tractusx.irs.component.RegisterBpnInvestigationBatchOrder;
import org.eclipse.tractusx.irs.component.enums.BatchStrategy;
import org.eclipse.tractusx.irs.component.enums.BomLifecycle;
import org.eclipse.tractusx.irs.component.enums.Direction;
Expand Down Expand Up @@ -75,7 +76,7 @@ void beforeEach() {
}

@Test
void shouldStoreBatchOrder() throws MalformedURLException {
void shouldStoreRegularBatchOrder() throws MalformedURLException {
// given
final RegisterBatchOrder registerBatchOrder = exampleBatchRequest();
given(irsConfiguration.getApiUrl()).willReturn(new URL(EXAMPLE_URL));
Expand All @@ -94,6 +95,26 @@ void shouldStoreBatchOrder() throws MalformedURLException {
Collectors.toList())).containsOnly(FIRST_GLOBAL_ASSET_ID, SECOND_GLOBAL_ASSET_ID);
}

@Test
void shouldStoreESSBatchOrder() throws MalformedURLException {
// given
final RegisterBpnInvestigationBatchOrder registerBatchOrder = exampleESSBatchRequest();
given(irsConfiguration.getApiUrl()).willReturn(new URL(EXAMPLE_URL));

// when
final UUID batchOrderId = service.create(registerBatchOrder);

// then
assertThat(batchOrderId).isNotNull();
assertThat(batchOrderStore.findAll()).hasSize(1);
assertThat(batchStore.findAll()).hasSize(1);

Batch actual = batchStore.findAll().stream().findFirst().orElseThrow();
assertThat(actual.getJobProgressList().stream().map(JobProgress::getIdentificationKey).map(
PartChainIdentificationKey::getGlobalAssetId).collect(
Collectors.toList())).containsOnly(FIRST_GLOBAL_ASSET_ID, SECOND_GLOBAL_ASSET_ID);
}

@Test
void shouldSplitIdentificationKeysIdIntoBatches() throws MalformedURLException {
// given
Expand Down Expand Up @@ -138,4 +159,17 @@ private static RegisterBatchOrder exampleBatchRequest() {
.build();
}

private static RegisterBpnInvestigationBatchOrder exampleESSBatchRequest() {
return RegisterBpnInvestigationBatchOrder.builder()
.keys(Set.of(PartChainIdentificationKey.builder().globalAssetId(FIRST_GLOBAL_ASSET_ID).build(),
PartChainIdentificationKey.builder().globalAssetId(SECOND_GLOBAL_ASSET_ID).build()))
.bomLifecycle(BomLifecycle.AS_PLANNED)
.timeout(1000)
.jobTimeout(500)
.batchStrategy(BatchStrategy.PRESERVE_JOB_ORDER)
.callbackUrl(EXAMPLE_URL)
.batchSize(10)
.build();
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -191,46 +191,16 @@ void shouldReturnFoundJobs() {
assertThat(jobs.totalElements()).isEqualTo(1);
}

@Test
void shouldTakeStatesInsteadOfDeprecatedParameter() {
final List<JobState> states = List.of(JobState.COMPLETED);
final List<JobState> jobStates = List.of(JobState.RUNNING);

testee.getJobsByState(states, jobStates, Pageable.ofSize(10));

verify(jobStore).findByStates(states);
}

@Test
void shouldTakeDeprecatedParameterWhenCorrectOneIsEmpty() {
final List<JobState> states = List.of();
final List<JobState> jobStates = List.of(JobState.RUNNING);

testee.getJobsByState(states, jobStates, Pageable.ofSize(10));

verify(jobStore).findByStates(jobStates);
}

@Test
void shouldTakeAllJobsWhenBothListEmpty() {
final List<JobState> states = List.of();
final List<JobState> jobStates = List.of();

testee.getJobsByState(states, jobStates, Pageable.ofSize(10));
testee.getJobsByState(states, Pageable.ofSize(10));

verify(jobStore).findAll();
}

@Test
void shouldTakeStatesInsteadOfEmptyDeprecatedParameter() {
final List<JobState> states = List.of(JobState.COMPLETED);
final List<JobState> jobStates = List.of();

testee.getJobsByState(states, jobStates, Pageable.ofSize(10));

verify(jobStore).findByStates(states);
}

@Test
void shouldThrowExceptionForInvalidAspectTypes() throws SchemaNotFoundException {
when(semanticsHubFacade.getAllAspectModels()).thenReturn(AspectModels.builder().models(List.of()).build());
Expand Down Expand Up @@ -258,8 +228,6 @@ void shouldThrowExceptionForUnknownAspectTypes() throws SchemaNotFoundException

@Test
void shouldThrowExceptionWhenLookupBpnAndBpdmEndpointUrlIsMissing() throws SchemaNotFoundException {
when(semanticsHubFacade.getAllAspectModels()).thenReturn(AspectModels.builder().models(List.of(AspectModel.builder().name("SingleLevelBomAsBuilt").build())).build());

final Executable executable = () -> testee.registerItemJob(registerJobWithLookupBPNs());

assertThrows(ResponseStatusException.class, executable);
Expand Down
Loading