Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add task-based history export tracking #14839

Merged
merged 55 commits into from
Dec 12, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
71ebc4a
Add `StoreExportAssociation` table
davelopez Oct 20, 2022
4341f16
Add abstract AsyncTasksManager
davelopez Oct 20, 2022
c9494e5
Allow registering concrete classes for interfaces in DI
davelopez Oct 20, 2022
87da0cd
Use AsyncTasksManager in Tasks API
davelopez Oct 20, 2022
6a72eb8
Add StoreExportTracker
davelopez Oct 21, 2022
d3e311d
Remove unused dependency in HistoryController
davelopez Oct 27, 2022
452d99d
Use StoreExportTracker with histories
davelopez Oct 27, 2022
cc81892
Use custom accept content type in exports API
davelopez Oct 27, 2022
baa3b64
Move ExportObjectType enum to schema
davelopez Oct 27, 2022
b28b95a
Update history exports API docs
davelopez Oct 27, 2022
00f2efe
Order exports by date
davelopez Oct 28, 2022
1ae7737
Refactor some schema model names
davelopez Oct 28, 2022
faa44cf
Add FileSources composable
davelopez Oct 31, 2022
e6bfa3f
Add pagination support to exports endpoint
davelopez Nov 2, 2022
c4b031e
Deprecate jeha-based export API endpoints
davelopez Nov 2, 2022
6dc4cb9
Allow configuring polling time in useTaskMonitor
davelopez Nov 3, 2022
254c927
Add basic task based history export UI
davelopez Nov 4, 2022
6a212a0
Add ShortTermStorage composable
davelopez Nov 4, 2022
f9a7e62
Add basic history direct download using STS
davelopez Nov 4, 2022
ae34f84
Refactor schemas for short term export models
davelopez Nov 4, 2022
b83b509
Refactor rename variables for consistency
davelopez Nov 4, 2022
4894631
Track direct downloads to reuse the STS export
davelopez Nov 14, 2022
3ab7b67
Use correct model_store_format when importing from uri
davelopez Nov 15, 2022
6d0548b
Implement history reimport from file source
davelopez Nov 16, 2022
4738f1f
Fix imports order
davelopez Nov 17, 2022
7e704b3
Add re-import confirmation
davelopez Nov 17, 2022
c59fced
Add STS duration to ShortTermStorageTarget
davelopez Nov 18, 2022
1f0d63c
Add expiration date to short term download links
davelopez Nov 18, 2022
e01e96d
Refactor export record details date management
davelopez Nov 23, 2022
d328a48
Add basic table with previous export records
davelopez Nov 23, 2022
67b8ba4
Disable celery in existing history export selenium test
davelopez Nov 23, 2022
5ee4952
Refactor loading of records to make single request
davelopez Nov 24, 2022
b4715a4
Reuse up to date download record if available
davelopez Nov 24, 2022
6ff62a8
Add more information and refactor messages
davelopez Nov 24, 2022
eb1fbe9
Optionally clear ExportForm input after export
davelopez Nov 25, 2022
2c0fa4a
Add more test coverage to ExportForm
davelopez Nov 25, 2022
9c42083
Add ExportOptions component
davelopez Nov 25, 2022
e338bad
Display export format in records
davelopez Nov 25, 2022
d6947c6
Simplify export result handling
davelopez Nov 25, 2022
67f3ed8
Various fixes around export parameters
davelopez Nov 28, 2022
b32a7f5
Add some jest tests
davelopez Nov 28, 2022
32a08e0
Add API test for history export tracking
davelopez Nov 28, 2022
c65a870
Fix mypy ignore type after version update
davelopez Nov 29, 2022
3a1ae06
Increase `test_export_tracking` coverage
davelopez Nov 29, 2022
b816fab
Add integration selenium test for export tracking
davelopez Nov 29, 2022
6432a29
Fix export record `ready` state + refactor test
davelopez Nov 30, 2022
4f02382
Convert service functions to TS
davelopez Dec 5, 2022
6122d1c
Display detailed error message if the export fails
davelopez Dec 5, 2022
175c640
Rename path parameter id to history_id
davelopez Dec 9, 2022
86761ab
Migrate ExportRecordModel to typescript
davelopez Dec 9, 2022
e2c6ca5
Migrate history export services to typescript
davelopez Dec 9, 2022
19a1759
Cast the response type in getExportRecords
davelopez Dec 9, 2022
fbd845f
Migrate jest tests to typescript
davelopez Dec 9, 2022
9c3809e
Add back ExportParamsModel class
davelopez Dec 10, 2022
e857985
Move test_history_export selenium test
davelopez Dec 10, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions client/src/components/Common/ExportForm.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,35 @@ describe("ExportForm.vue", () => {
it("should localize button text", async () => {
expect(wrapper.find(".export-button").text()).toBeLocalizationOf("Export");
});

it("should emit 'export' event with correct inputs on export button click", async () => {
await wrapper.setData({
name: "export.tar.gz",
directory: "gxfiles://",
});
expect(wrapper.emitted()).not.toHaveProperty("export");

await wrapper.find(".export-button").trigger("click");

expect(wrapper.emitted()).toHaveProperty("export");
expect(wrapper.emitted()["export"][0][0]).toBe("gxfiles://");
expect(wrapper.emitted()["export"][0][1]).toBe("export.tar.gz");
});

it("should clear the inputs after export when clearInputAfterExport is enabled", async () => {
await wrapper.setProps({
clearInputAfterExport: true,
});
await wrapper.setData({
name: "export.tar.gz",
directory: "gxfiles://",
});
expect(wrapper.vm.directory).toEqual("gxfiles://");
expect(wrapper.vm.name).toEqual("export.tar.gz");

await wrapper.find(".export-button").trigger("click");

expect(wrapper.vm.directory).toBe(null);
expect(wrapper.vm.name).toBe(null);
});
});
10 changes: 9 additions & 1 deletion client/src/components/Common/ExportForm.vue
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<template>
<div>
<div class="export-to-remote-file">
<b-form-group
id="fieldset-directory"
label-for="directory"
Expand Down Expand Up @@ -32,6 +32,10 @@ export default {
type: String,
default: "archive",
},
clearInputAfterExport: {
type: Boolean,
default: false,
},
},
data() {
return {
Expand Down Expand Up @@ -59,6 +63,10 @@ export default {
methods: {
doExport() {
this.$emit("export", this.directory, this.name);
if (this.clearInputAfterExport) {
this.directory = null;
this.name = null;
}
},
},
};
Expand Down
127 changes: 127 additions & 0 deletions client/src/components/Common/ExportRecordDetails.vue
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
<script setup>
import { computed } from "vue";
import { BAlert, BCard, BCardTitle } from "bootstrap-vue";
import LoadingSpan from "components/LoadingSpan";
import { FontAwesomeIcon } from "@fortawesome/vue-fontawesome";
import { library } from "@fortawesome/fontawesome-svg-core";
import { faExclamationCircle, faExclamationTriangle, faCheckCircle, faClock } from "@fortawesome/free-solid-svg-icons";
import { ExportRecordModel } from "./models/exportRecordModel";

library.add(faExclamationCircle, faExclamationTriangle, faCheckCircle, faClock);

const props = defineProps({
record: {
type: ExportRecordModel,
required: true,
},
objectType: {
type: String,
required: true,
},
actionMessage: {
type: String,
default: null,
},
actionMessageVariant: {
type: String,
default: "info",
},
});

const emit = defineEmits(["onReimport", "onDownload", "onActionMessageDismissed"]);

const title = computed(() => (props.record.isReady ? `Exported` : `Export started`));
const preparingMessage = computed(
() => `Preparing export. This may take some time depending on the size of your ${props.objectType}`
);

async function reimportObject() {
emit("onReimport", props.record);
}

function downloadObject() {
emit("onDownload", props.record);
}

function onMessageDismissed() {
emit("onActionMessageDismissed");
}
</script>

<template>
<b-card class="export-record-details">
<b-card-title>
<b>{{ title }}</b> {{ props.record.elapsedTime }}
</b-card-title>
<p v-if="!props.record.isPreparing">
Format: <b class="record-archive-format">{{ props.record.modelStoreFormat }}</b>
</p>
<span v-if="props.record.isPreparing">
<loading-span :message="preparingMessage" />
</span>
<div v-else>
<div v-if="props.record.hasFailed">
<font-awesome-icon
icon="exclamation-circle"
class="text-danger record-failed-icon"
title="Export failed" />
<span>
Something failed during this export. Please try again and if the problem persist contact your
administrator.
</span>
<b-alert show variant="danger">{{ props.record.errorMessage }}</b-alert>
</div>
<div v-else-if="props.record.isUpToDate" title="Up to date">
<font-awesome-icon icon="check-circle" class="text-success record-up-to-date-icon" />
<span> This export record contains the latest changes of the {{ props.objectType }}. </span>
</div>
<div v-else>
<font-awesome-icon icon="exclamation-triangle" class="text-warning record-outdated-icon" />
<span>
This export is outdated and contains the changes of this {{ props.objectType }} from
{{ props.record.elapsedTime }}.
</span>
</div>

<p v-if="props.record.canExpire" class="mt-3">
<span v-if="props.record.hasExpired">
<font-awesome-icon icon="clock" class="text-danger record-expired-icon" /> This download link has
expired.
</span>
<span v-else>
<font-awesome-icon icon="clock" class="text-warning record-expiration-warning-icon" /> This download
link expires {{ props.record.expirationElapsedTime }}.
</span>
</p>

<div v-if="props.record.isReady">
<p class="mt-3">You can do the following actions with this {{ props.objectType }} export:</p>
<b-alert
v-if="props.actionMessage !== null"
:variant="props.actionMessageVariant"
show
fade
dismissible
@dismissed="onMessageDismissed">
{{ props.actionMessage }}
</b-alert>
<div v-else class="actions">
<b-button
v-if="props.record.canDownload"
class="record-download-btn"
variant="primary"
@click="downloadObject">
Download
</b-button>
<b-button
v-if="props.record.canReimport"
class="record-reimport-btn"
variant="primary"
@click="reimportObject">
Reimport
</b-button>
</div>
</div>
</div>
</b-card>
</template>
130 changes: 130 additions & 0 deletions client/src/components/Common/ExportRecordTable.vue
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
<script setup>
import { computed, ref } from "vue";
import { BCard, BButton, BButtonGroup, BButtonToolbar, BCollapse, BTable, BLink } from "bootstrap-vue";
import { FontAwesomeIcon } from "@fortawesome/vue-fontawesome";
import { library } from "@fortawesome/fontawesome-svg-core";
import {
faExclamationCircle,
faCheckCircle,
faDownload,
faFileImport,
faSpinner,
} from "@fortawesome/free-solid-svg-icons";

library.add(faExclamationCircle, faCheckCircle, faDownload, faFileImport, faSpinner);

const props = defineProps({
records: {
type: Array,
required: true,
},
});

const emit = defineEmits(["onReimport", "onDownload"]);

const fields = [
{ key: "elapsedTime", label: "Exported" },
{ key: "format", label: "Format" },
{ key: "expires", label: "Expires" },
{ key: "isUpToDate", label: "Up to date", class: "text-center" },
{ key: "isReady", label: "Ready", class: "text-center" },
{ key: "actions", label: "Actions" },
];

const isExpanded = ref(false);
const title = computed(() => (isExpanded.value ? `Hide export records` : `Show export records`));

async function reimportObject(record) {
emit("onReimport", record);
}

function downloadObject(record) {
emit("onDownload", record);
}
</script>

<template>
<div>
<b-link
:class="isExpanded ? null : 'collapsed'"
:aria-expanded="isExpanded ? 'true' : 'false'"
aria-controls="collapse-previous"
@click="isExpanded = !isExpanded">
{{ title }}
</b-link>
<b-collapse id="collapse-previous" v-model="isExpanded">
<b-card>
<b-table :items="props.records" :fields="fields">
<template v-slot:cell(elapsedTime)="row">
<span :title="row.item.date">{{ row.value }}</span>
</template>
<template v-slot:cell(format)="row">
<span>{{ row.item.modelStoreFormat }}</span>
</template>
<template v-slot:cell(expires)="row">
<span v-if="row.item.hasExpired">Expired</span>
<span v-else-if="row.item.expirationDate" :title="row.item.expirationDate">{{
row.item.expirationElapsedTime
}}</span>
<span v-else>No</span>
</template>
<template v-slot:cell(isUpToDate)="row">
<font-awesome-icon
v-if="row.item.isUpToDate"
icon="check-circle"
class="text-success"
title="This export record contains the latest changes." />
<font-awesome-icon
v-else
icon="exclamation-circle"
class="text-danger"
title="This export record is outdated. Please consider generating a new export if you need the latest changes." />
</template>
<template v-slot:cell(isReady)="row">
<font-awesome-icon
v-if="row.item.isReady"
icon="check-circle"
class="text-success"
title="Ready to download or import." />
<font-awesome-icon
v-else-if="row.item.isPreparing"
icon="spinner"
spin
class="text-info"
title="Exporting in progress..." />
<font-awesome-icon
v-else-if="row.item.hasExpired"
icon="exclamation-circle"
class="text-danger"
title="The export has expired." />
<font-awesome-icon
v-else
icon="exclamation-circle"
class="text-danger"
title="The export failed." />
</template>
<template v-slot:cell(actions)="row">
<b-button-toolbar aria-label="Actions">
<b-button-group>
<b-button
v-b-tooltip.hover.bottom
:disabled="!row.item.canDownload"
title="Download"
@click="downloadObject(row.item)">
<font-awesome-icon icon="download" />
</b-button>
<b-button
v-b-tooltip.hover.bottom
:disabled="!row.item.canReimport"
title="Reimport"
@click="reimportObject(row.item)">
<font-awesome-icon icon="file-import" />
</b-button>
</b-button-group>
</b-button-toolbar>
</template>
</b-table>
</b-card>
</b-collapse>
</div>
</template>
61 changes: 61 additions & 0 deletions client/src/components/Common/models/exportRecordModel.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
import { ExportRecordModel } from "./exportRecordModel";
import {
EXPECTED_EXPIRATION_DATE,
EXPIRED_STS_DOWNLOAD_RESPONSE,
FAILED_DOWNLOAD_RESPONSE,
FILE_SOURCE_STORE_RESPONSE,
RECENT_STS_DOWNLOAD_RESPONSE,
} from "./testData/exportData";

describe("ExportRecordModel", () => {
describe("STS Download Record", () => {
const stsDownloadRecord = new ExportRecordModel(RECENT_STS_DOWNLOAD_RESPONSE);

it("should be considered temporal (STS) when it has a short term storage ID defined", () => {
expect(stsDownloadRecord.isStsDownload).toBe(true);
expect(stsDownloadRecord.stsDownloadId).toBeTruthy();
});

it("should allow download when ready and not yet expired", () => {
expect(stsDownloadRecord.isReady).toBe(true);
expect(stsDownloadRecord.hasExpired).toBe(false);
expect(stsDownloadRecord.canDownload).toBe(true);
});
});

describe("Expired STS Download Record", () => {
const expiredDownloadRecord = new ExportRecordModel(EXPIRED_STS_DOWNLOAD_RESPONSE);

it("should calculate the correct expiration date", () => {
expect(expiredDownloadRecord.canExpire).toBe(true);
expect(expiredDownloadRecord.expirationDate).toStrictEqual(EXPECTED_EXPIRATION_DATE());
});

it("should not allow download when expired", () => {
expect(expiredDownloadRecord.hasExpired).toBe(true);
expect(expiredDownloadRecord.canDownload).toBe(false);
expect(expiredDownloadRecord.isReady).toBe(false);
});
});

describe("Failed STS Download Record", () => {
const failedDownloadRecord = new ExportRecordModel(FAILED_DOWNLOAD_RESPONSE);

it("should not be downloadable", () => {
expect(failedDownloadRecord.isReady).toBe(false);
expect(failedDownloadRecord.isPreparing).toBe(false);
expect(failedDownloadRecord.hasExpired).toBe(false);
expect(failedDownloadRecord.canDownload).toBe(false);
});
});

describe("File Source Storage Record", () => {
const failedDownloadRecord = new ExportRecordModel(FILE_SOURCE_STORE_RESPONSE);

it("should be importable", () => {
expect(failedDownloadRecord.isReady).toBe(true);
expect(failedDownloadRecord.canReimport).toBe(true);
expect(failedDownloadRecord.importUri).toBeTruthy();
});
});
});
Loading