Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

webui: Add table to display the status of recent ingestion jobs. #350

Merged
merged 29 commits into from
Apr 22, 2024

Conversation

junhaoliao
Copy link
Member

References

Compression job statuses are currently only displayed to users by the compress.sh script.

Description

This PR adds a Compression Job Table in Ingestion View and the corresponding backend handling. The changes should also improve User Experience for tracking S3 job submissions, which is not yet ported to this repository.

Validation performed

  1. Built and started clp-package.
  2. Visited WebUI address in a browser. Observed the page to load without errors, and "Space Savings" showed 0.00% because no file has been compressed.
    image
  3. Kept the browser opened. In the console, ran command ./compress.sh ~/samples/hive-24hr/ to compress sample logs. Observed the compression job table showed up as the compression job quickly finished and the compression stats showed up as well.
  4. Submitted an invalid job by command ./compress.sh non-existent-path. Observed the job record showed up in the table and hovering on the failure icon show a brief reason for the failure:
    image
  5. Launched 10 compression jobs at the same time (i.e., ./compress.sh ~/samples/hive-24hr/ &`). Observed the jobs to show up with either a PENDING or RUNNING status.
    image
  6. Eventually, all those jobs finished successfully.
    image

# Conflicts:
#	components/clp-package-utils/clp_package_utils/scripts/start_clp.py
#	components/webui/imports/api/ingestion/collections.js
#	components/webui/imports/api/ingestion/server/publications.js
#	components/webui/imports/api/search/constants.js
#	components/webui/imports/api/search/server/publications.js
#	components/webui/imports/ui/IngestView/IngestView.jsx
#	components/webui/imports/ui/SearchView/SearchResultsTable.jsx
#	components/webui/imports/ui/SearchView/SearchView.jsx
#	components/webui/imports/utils/DbManager.js
#	components/webui/server/main.js
#	components/webui/settings.json
Copy link
Member

@kirkrodrigues kirkrodrigues left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about:

webui: Add table to display the status of recent ingestion jobs. (#350)

@junhaoliao junhaoliao changed the title webui: Add <CompressionJobTable/>. webui: Add table to display the status of recent ingestion jobs. Apr 22, 2024
@junhaoliao junhaoliao merged commit 10e40d8 into y-scope:main Apr 22, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants