Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update jwqldb page with django model names #1681

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
Open

Update jwqldb page with django model names #1681

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f3dea37
Update jwqldb page with django model names
bhilbert4 Feb 14, 2025
9938caa
Remove unused lines
bhilbert4 Feb 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions jwql/utils/constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -581,6 +581,9 @@
"NIRSpec": "Mast.Jwst.Filtered.Nirspec",
}

# Database tables to exclude from the JWQLDB page
JWQLDB_EXCLUDED = ['RootFileInfo', 'Archive', 'Observation']

# JWST data products
JWST_DATAPRODUCTS = [
"IMAGE",
Expand Down
21 changes: 11 additions & 10 deletions jwql/website/apps/jwql/templates/jwqldb_table_viewer.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,18 +13,18 @@ <h2>Explore JWQL database tables through the web browser</h2><hr>

<p align="justify">
This page allows users to interactively explore the JWQL database tables. The main function of this tool is to
visually inspect data through the JWQL Web Application. Simply select a table from the dropdown menu to begin,
if further investigation of the table is needed, you can click the "Download Data" button to obtain the displayed
table as a CSV file.
visually inspect data through the JWQL Web Application. Simply select a table from the dropdown menu to display
the contents below. If further investigation of the table is needed, you can click the "Download Data" button to
obtain the displayed table as a CSV file.

<br><br>
The database table naming convention follows:

<xmp><instrument>_<monitor>_<stat_type></stat_type></xmp>
<xmp><Instrument><Monitor><Stat_type></xmp>

For example, if a user is interested in the statistics from the NIRCAM Bias Monitor, the table to investigate would be

<xmp>nircam_bias_stats</xmp>
<xmp>NircamBiasStats</xmp>
</p>

<hr>
Expand All @@ -34,14 +34,15 @@ <h2>Explore JWQL database tables through the web browser</h2><hr>
<h4>Select JWQL Database Table</h4>
<div class="db_table_select">
<select name="db_table_select" id="db_table_select" onchange="this.form.submit()">

{% if table_name %}
<option value="{{ table_name }}" selected>{{ table_name }}</option>
{% endif %}
{% for instrument in all_jwql_tables %}
<optgroup label="{{ instrument }}">
{% for table in all_jwql_tables[instrument] %}
<option value="{{ table }}">{{ table }}</option>
{% endfor %}
{% if tablename %}
<option value="{{ tablename }}" selected>{{ tablename }}</option>
{% endif %}
</optgroup>
{% endfor %}
</select>
Expand All @@ -51,7 +52,7 @@ <h4>Select JWQL Database Table</h4>
<!-- If tablename is passed, render table. -->
<!-- Table is not rendered when coming from home/monitor views -->
{% if table_name %}
<h4> {{ table_name|safe }} </h4><hr>
<h5> {{ table_name|safe }} </h5><hr>
<a href="{{ '/download_table/%s'%table_name }}" name=download_data class="btn btn-primary my-2" type="submit" value="{{ tablename }}">Download Data</a>
<table id="jwqltable" class="display" style="width:100%">
<thead>
Expand Down Expand Up @@ -84,7 +85,7 @@ <h4> {{ table_name|safe }} </h4><hr>
<h2>Importing A Table Using Python's Pandas Library</h2><hr>
<p>
After downloading JWQLDB data, it is simple to import this data in a python session
using pandas. Visit the <a href="https://pandas.pydata.org/docs/" target="_blank">pandas documentation</a>
using pandas. Visit the <a href="https://pandas.pydata.org/docs/" target="_blank">pandas documentation</a>
for more information.
<pre>
<code class="python">
Expand Down
18 changes: 9 additions & 9 deletions jwql/website/apps/jwql/views.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,8 @@
from django.shortcuts import redirect, render
from sqlalchemy import inspect

from jwql.database.database_interface import load_connection
from jwql.utils import monitor_utils
from jwql.utils.constants import JWST_INSTRUMENT_NAMES_MIXEDCASE, QUERY_CONFIG_TEMPLATE, URL_DICT, QueryConfigKeys
from jwql.utils.constants import JWQLDB_EXCLUDED, JWST_INSTRUMENT_NAMES_MIXEDCASE, QUERY_CONFIG_TEMPLATE, URL_DICT, QueryConfigKeys
from jwql.utils.interactive_preview_image import InteractivePreviewImg
from jwql.utils.utils import filename_parser, get_base_url, get_config, get_rootnames_for_instrument_proposal, query_unformat

Expand All @@ -81,6 +80,7 @@
get_image_info,
get_instrument_looks,
get_rootnames_from_query,
import_all_models,
random_404_page,
text_scrape,
thumbnails_ajax,
Expand Down Expand Up @@ -645,21 +645,21 @@ def jwqldb_table_viewer(request, tablename_param=None):
else:
table_meta = build_table(tablename)

_, _, engine, _ = load_connection(get_config()['connection_string'])
all_jwql_tables = inspect(engine).get_table_names()

if 'django_migrations' in all_jwql_tables:
all_jwql_tables.remove('django_migrations') # No necessary information.
all_jwql_tables = import_all_models()

jwql_tables_by_instrument = {}
instruments = ['nircam', 'nirspec', 'niriss', 'miri', 'fgs']

# Sort tables by instrument
for instrument in instruments:
jwql_tables_by_instrument[instrument] = [tablename for tablename in all_jwql_tables if instrument in tablename]
jwql_tables_by_instrument[instrument] = [tablename for tablename in all_jwql_tables if instrument in tablename.lower() and tablename not in JWQLDB_EXCLUDED]

# Wisp finder DB doesn't follow the naming convention. Add it here.
jwql_tables_by_instrument['nircam'].append('WispFinderB4QueryHistory')

# Don't forget tables that dont contain instrument specific instrument information.
jwql_tables_by_instrument['general'] = [table for table in all_jwql_tables if not any(instrument in table for instrument in instruments)]
jwql_tables_by_instrument['general'] = [table for table in all_jwql_tables if not any(instrument in table.lower() for instrument in instruments) and table not in JWQLDB_EXCLUDED]
jwql_tables_by_instrument['general'].remove('WispFinderB4QueryHistory')

template = 'jwqldb_table_viewer.html'

Expand Down