Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic introductory material for GUIDE tutorials #707

Merged
merged 17 commits into from
Mar 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
06ed4f4
Basic introductory material for GUIDE tutorials
garrettmflynn Mar 27, 2024
08874d8
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 27, 2024
4cbd483
Merge branch 'main' into tutorial-base
CodyCBakerPhD Mar 27, 2024
06def87
reorganize based on feedback. add screenshots
garrettmflynn Mar 27, 2024
ed4034f
Merge branch 'tutorial-base' of https://github.com/NeurodataWithoutBo…
garrettmflynn Mar 27, 2024
3d485da
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 27, 2024
add418e
Update dataset section name
garrettmflynn Mar 27, 2024
e1e89d4
Merge branch 'tutorial-base' of https://github.com/NeurodataWithoutBo…
garrettmflynn Mar 27, 2024
1c41bab
Update dataset.rst
garrettmflynn Mar 27, 2024
b0e9303
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 27, 2024
cf36f8b
Crop dataset creation screenshot and rename to avoid numbers
garrettmflynn Mar 27, 2024
fe5eecf
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 27, 2024
a318763
Ensure both states of the dataset generation page are captured
garrettmflynn Mar 27, 2024
bf72f6a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 27, 2024
a6e5f71
Add created state screenshot
garrettmflynn Mar 27, 2024
6d23d83
Merge branch 'tutorial-base' of https://github.com/NeurodataWithoutBo…
garrettmflynn Mar 27, 2024
27c864e
Update docs/tutorials/multiple_sessions.rst
garrettmflynn Mar 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
*.[oa]
*~

_build
dist
out
tests/screenshots
Expand Down
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,6 @@ The resulting files are fully compliant with the best practices expected of the
:maxdepth: 2

installation
tutorials/index
format_support
developer_guide
48 changes: 48 additions & 0 deletions docs/tutorials/dataset.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
Dataset Generation
==================

Our tutorials focus on converting extracellular electrophysiology data in the SpikeGLX and Phy formats.
To get you started as quickly as possible, we’ve created a way to generate this Neuropixel-like dataset at the click of a button!

.. note::
The **SpikeGLX** data format stores electrophysiology recordings.

The **Phy** data format stores spike sorting results.

Navigate to the **Settings** page using the main sidebar. Then press the **Generate** button in the top-right corner to initiate the dataset creation.

.. figure:: ./screenshots/dataset-creation.png
:align: center
:alt: Dataset Creation Screen

Press the Generate button on the Settings page to create the dataset.

The generated dataset will be organized as follows:

.. code-block:: bash

dataset/
├── mouse1/
│ ├── mouse1_Session1/
│ │ ├── mouse1_Session1_g0/
│ │ │ ├── mouse1_Session1_g0_imec/
│ │ │ │ ├── mouse1_Session1_g0_imec.ap.bin
│ │ │ │ ├── mouse1_Session1_g0_imec.ap.meta
│ │ │ │ ├── mouse1_Session1_g0_imec.lf.bin
│ │ │ │ └── mouse1_Session1_g0_imec.lf.meta
│ │ │ └── mouse1_Session1_phy/
│ │ │
│ │ └── mouse1_Session2/
│ │ ├── mouse1_Session2_g0/
│ │ │ ...
│ │ └── mouse1_Session2_phy/
│ │ ...
│ │
└── mouse2/
├── mouse2_Session1/
│ ...
└── mouse2_Session2/
...

Now you’re ready to start your first conversion using the NWB GUIDE!
3 changes: 3 additions & 0 deletions docs/tutorials/dataset_publication.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Dataset Publication
=======================================
Coming soon...
17 changes: 17 additions & 0 deletions docs/tutorials/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
Tutorials
=======================================
The NWB Graphical User Interface for Data Entry (GUIDE) is a desktop tool for converting neurophysiological data
to the Neurodata Without Borders (NWB) format and uploading to the DANDI Archive. In these tutorials, we detail this
process from initial setup to final upload.

Before you begin these tutorials, **you'll need to generate the tutorial dataset** using the instructions on the Dataset page.


.. toctree::
:maxdepth: 2

dataset
single_session
multiple_sessions
dataset_publication
next_steps
3 changes: 3 additions & 0 deletions docs/tutorials/multiple_sessions.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Managing Multiple Sessions
==========================
Coming soon...
3 changes: 3 additions & 0 deletions docs/tutorials/next_steps.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
What's Next?
=======================================
Coming soon...
Binary file added docs/tutorials/screenshots/dataset-created.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/tutorials/screenshots/dataset-creation.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions docs/tutorials/single_session.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Converting a Single Session
=======================================
Coming soon...
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,8 @@
"deploy:electron:linux": "electron-builder build --linux --publish always",
"storybook": "storybook dev -p 6006",
"build-storybook": "storybook build",
"chromatic": "npx chromatic"
"chromatic": "npx chromatic",
"build:docs": "sphinx-build -b html docs/ _build/html/"
},
"lint-staged": {
"./**/*.{html|css|js|md|yml}": [
Expand Down
50 changes: 40 additions & 10 deletions tests/e2e.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,25 @@ import { fileURLToPath } from 'node:url'
import { homedir } from 'node:os'

import paths from "../paths.config.json" assert { type: "json" };
import { ScreenshotOptions } from 'puppeteer'

// ------------------------------------------------------------------
// ------------------------ Path Definitions ------------------------
// ------------------------------------------------------------------

const __dirname = dirname(fileURLToPath(import.meta.url));
const screenshotPath = join(__dirname, 'screenshots')
const screenshotPath = join(__dirname, '..', 'docs', 'tutorials', 'screenshots')
const guideRootPath = join(homedir(), paths.root)
const testRootPath = join(guideRootPath, '.test')
const testDataRootPath = join(testRootPath, 'test-data')
const testDataPath = join(testDataRootPath, 'data')
const testDatasetPath = join(testDataRootPath, 'dataset')

const windowDims = {
width: 1280,
height: 800
}

const alwaysDelete = [
join(testRootPath, 'pipelines'),
join(testRootPath, 'conversions'),
Expand Down Expand Up @@ -65,7 +71,8 @@ const subjectInfo = {
age: 'P30D'
}

const regenerateTestData = !existsSync(testDataRootPath) || false // Generate only if doesn't exist
// const regenerateTestData = !existsSync(testDataRootPath) || false // Generate only if doesn't exist
const regenerateTestData = true // Force regeneration

const dandiInfo = {
id: '212750',
Expand Down Expand Up @@ -96,11 +103,14 @@ describe('E2E Test', () => {

const references = connect()

let nScreenshots = 0
const takeScreenshot = async (label, delay = 0) => {
const takeScreenshot = async (label, delay = 0, options: ScreenshotOptions = { fullPage: true }) => {
if (delay) await sleep(delay)
await references.page.screenshot({ path: join(screenshotPath, `${nScreenshots}-${label}.png`), fullPage: true });
nScreenshots++

const pathToScreenshot = join(screenshotPath, `${label}.png`)

if (existsSync(pathToScreenshot)) return console.error(`Screenshot already exists: ${pathToScreenshot}`)

await references.page.screenshot({ path: pathToScreenshot, ...options });
}

const evaluate = async (...args) => await references.page.evaluate(...args)
Expand Down Expand Up @@ -150,7 +160,18 @@ describe('E2E Test', () => {

datasetTestFunction('Create tutorial dataset', async () => {

const outputLocation = await evaluate(async () => {
const x = 250 // Sidebar size
const width = windowDims.width - x

const screenshotClip = {
x,
y: 0,
width,
height: 220
}


await evaluate(async () => {

// Transition to settings page
const dashboard = document.querySelector('nwb-dashboard')
Expand All @@ -159,11 +180,20 @@ describe('E2E Test', () => {
// Generate test data
const page = dashboard.page
page.deleteTestData()
return await page.generateTestData()
})

await takeScreenshot('dataset-creation', 300, { clip: screenshotClip })

const outputLocation = await evaluate(async () => {
const dashboard = document.querySelector('nwb-dashboard')
const page = dashboard.page
const outputLocation = await page.generateTestData()
page.requestUpdate()
return outputLocation
})

// Take image after dataset generation
await takeScreenshot('dataset-creation', 500)
await takeScreenshot('dataset-created', 500, { clip: screenshotClip })

expect(existsSync(outputLocation)).toBe(true)

Expand Down Expand Up @@ -503,7 +533,7 @@ describe('E2E Test', () => {
})

test('Ensure there is one completed pipeline', async () => {
await takeScreenshot('home-page', 100)
await takeScreenshot('home-page-complete', 100)
const nPipelines = await evaluate(() => document.getElementById('guided-div-resume-progress-cards').children.length)
expect(nPipelines).toBe(1)
})
Expand Down
Loading