Project: Efficient Handling and Progressive Loading of Compressed Multiframe DICOM Images

[OzgeYurtsever]

Category

Cloud / Web

Key Investigators

• Ozge Yurtsever (Stanford, USA)
• Emel Alkim (Stanford, USA)

Project Description

Loading compressed multiframe DICOM images as a whole causes frequent browser crashes, particularly on Microsoft machines. This issue arises due to the large file size of the DICOM images, exceeding the browser's memory capacity.

The browser's rendering engine attempts to load the entire file into memory, due to the significant size of these images, the browser can quickly exhaust its allocated memory, leading to crashes or unresponsive behavior.

This issue affects both ePAD and OHIF with the latest WADO-loader version.

Objective

Initiate a discourse about the methodologies for saving, storing, and reading DICOM data, and explore strategies for optimizing the handling of compressed multiframe images to achieve enhanced efficiency and avoid browser crashing.

Approach and Plan

Instead of loading the entire DICOM file at once, the image loading process can be modified to load the image in smaller chunks or frames progressively. This approach may allow the browser to handle smaller portions of the image, reducing the memory burden and enhancing overall stability.

Progress and Next Steps

We attempted to adapt a solution approach inspired by the PR link below. The link's solution specifically addresses uncompressed images. In our case, we tried a similar method to handle compressed images within the dicom-parser library, unfortunately, the attempted solution did not yield the desired outcome.

PR link: cornerstonejs/cornerstoneWADOImageLoader#454 (comment)
Ticket link: cornerstonejs/dicomParser#248

Illustrations

Background and References

Unfortunately the ultrasound images are not deindentified, we can not provide sample data yet. We are working on getting a data set.

Related libraries:
https://github.com/cornerstonejs/cornerstoneWADOImageLoader
https://github.com/cornerstonejs/dicomParser

[sjh26]

@OzgeYurtsever Could you add a title for the project?

[fedorov]

@OzgeYurtsever can you provide the characteristics of the dataset that would allow reproducing the issue? We may be able to locate a publicly available sample in Imaging Data Commons.

[OzgeYurtsever]

@fedorov Two key variables contribute to this problem:

• firstly, the image must be in a multiframe format, such as those in ultrasound or nuclear medicine,
• secondly, it needs to be retrieved in a compressed state.

[fedorov]

The query below will identify all MR US series from IDC along with the URL and total file size.

SELECT
  SeriesInstanceUID,
  StudyInstanceUID,
  NumberOfFrames,
  SUM(instance_size)/POW(1024,3) AS series_size,
  CONCAT("https://viewer.imaging.datacommons.cancer.gov/viewer/",StudyInstanceUID,"?seriesInstanceUID=",SeriesInstanceUID) as viewer_url
FROM
  `bigquery-public-data.idc_current.dicom_all`
WHERE
  Modality = "US"
  AND SOPClassUID = "1.2.840.10008.5.1.4.1.1.3.1" # 	Ultrasound Multi-frame Image IOD
GROUP BY
  StudyInstanceUID,
  SeriesInstanceUID,
  NumberOfFrames
ORDER BY
  series_size desc

Here's a sample series returned: https://viewer.imaging.datacommons.cancer.gov/viewer/1.3.6.1.4.1.14519.5.2.1.1600.1201.280261060505804787745195562783?seriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.1600.1201.169862383836550038574427412141

None of the series I tried crashed the viewer.

If you are not sure what to do with the query above or how to work with IDC, you can get up to speed by following this tutorial: https://github.com/ImagingDataCommons/IDC-Tutorials/tree/master/notebooks/getting_started.

[github-actions]

Project Page Pull Request Creation

✅ COMPLETED: See #683

[OzgeYurtsever]

The query below will identify all MR US series from IDC along with the URL and total file size.

SELECT
  SeriesInstanceUID,
  StudyInstanceUID,
  NumberOfFrames,
  SUM(instance_size)/POW(1024,3) AS series_size,
  CONCAT("https://viewer.imaging.datacommons.cancer.gov/viewer/",StudyInstanceUID,"?seriesInstanceUID=",SeriesInstanceUID) as viewer_url
FROM
  `bigquery-public-data.idc_current.dicom_all`
WHERE
  Modality = "US"
  AND SOPClassUID = "1.2.840.10008.5.1.4.1.1.3.1" # 	Ultrasound Multi-frame Image IOD
GROUP BY
  StudyInstanceUID,
  SeriesInstanceUID,
  NumberOfFrames
ORDER BY
  series_size desc

Here's a sample series returned: https://viewer.imaging.datacommons.cancer.gov/viewer/1.3.6.1.4.1.14519.5.2.1.1600.1201.280261060505804787745195562783?seriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.1600.1201.169862383836550038574427412141

None of the series I tried crashed the viewer.

If you are not sure what to do with the query above or how to work with IDC, you can get up to speed by following this tutorial: https://github.com/ImagingDataCommons/IDC-Tutorials/tree/master/notebooks/getting_started.

@fedorov Thank you for your reply. I tried the OHIF viewer demo locally with a breast ultrasound, but it crashed almost immediately. Unfortunately, I am unable to share the images I uploaded due to the visible PID. I have attached a screenshot of the crash. Do you think using the OHIF demo may not provide the exact same experience as what you shared earlier? Does OHIF demo page utilize the same image retrieving and loading methods with the one you shared above?

[fedorov]

All I am trying to say is that if you can reproduce the problem with a public dataset, I think this can help debug the problem. I gave you some pointers to public datasets you can try to use to reproduce the problem. I do not know what is the causing the problem.

[OzgeYurtsever]

Thank you, I will try to find a dataset from the public sources that causes the same issue.

[emelalkim]

Hi @fedorov, I tried looking at Data Commons. Is there any compressed data there? Or is there a way to specify transfer syntax? I couldn't find it. This issue seems to be related to the images being compressed. (We are still trying to get some data we can share)

[fedorov]

@emelalkim you can use any DICOM attribute - including TransferSyntaxUID - as part of SQL query. If this turns out too difficult to figure out this week, I am happy to help at the PW!

[emelalkim]

Thanks @fedorov,
I tried a query like this and opened one of them but I'm not sure if it is compressed as response header for transfer syntax is /.
Let me see if you see something obvious, otherwise let's check it out at PW.
SELECT
SeriesInstanceUID,
StudyInstanceUID,
NumberOfFrames,
SUM(instance_size)/POW(1024,3) AS series_size,
CONCAT("https://viewer.imaging.datacommons.cancer.gov/viewer/",StudyInstanceUID,"?seriesInstanceUID=",SeriesInstanceUID, "&TransferSyntaxUID=1.2.840.10008.1.2.4.91") as viewer_url
FROM
bigquery-public-data.idc_current.dicom_all
WHERE
Modality = "US"
AND SOPClassUID = "1.2.840.10008.5.1.4.1.1.3.1" # Ultrasound Multi-frame Image IOD
GROUP BY
StudyInstanceUID,
SeriesInstanceUID,
NumberOfFrames
ORDER BY
series_size desc

[fedorov]

Emel, the URL is just a convenience shortcut to give you the URL to the viewer to open in the browser the series in each row. You should add the attributes to the WHERE statement. Here's the query that takes into account your new condition:

SELECT
  SeriesInstanceUID,
  StudyInstanceUID,
  NumberOfFrames,
  SUM(instance_size)/POW(1024,3) AS series_size,
  CONCAT("https://viewer.imaging.datacommons.cancer.gov/viewer/",StudyInstanceUID,"?seriesInstanceUID=",SeriesInstanceUID) as viewer_url
FROM
  `bigquery-public-data.idc_current.dicom_all`
WHERE
  Modality = "US"
  AND SOPClassUID = "1.2.840.10008.5.1.4.1.1.3.1" # 	Ultrasound Multi-frame Image IOD
  AND TransferSyntaxUID = "1.2.840.10008.1.2.4.91" # select JPEG2000 compression
GROUP BY
  StudyInstanceUID,
  SeriesInstanceUID,
  NumberOfFrames
ORDER BY
  series_size desc

Unfortunately, there are no US (or any other modality, except Slide Microscopy) with JPEG2000 compression in IDC :-(

[emelalkim]

Yeah, I had already tried that, and a couple other compression types. I was hoping I could give the Transfer Syntax to OHIF. :(

[emelalkim]

Hi @fedorov,
I was able to clean one multiframe DICOM. It is available at sample image
And after some troubleshooting we think it is about the size of the file, rather than compression. The sample image is uncompressed and a little over 500MB. In windows machines, both with Chrome and Edge, it crashes OHIF viewer almost instantly