Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDFView fails to find input files on command line when using relative paths #171

Open
lee218llnl opened this issue Jan 18, 2024 · 26 comments
Assignees
Labels
Component - HDFView Improvements to the visual interface layer Priority - 1. High 🔼 These are important issues that should be resolved in the next release Type - Bug Please report security issues to [email protected] instead of creating an issue on GitHub
Milestone

Comments

@lee218llnl
Copy link

I initially thought this was a bug in spack/spack#42153, but I think this may be an hdfview issue. When using the "# Default invocation when using modules" execution line in the hdfview.sh script, it appears that the argument -Dhdfview.root="$INSTALLDIR" causes hdfview to look in $INSTALLDIR for input files with a relative path. More concretely, if I run hdfview file.h5, I can see via strace that hdfview ends up looking for $INSTALLDIR/file.h5. It works OK if I run using the full path to the input file, such as hdfview $PWD/file.h5.

@derobins derobins added Type - Bug Please report security issues to [email protected] instead of creating an issue on GitHub Component - HDFView Improvements to the visual interface layer Priority - 1. High 🔼 These are important issues that should be resolved in the next release labels Jan 23, 2024
@derobins derobins added this to the 3.3.2 milestone Jan 23, 2024
@derobins derobins changed the title hdfview fails to find input files on command line when using relative paths HDFView fails to find input files on command line when using relative paths Jan 23, 2024
@byrnHDF
Copy link
Collaborator

byrnHDF commented Jan 25, 2024

Can you look at the https://github.com/HDFGroup/hdfview/blob/master/docs/WorkingDirectory.txt file and see if there is a workable solution for you, or we would welcome a suggestion.

@lee218llnl
Copy link
Author

I tried various things include changing the java invocation line to:

"$JAVABIN/java" "$JAVAOPTS" -Djava.library.path="$INSTALLDIR:$INSTALLDIR/ext" -Dhdfview.root="$INSTALLDIR" -cp "$INSTALLDIR/*" hdf.view.HDFView -Dhdfview.workdir=$PWD startDir=$PWD "$@"

and it still does not look in $PWD for the specified file:

[lee218@oslic8:h5py]$ strace -f -s 999 ~/myhdfview2 dummy.h5 |& grep dummy.h5
execve("/g/g0/lee218/myhdfview2", ["/g/g0/lee218/myhdfview2", "dummy.h5"], 0x7fffffffaf20 /* 115 vars */) = 0
write(2, "+ /usr/tce/packages/openjdk/openjdk-11.0.15_10/bin/java -Xmx1024M -Djava.library.path=/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh:/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh/ext -Dhdfview.root=/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh -cp '/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh/*' hdf.view.HDFView -Dhdfview.workdir=/g/g0/lee218/test/python/h5py startDir=/g/g0/lee218/test/python/h5py dummy.h5\n", 678+ /usr/tce/packages/openjdk/openjdk-11.0.15_10/bin/java -Xmx1024M -Djava.library.path=/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh:/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh/ext -Dhdfview.root=/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh -cp '/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh/*' hdf.view.HDFView -Dhdfview.workdir=/g/g0/lee218/test/python/h5py startDir=/g/g0/lee218/test/python/h5py dummy.h5
[pid 4074792] execve("/usr/tce/packages/openjdk/openjdk-11.0.15_10/bin/java", ["/usr/tce/packages/openjdk/openjdk-11.0.15_10/bin/java", "-Xmx1024M", "-Djava.library.path=/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh:/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh/ext", "-Dhdfview.root=/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh", "-cp", "/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh/*", "hdf.view.HDFView", "-Dhdfview.workdir=/g/g0/lee218/test/python/h5py", "startDir=/g/g0/lee218/test/python/h5py", "dummy.h5"], 0x55555588ced0 /* 116 vars */) = 0
[pid 4074793] read(29, "/usr/tce/packages/openjdk/openjdk-11.0.15_10/bin/java\0-Xmx1024M\0-Djava.library.path=/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh:/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh/ext\0-Dhdfview.root=/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh\0-cp\0/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh/*\0hdf.view.HDFView\0-Dhdfview.workdir=/g/g0/lee218/test/python/h5py\0startDir=/g/g0/lee218/test/python/h5py\0dummy.h5\0", 4096) = 674
[pid 4074793] stat("/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh/dummy.h5", 0x15555551e6c0) = -1 ENOENT (No such file or directory)
[pid 4074793] stat("/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh/dummy.h5", 0x15555551e6c0) = -1 ENOENT (No such file or directory)
[pid 4074793] stat("/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh/dummy.h5", 0x15555551e6c0) = -1 ENOENT (No such file or directory)
[pid 4074793] stat("/usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh/dummy.h5", 0x15555551e6f0) = -1 ENOENT (No such file or directory)
[pid 4074793] read(34, "#Thu Jan 25 09:40:18 PST 2024\nmodule.treeview=Default\nusers.guide=/usr/tce/backend/installations/linux-rhel8-x86_64/intel-2021.6.0/hdfview-3.3.0-7im76vfgbwfezxpxwoydpswyamramxzz/share/doc/UsersGuide/index.html\nmodule.metadataview=Default\nmodule.paletteview=Default\nmodule.imageview=Default\nmodule.fileformat.HDF4=hdf.object.h4.H4File\nmodule.fileformat.NetCDF3=hdf.object.nc2.NC2File\nmodule.fileformat.HDF5=hdf.object.h5.H5File\nmodule.fileformat.FITS=hdf.object.fits.FitsFile\nimage.contrast=general\nrecent.file3=/g/g0/lee218/test/hdf5/test.h5\nmodule.tableview=Default\nrecent.file4=/g/g0/lee218/e3sm_hang.pdf\nrecent.file1=/g/g0/lee218/test/python/h5py/dummy.h5\nrecent.file2=/usr/WS2/lee218/test/hdf5/test.h5\nrecent.file0=/usr/WS2/lee218/test/python/h5py/dummy.h5\n", 8192) = 761
[pid 4074793] stat("/usr/WS2/lee218/test/python/h5py/dummy.h5",  <unfinished ...>
[pid 4074793] stat("/g/g0/lee218/test/python/h5py/dummy.h5",  <unfinished ...>
[pid 4074793] write(34, "#Thu Jan 25 09:41:50 PST 2024\nmodule.treeview=Default\nusers.guide=/usr/tce/backend/installations/linux-rhel8-x86_64/intel-2021.6.0/hdfview-3.3.0-7im76vfgbwfezxpxwoydpswyamramxzz/share/doc/UsersGuide/index.html\nmodule.metadataview=Default\nmodule.paletteview=Default\nmodule.imageview=Default\nmodule.fileformat.HDF4=hdf.object.h4.H4File\nmodule.fileformat.NetCDF3=hdf.object.nc2.NC2File\nmodule.fileformat.HDF5=hdf.object.h5.H5File\nmodule.fileformat.FITS=hdf.object.fits.FitsFile\nimage.contrast=general\nrecent.file3=/g/g0/lee218/test/hdf5/test.h5\nmodule.tableview=Default\nrecent.file4=/g/g0/lee218/e3sm_hang.pdf\nrecent.file1=/g/g0/lee218/test/python/h5py/dummy.h5\nrecent.file2=/usr/WS2/lee218/test/hdf5/test.h5\nrecent.file0=/usr/WS2/lee218/test/python/h5py/dummy.h5\n", 761) = 761

It looks like it is still looking for the specified "dummy.h5" in /usr/tce/backend/installations/linux-rhel8-x86_64/oneapi-2022.1.0/hdfview-3.3.0-olq4dtqra45vclrqmym5gwijzokox6vh/dummy.h5, which is the installation directory, rather than $PWD. Note that the strace output where it actually finds the proper location is just because its in my "recent file" history.

Let me know if I am specifying the start/work directory incorrectly.

@byrnHDF
Copy link
Collaborator

byrnHDF commented Jan 25, 2024

The -D define should go before the program clas - see the hdfview.root define location.
Also make sure that the ini file does not override your command line. You should be able to open the user options preferences to see what everything is set to (without a file).

We do test these options but maybe I am doing something different, But startDir should have worked. Did you get a log file, you can enable that by adding some defines, the USING_logging.txt file.

I will check my test history.

@lee218llnl
Copy link
Author

I moved the -Dhdfview.workdir=$PWD arg to before the hdf.view.HDFVIEW arg and it still did not work. Can you reproduce the behavior that I reported and do some testing and send me the script that works for you so I don't have to keep fumbling around? Thanks.

@byrnHDF
Copy link
Collaborator

byrnHDF commented Jan 25, 2024

Note that I did use the latest 3.3.1 version however everything worked with the define on the command line
-Dhdfview.workdir="$PWD" <---- note the quotes
or adding $PWD to the HDFView.cfg file with the launcher.
I did not try the startdir option.

@byrnHDF
Copy link
Collaborator

byrnHDF commented Jan 25, 2024

The launcher worked (after edit) without any other changes. However, the supplied hdfview.sh does need to set the the INSTALLDIR up two dirs from the .sh location (export INSTALLDIR=$dir/../..).

@lee218llnl
Copy link
Author

still no luck with me when I run:

"$JAVABIN/java" "$JAVAOPTS" -Djava.library.path="$INSTALLDIR:$INSTALLDIR/ext" -Dhdfview.root="$INSTALLDIR" -cp "$INSTALLDIR/*" -Dhdfview.workdir="$PWD" hdf.view.HDFView "$@"

This is an hdfview 3.3.0 installation, if that makes a difference. Also, in my strace, I don't see it accessing an HDFView.cfg file.

@byrnHDF
Copy link
Collaborator

byrnHDF commented Jan 26, 2024

I don't think this workdir process changed from 3.3.0 but I will check.
There are two ways to invoke HDFView, from the launcher in the $INSTALLDIR/bin folder or the script from the $INSTALLDIR/lib/app folder. The Launcher uses the cfg file.

@byrnHDF
Copy link
Collaborator

byrnHDF commented Jan 26, 2024

Looks like we fixed the workdir option in 3.3.1 and forgot to announce it in the release notes.
Please try 3.3.1.

@lee218llnl
Copy link
Author

still fails for me with 3.3.1 (installed with spack, but added new version and checksum).

I have my script run this command:

"$JAVABIN/java" "$JAVAOPTS" -Djava.library.path="$INSTALLDIR:$INSTALLDIR/ext" -Dhdfview.root="$INSTALLDIR" -cp "$INSTALLDIR/*" -Dhdfview.workdir="$PWD" hdf.view.HDFView "$@"

and it does not appear to be looking for an HDFview.cfg file:

[lee218@rzwhippet18:spack]$ strace -f -s 9999 /usr/WS2/lee218/delete/spack/opt/spack/linux-rhel8-ivybridge/gcc-10.3.1/hdfview-3.3.1-ziimwueprhe3m5adhnl35ca3jgifkc4t/bin/hdfview $PWD/data.h5 |& grep -i hdfview.cfg
[lee218@rzwhippet18:spack]$ echo $?
1

@byrnHDF
Copy link
Collaborator

byrnHDF commented Jan 30, 2024

and it does not appear to be looking for an HDFview.cfg file:
The shell script would not use the .cfg, only the launcher binary.

I'm guessing you appending the filename to script, that I did not try. After launching HDFView I went to open file menu and the PWD was the selected folder.

I will need to try, but imagine that the HDFView argument processing may be not prepending the workdir.

@lee218llnl
Copy link
Author

please read the initial issue report, where I explicitly stated that the issue was opening a file with a relative path from the command line. I hope you can appreciate that I have wasted a fair amount of time on this.

@byrnHDF
Copy link
Collaborator

byrnHDF commented Jan 30, 2024

Actually, have you tried to add the "-start=$PWD" to the command;

... hdf.view.HDFView -start="$PWD" "$@"

@byrnHDF
Copy link
Collaborator

byrnHDF commented Jan 30, 2024

See WorkingDirectory.txt for a description of arguments to HDFView. (-start is at line 16)

@lee218llnl
Copy link
Author

still no luck:

[lee218@rzwhippet18:spack]$ tail -n 1 /usr/WS2/lee218/delete/spack/opt/spack/linux-rhel8-ivybridge/gcc-10.3.1/hdfview-3.3.1-ziimwueprhe3m5adhnl35ca3jgifkc4t/bin/hdfview 
"$JAVABIN/java" "$JAVAOPTS" -Djava.library.path="$INSTALLDIR:$INSTALLDIR/ext" -Dhdfview.root="$INSTALLDIR" -cp "$INSTALLDIR/*" -Dhdfview.workdir="$PWD" hdf.view.HDFView -start="$PWD" "$@"
[lee218@rzwhippet18:spack]$ echo $PWD
/usr/workspace/lee218/delete/spack
[lee218@rzwhippet18:spack]$ strace -f -s 9999 /usr/WS2/lee218/delete/spack/opt/spack/linux-rhel8-ivybridge/gcc-10.3.1/hdfview-3.3.1-ziimwueprhe3m5adhnl35ca3jgifkc4t/bin/hdfview data.h5  |& grep data.h5 | grep stat | head -n 4
[pid 421255] stat("/usr/WS2/lee218/delete/spack/opt/spack/linux-rhel8-ivybridge/gcc-10.3.1/hdfview-3.3.1-ziimwueprhe3m5adhnl35ca3jgifkc4t/data.h5", 0x15555551c6c0) = -1 ENOENT (No such file or directory)
[pid 421255] stat("/usr/WS2/lee218/delete/spack/opt/spack/linux-rhel8-ivybridge/gcc-10.3.1/hdfview-3.3.1-ziimwueprhe3m5adhnl35ca3jgifkc4t/data.h5", 0x15555551c6c0) = -1 ENOENT (No such file or directory)
[pid 421255] stat("/usr/WS2/lee218/delete/spack/opt/spack/linux-rhel8-ivybridge/gcc-10.3.1/hdfview-3.3.1-ziimwueprhe3m5adhnl35ca3jgifkc4t/data.h5", 0x15555551c6c0) = -1 ENOENT (No such file or directory)
[pid 421255] stat("/usr/WS2/lee218/delete/spack/opt/spack/linux-rhel8-ivybridge/gcc-10.3.1/hdfview-3.3.1-ziimwueprhe3m5adhnl35ca3jgifkc4t/data.h5", 0x15555551c6f0) = -1 ENOENT (No such file or directory)

Can you please do the testing yourself and let me know when you have a proper fix?

@byrnHDF
Copy link
Collaborator

byrnHDF commented Jan 30, 2024

The simple workaround is to add "$PWD" to the "$@" -> "$PWD/$@"

@lee218llnl
Copy link
Author

modifying the hdfview launcher script to modify the user's args to "$PWD/$@" would break things if the user specifies a full path or wants to specify any other command-line arguments. I already told my users that they can run hdfview $PWD/file.h5 from the command line, so that is our workaround. If you decide that this is something you want to fix, then please let me know. I don't plan on spending any more time on this.

@byrnHDF
Copy link
Collaborator

byrnHDF commented Jan 30, 2024

Will try to fix it in the next release this spring as it looks like it is an implementation bug. It seems to not consider the startDir setting(looks like multiple ways of setting it as well) and tries to pull the dir from the file.

@lee218llnl
Copy link
Author

any updates on a fix and release?

@byrnHDF
Copy link
Collaborator

byrnHDF commented May 21, 2024

Changes completed - Having trouble signing the packages.

@byrnHDF
Copy link
Collaborator

byrnHDF commented May 30, 2024

Completed for next release

@byrnHDF byrnHDF closed this as completed May 30, 2024
@lee218llnl
Copy link
Author

I see the same issue with version 3.3.2 when I specify the relative path:

[lee218@dane2:h5py]$ strace -f -s 9999 /usr/WS2/lee218/delete/HDFView-3.3.2/build/HDF_Group/HDFView/3.3.2/hdfview.sh dummy.h5 >& out.txt ; grep dummy.h5 out.txt | grep stat
[pid 648963] stat("/usr/WS2/lee218/delete/HDFView-3.3.2/build/HDF_Group/HDFView/3.3.2/dummy.h5", 0x15555550a6b0) = -1 ENOENT (No such file or directory)
[pid 648963] stat("/usr/WS2/lee218/delete/HDFView-3.3.2/build/HDF_Group/HDFView/3.3.2/dummy.h5", 0x15555550a6b0) = -1 ENOENT (No such file or directory)
[pid 648963] stat("/usr/WS2/lee218/delete/HDFView-3.3.2/build/HDF_Group/HDFView/3.3.2/dummy.h5", 0x15555550a6b0) = -1 ENOENT (No such file or directory)
[pid 648963] stat("/usr/WS2/lee218/delete/HDFView-3.3.2/build/HDF_Group/HDFView/3.3.2/dummy.h5", 0x15555550a6d0) = -1 ENOENT (No such file or directory)
[pid 648963] stat("/g/g0/lee218/test/python/h5py/dummy.h5", {st_mode=S_IFREG|0600, st_size=800, ...}) = 0

this is what I see with a full path to dummy.h5:

[lee218@dane2:h5py]$ strace -f -s 9999 /usr/WS2/lee218/delete/HDFView-3.3.2/build/HDF_Group/HDFView/3.3.2/hdfview.sh $PWD/dummy.h5 >& out.txt ; grep dummy.h5 out.txt | grep stat
[pid 649002] stat("/g/g0/lee218/test/python/h5py/dummy.h5", {st_mode=S_IFREG|0600, st_size=800, ...}) = 0
[pid 649002] stat("/g/g0/lee218/test/python/h5py/dummy.h5", {st_mode=S_IFREG|0600, st_size=800, ...}) = 0
[pid 649002] stat("/g/g0/lee218/test/python/h5py/dummy.h5", {st_mode=S_IFREG|0600, st_size=800, ...}) = 0
[pid 649002] stat("/g/g0/lee218/test/python/h5py/dummy.h5", {st_mode=S_IFREG|0600, st_size=800, ...}) = 0
[pid 649002] stat("/g/g0/lee218/test/python/h5py/dummy.h5", {st_mode=S_IFREG|0600, st_size=800, ...}) = 0
[pid 649002] stat("/g/g0/lee218/test/python/h5py/dummy.h5", {st_mode=S_IFREG|0600, st_size=800, ...}) = 0
[pid 649002] stat("/g/g0/lee218/test/python/h5py/dummy.h5", {st_mode=S_IFREG|0600, st_size=800, ...}) = 0
[pid 649002] stat("/g/g0/lee218/test/python/h5py/dummy.h5", {st_mode=S_IFREG|0600, st_size=800, ...}) = 0
[pid 649002] stat("/g/g0/lee218/test/python/h5py/dummy.h5", {st_mode=S_IFREG|0600, st_size=800, ...}) = 0
[pid 649002] lstat("/g/g0/lee218/test/python/h5py/dummy.h5", {st_mode=S_IFREG|0600, st_size=800, ...}) = 0

@lee218llnl
Copy link
Author

And in the GUI, with the relative path, the .h5 file does not display, while it does display with the full path.

@byrnHDF
Copy link
Collaborator

byrnHDF commented Sep 23, 2024

Okay, I will need to go back and see where my testing failed.

@byrnHDF
Copy link
Collaborator

byrnHDF commented Sep 23, 2024

I think I have found the problem, maybe I missed the change I tested, however, the input files should be based off the "startDir" and not the "rootDir" for relative files. This happens in the hdf/view/HDFView.java file main function around line 2813.

@byrnHDF byrnHDF reopened this Sep 23, 2024
@byrnHDF
Copy link
Collaborator

byrnHDF commented Sep 23, 2024

I just created some snapshots for this at https://github.com/HDFGroup/hdfview/releases/tag/HDFView-3.3.99

This should fix the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component - HDFView Improvements to the visual interface layer Priority - 1. High 🔼 These are important issues that should be resolved in the next release Type - Bug Please report security issues to [email protected] instead of creating an issue on GitHub
Projects
None yet
Development

No branches or pull requests

3 participants