Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cathist failing due to PVL format/parsing bug in History object #1535

Closed
ascbot opened this issue Feb 12, 2019 · 6 comments
Closed

cathist failing due to PVL format/parsing bug in History object #1535

ascbot opened this issue Feb 12, 2019 · 6 comments
Labels
bug Something isn't working

Comments

@ascbot
Copy link
Contributor

ascbot commented Feb 12, 2019


Author Name: Kris Becker (Kris Becker)

Original Assignee: Tracie Sucharski


The cathist application is failing on some VIMS cubes because the PVL-based History object is ill-formed. Here is the corrupted contents of the History object of /work/projects/cassini/VIMS/Titan_UA_Raw/S60/V1654449360_4.ir.cub:

Object = vims2isis
IsisVersion = "3.4.2.4758 beta | 2012-08-28"
ProgramVersion = 2011-12-16
ProgramPath = /usgs/pkgs/isis3production2012-10-09/isis/bin
ExecutionDateTime = 2012-10-15T22:04:17
HostName = cn6.ib.astrohpc
UserName = tbecker
Description = "Converts Vims images to Isis 3 format"

Group = UserParameters
FROM = /archive/projects/cassini/VIMS/UnivAZraw/tour/S60/cubes/GLOBMAP002-
//V1654449360_4.QUB
VIS = /scratch/tbecker/Cassini/VIMS/Titan_UA_Raw/S60/V1654449360_4.vis.c-
ub
IR = /scratch/tbecker/Cassini/VIMS/Titan_UA_Raw/S60/V1654449360_4.ir.cub
End_Group
End_Object

Because the object is corrupted, one way to inspect contents is using dd:

dd ibs=1 skip=2233856 if=/work/projects/cassini/VIMS/Titan_UA_Raw/S60/V1654449360_4.ir.cub | more

The error generated from cathist is:

algol[187]: cathist from=/work/projects/cassini/VIMS/Titan_UA_Raw/S60/V1654449360_4.ir.cub
ERROR Error in PVL file on line [14].
ERROR Unable to read PVL keyword [FROM = /archive/projects/cassini/VIMS/UnivAZraw/tour/S60/cubes/GLOBMAP002-//V1654449360_4.QUB
VIS = /scratch/tbecker/Cassini/VIMS/Titan_UA_Raw/S60/V1654449360_4.vis.cub].
ERROR Expected a comment in PVL but found [FROM = /archive/projects/cassini/VIMS/UnivAZraw/tour/S60/cubes/GLOBMAP002-//V1654449360_4.QUB
VIS = /scratch/tbecker/Cassini/VIMS/Titan_UA_Raw/S60/V1654449360_4.vis.cub].

It appears the PVL formatting algorithm is breaking up the filename at the first occurring slash of two slashes and putting it on the next line. The reader/parser is then interpreting this as a comment.

(Perhaps an easy fix is to surround filenames with // in double quotes (i.e., as a string)?)

Steps to reproduce:

During the ingestion of the raw VIMS EDR with vims2isis, a second slash is placed between the directory and the filename in the FROM parameter - which is valid on Linux/Unix machines. When the history is created, as luck would have it, it breaks the filename at the first slash of two successive slashes putting the remainder of the file name on the next line. Parser has trouble with this.

So, run vims2isis as described above, then cathist.

@ascbot ascbot added this to the 3.4.4 (2013-06-25 Jun) milestone Feb 12, 2019
@ascbot ascbot added the bug Something isn't working label Feb 12, 2019
@ascbot
Copy link
Contributor Author

ascbot commented Feb 12, 2019


Original Redmine Comment
Author Name: Tammy Becker (Tammy Becker)
Original Date: 2012-10-22T18:32:27Z


Pipeline scripts will frequently have double slashes in pathnames, if only to ensure one exists for a path. Failure to recover processing history with 'cathist' or be able to run 'errors' on a large print file is not acceptable.

@ascbot
Copy link
Contributor Author

ascbot commented Feb 12, 2019


Original Redmine Comment
Author Name: Tammy Becker (Tammy Becker)
Original Date: 2012-12-01T00:22:37Z


Relationship added to 569 only because it seems to be a possibly related or might be impacted by the PVL refactor...this will need to be confirmed by a developer

@ascbot
Copy link
Contributor Author

ascbot commented Feb 12, 2019


Original Redmine Comment
Author Name: Anonymous (Anonymous)
Original Date: 2013-03-19T21:31:57Z


/work/projects/cassini/VIMS/Titan_UA_Raw/S60

ERROR Expected a comment in PVL but found ...

@ascbot
Copy link
Contributor Author

ascbot commented Feb 12, 2019


Original Redmine Comment
Author Name: Tracie Sucharski (Tracie Sucharski)
Original Date: 2013-06-05T21:46:44Z


IMPACT

This will fix bugs in any application which parses Pvl, where keywords happen to break at a "//" (double slash), with the "//" appearing as the first 2 characters on a line. Isis Pvl recognizes these as a comment line.

@ascbot
Copy link
Contributor Author

ascbot commented Feb 12, 2019


Original Redmine Comment
Author Name: Tracie Sucharski (Tracie Sucharski)
Original Date: 2013-06-05T21:55:04Z


I've made a fix and there is a test area in:

/work/projects/isis/latest/m01230/isis

@ascbot
Copy link
Contributor Author

ascbot commented Feb 12, 2019


Original Redmine Comment
Author Name: Kris Becker (Kris Becker)
Original Date: 2013-06-05T23:37:39Z


All looks good. Thanks...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant