Skip to content

Commit

Permalink
tools/chdman.cpp: Fixed numerous issues, including:
Browse files Browse the repository at this point in the history 
Support input start/size options for createdvd.

Fixed not reporting an error on unrecognised command line options.

Fixed --fix/-f option for verify command not working.

Report an error when conflicting options are supplied (e.g. hard disk
template and C/H/S geometry, or input start offset in both bytes and
hunks).  Previously the results would be unpredictable.

Detect more invalid combinations of options, and detect when output unit
size or hunk size doesn't match parent.

Changed order of processing options for createhd so using a template
cannot not inadvertently result in an invalid combination of sector size
and hunk size.

Don't require an explicit unit size for createraw if an output parent
CHD file is supplied.

Fixed an object leak in createcd.
  • Loading branch information
@cuavas
cuavas committed Feb 9, 2024
1 parent d88e127 commit 5731492
Show file tree
Hide file tree
Showing 4 changed files with 609 additions and 572 deletions.
109 changes: 86 additions & 23 deletions docs/source/tools/chdman.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,25 @@ used by multiple commands:
most commands that operate on CHD format input files. This option must be
used if the input file is a *delta CHD*, storing only the hunks that differ
from its parent CHD,
--inputstartbyte <offset> / -isb <offset>
Specify the offset to the data in the input file in bytes. This is useful
for creating CHD format files from input files that contain a header before
the start of the data, or for extracting partial content from a CHD format
file. May not be specified in combination with the
``--inputstarthunk``/``-ish`` option.
--inputstarthunk <offset> / -ish <offset>
Specify the offset to the data in the input file in hunks. May not be
specified in combination with the ``--inputstartbyte``/``-isb`` option.
--inputbytes <length> / -ib <length>
Specify the amount of input data to use in bytes, starting from the offset
to the data in the input file. This is useful for creating CHD format files
from input files that contain additional content after the data, or for
extracting partial content from a CHD format file. May not be specified in
combination with the ``--inputhunks``/``-ih`` option.
--inputhunks <length> / -ih <length>
Specify the amount of input data to use in hunks, starting from the offset
to the data in the input file. May not be specified in combination with the
``--inputbytes``/``-ib`` option.
--output <file> / -o <file>
Specify the output file name. This option is required for commands that
produce output files. The output file formats supported depend on the
Expand Down Expand Up @@ -86,13 +105,19 @@ verify
~~~~~~

Verify the integrity of a CHD format file. The input file must be a read-only
CHD format file (the integrity of writable CHD files cannot be verified).
CHD format file (the integrity of writable CHD files cannot be verified). Note
that this command modifies its input file if the ``--fix``/``-f`` option is
specified.

Common options supported:

* ``--input <file>`` / ``-i <file>`` (required)
* ``--inputparent <chdfile>`` / ``-ip <chdfile>``

Additional options:

* ``--fix`` / ``-f``

createraw
~~~~~~~~~

Expand All @@ -101,20 +126,34 @@ Create a CHD format file from a raw media image.
Common options supported:

* ``--input <file>`` / ``-i <file>`` (required)
* ``--inputstartbyte <offset>`` / ``-isb <offset>``
* ``--inputstarthunk <offset>`` / ``-ish <offset>``
* ``--inputbytes <length>`` / ``-ib <length>``
* ``--inputhunks <length>`` / ``-ih <length>``
* ``--output <file>`` / ``-o <file>`` (required)
* ``--outputparent <chdfile>`` / ``-op <chdfile>``
* ``--compression none|<type>[,<type>]...`` / ``-c none|<type>[,<type>]...``
* ``--hunksize <bytes>`` / ``-hs <bytes>`` (required)
* ``--hunksize <bytes>`` / ``-hs <bytes>``
* ``--force`` / ``-f``
* ``--numprocessors <count>`` / ``-np <count>``

Additional options:

* ``--unitsize <bytes>`` / ``-us <bytes>`` (required)
* ``--inputstartbyte <offset>`` / ``-isb <offset>``
* ``--inputstarthunk <offset>`` / ``-ish <offset>``
* ``--inputbytes <length>`` / ``-ib <length>``
* ``--inputhunks <length>`` / ``-ih <length>``
--unitsize <bytes> / -us <bytes> (required)
The unit size for the output CHD file in bytes. This is the smallest unit
of data that can be addressed within the CHD file. It should match the
sector size or page size of the source media. The hunk size must be a whole
multiple of the unit size. The unit size must be specified if no parent CHD
file for the output is supplied. If a parent CHD file for the output is
supplied, the unit size must match the unit size of the parent CHD file.

If the ``--hunksize`` or ``-hs`` option is not supplied, the default will be:

* The hunk size of the parent CHD file if a parent CHD file for the output is
supplied.
* The smallest whole multiple of the unit size not larger than 4 KiB if the unit
size is not larger than 4 KiB (4096 bytes).
* The unit size if it is larger than 4 KiB (4096 bytes).

If the ``--compression`` or ``-c`` option is not supplied, it defaults to
``lzma,zlib,huff,flac``.
Expand All @@ -126,7 +165,11 @@ Create a CHD format hard disk image file.

Common options supported:

* ``--input <file>`` / ``-i <file>`` (required)
* ``--input <file>`` / ``-i <file>``
* ``--inputstartbyte <offset>`` / ``-isb <offset>``
* ``--inputstarthunk <offset>`` / ``-ish <offset>``
* ``--inputbytes <length>`` / ``-ib <length>``
* ``--inputhunks <length>`` / ``-ih <length>``
* ``--output <file>`` / ``-o <file>`` (required)
* ``--outputparent <chdfile>`` / ``-op <chdfile>``
* ``--compression none|<type>[,<type>]...`` / ``-c none|<type>[,<type>]...``
Expand All @@ -140,13 +183,23 @@ Additional options:
* ``--size <bytes>`` / ``-s <bytes>``
* ``--chs <cylinders>,<heads>,<sectors>`` / ``-chs <cylinders>,<heads>,<sectors>``
* ``--template <template>`` / ``-tp <template>``
* ``--inputstartbyte <offset>`` / ``-isb <offset>``
* ``--inputstarthunk <offset>`` / ``-ish <offset>``
* ``--inputbytes <length>`` / ``-ib <length>``
* ``--inputhunks <length>`` / ``-ih <length>``

Creates a blank (zero-filled) hard disk image if no input file is supplied. The
input start/length (``--inputstartbyte``/``-isb``,
``--inputstarthunk``/``-ish``, ``--inputbytes``/``-ib`` and
``--inputhunks``/``-ih`` options) cannot be used if no input file is supplied.

If the ``--hunksize`` or ``-hs`` option is not supplied, the default will be:

* The hunk size of the parent CHD file if a parent CHD file for the output is
supplied.
* The smallest whole multiple of the sector size not larger than 4 KiB if the
sector size is not larger than 4 KiB (4096 bytes).
* The sector size if it is larger than 4 KiB (4096 bytes).

If the ``--compression`` or ``-c`` option is not supplied, it defaults to
``lzma,zlib,huff,flac``.
``lzma,zlib,huff,flac`` if an input file is supplied, or ``none`` if no input
file is supplied.

createcd
~~~~~~~~
Expand All @@ -163,6 +216,10 @@ Common options supported:
* ``--force`` / ``-f``
* ``--numprocessors <count>`` / ``-np <count>``

If the ``--hunksize`` or ``-hs`` option is not supplied, the default will be
the hunk size of the parent CHD if a parent CHD file for the output is supplied,
or eight sectors per hunk (18,816 bytes) otherwise.

If the ``--compression`` or ``-c`` option is not supplied, it defaults to
``cdlz,cdzl,cdfl``.

Expand All @@ -174,13 +231,21 @@ Create a CHD format DVD-ROM image file.
Common options supported:

* ``--input <file>`` / ``-i <file>`` (required)
* ``--inputstartbyte <offset>`` / ``-isb <offset>``
* ``--inputstarthunk <offset>`` / ``-ish <offset>``
* ``--inputbytes <length>`` / ``-ib <length>``
* ``--inputhunks <length>`` / ``-ih <length>``
* ``--output <file>`` / ``-o <file>`` (required)
* ``--outputparent <chdfile>`` / ``-op <chdfile>``
* ``--compression none|<type>[,<type>]...`` / ``-c none|<type>[,<type>]...``
* ``--hunksize <bytes>`` / ``-hs <bytes>`` (required)
* ``--force`` / ``-f``
* ``--numprocessors <count>`` / ``-np <count>``

If the ``--hunksize`` or ``-hs`` option is not supplied, the default will be
the hunk size of the parent CHD if a parent CHD file for the output is supplied,
or two sectors per hunk (4096 bytes) otherwise.

If the ``--compression`` or ``-c`` option is not supplied, it defaults to
``lzma,zlib,huff,flac``.

Expand Down Expand Up @@ -216,15 +281,12 @@ Common options supported:

* ``--input <file>`` / ``-i <file>`` (required)
* ``--inputparent <chdfile>`` / ``-ip <chdfile>``
* ``--output <file>`` / ``-o <file>`` (required)
* ``--force`` / ``-f``

Additional options:

* ``--inputstartbyte <offset>`` / ``-isb <offset>``
* ``--inputstarthunk <offset>`` / ``-ish <offset>``
* ``--inputbytes <length>`` / ``-ib <length>``
* ``--inputhunks <length>`` / ``-ih <length>``
* ``--output <file>`` / ``-o <file>`` (required)
* ``--force`` / ``-f``

extracthd
~~~~~~~~~
Expand All @@ -235,15 +297,12 @@ Common options supported:

* ``--input <file>`` / ``-i <file>`` (required)
* ``--inputparent <chdfile>`` / ``-ip <chdfile>``
* ``--output <file>`` / ``-o <file>`` (required)
* ``--force`` / ``-f``

Additional options:

* ``--inputstartbyte <offset>`` / ``-isb <offset>``
* ``--inputstarthunk <offset>`` / ``-ish <offset>``
* ``--inputbytes <length>`` / ``-ib <length>``
* ``--inputhunks <length>`` / ``-ih <length>``
* ``--output <file>`` / ``-o <file>`` (required)
* ``--force`` / ``-f``

extractcd
~~~~~~~~~
Expand All @@ -270,6 +329,10 @@ Common options supported:

* ``--input <file>`` / ``-i <file>`` (required)
* ``--inputparent <chdfile>`` / ``-ip <chdfile>``
* ``--inputstartbyte <offset>`` / ``-isb <offset>``
* ``--inputstarthunk <offset>`` / ``-ish <offset>``
* ``--inputbytes <length>`` / ``-ib <length>``
* ``--inputhunks <length>`` / ``-ih <length>``
* ``--output <file>`` / ``-o <file>`` (required)
* ``--force`` / ``-f``

Expand Down
32 changes: 15 additions & 17 deletions src/lib/util/chd.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -585,7 +585,7 @@ void chd_file::set_parent_sha1(util::sha1_t parent)
}

/**
* @fn std::error_condition chd_file::create(util::random_read_write::ptr &&file, uint64_t logicalbytes, uint32_t hunkbytes, uint32_t unitbytes, chd_codec_type compression[4])
* @fn std::error_condition chd_file::create(util::random_read_write::ptr &&file, uint64_t logicalbytes, uint32_t hunkbytes, uint32_t unitbytes, const chd_codec_type (&compression)[4])
*
* @brief -------------------------------------------------
* create - create a new file with no parent using an existing opened file handle
Expand All @@ -605,7 +605,7 @@ std::error_condition chd_file::create(
uint64_t logicalbytes,
uint32_t hunkbytes,
uint32_t unitbytes,
chd_codec_type compression[4])
const chd_codec_type (&compression)[4])
{
// make sure we don't already have a file open
if (m_file)
Expand All @@ -626,7 +626,7 @@ std::error_condition chd_file::create(
}

/**
* @fn std::error_condition chd_file::create(util::random_read_write::ptr &&file, uint64_t logicalbytes, uint32_t hunkbytes, chd_codec_type compression[4], chd_file &parent)
* @fn std::error_condition chd_file::create(util::random_read_write::ptr &&file, uint64_t logicalbytes, uint32_t hunkbytes, const chd_codec_type (&compression)[4], chd_file &parent)
*
* @brief -------------------------------------------------
* create - create a new file with a parent using an existing opened file handle
Expand All @@ -645,7 +645,7 @@ std::error_condition chd_file::create(
util::random_read_write::ptr &&file,
uint64_t logicalbytes,
uint32_t hunkbytes,
chd_codec_type compression[4],
const chd_codec_type (&compression)[4],
chd_file &parent)
{
// make sure we don't already have a file open
Expand All @@ -667,7 +667,7 @@ std::error_condition chd_file::create(
}

/**
* @fn std::error_condition chd_file::create(std::string_view filename, uint64_t logicalbytes, uint32_t hunkbytes, uint32_t unitbytes, chd_codec_type compression[4])
* @fn std::error_condition chd_file::create(std::string_view filename, uint64_t logicalbytes, uint32_t hunkbytes, uint32_t unitbytes, const chd_codec_type (&compression)[4])
*
* @brief -------------------------------------------------
* create - create a new file with no parent using a filename
Expand All @@ -687,7 +687,7 @@ std::error_condition chd_file::create(
uint64_t logicalbytes,
uint32_t hunkbytes,
uint32_t unitbytes,
chd_codec_type compression[4])
const chd_codec_type (&compression)[4])
{
// make sure we don't already have a file open
if (m_file)
Expand All @@ -712,7 +712,7 @@ std::error_condition chd_file::create(
}

/**
* @fn std::error_condition chd_file::create(std::string_view filename, uint64_t logicalbytes, uint32_t hunkbytes, chd_codec_type compression[4], chd_file &parent)
* @fn std::error_condition chd_file::create(std::string_view filename, uint64_t logicalbytes, uint32_t hunkbytes, const chd_codec_type (&compression)[4], chd_file &parent)
*
* @brief -------------------------------------------------
* create - create a new file with a parent using a filename
Expand All @@ -731,7 +731,7 @@ std::error_condition chd_file::create(
std::string_view filename,
uint64_t logicalbytes,
uint32_t hunkbytes,
chd_codec_type compression[4],
const chd_codec_type (&compression)[4],
chd_file &parent)
{
// make sure we don't already have a file open
Expand Down Expand Up @@ -1969,9 +1969,9 @@ std::error_condition chd_file::compress_v5_map()
{
uint8_t curcomp = m_rawmap[hunknum * 12 + 0];

// promote self block references to more compact forms
if (curcomp == COMPRESSION_SELF)
{
// promote self block references to more compact forms
uint32_t refhunk = get_u48be(&m_rawmap[hunknum * 12 + 4]);
if (refhunk == last_self)
curcomp = COMPRESSION_SELF_0;
Expand All @@ -1981,10 +1981,9 @@ std::error_condition chd_file::compress_v5_map()
max_self = std::max(max_self, refhunk);
last_self = refhunk;
}

// promote parent block references to more compact forms
else if (curcomp == COMPRESSION_PARENT)
{
// promote parent block references to more compact forms
uint32_t refunit = get_u48be(&m_rawmap[hunknum * 12 + 4]);
if (refunit == mulu_32x32(hunknum, m_hunkbytes) / m_unitbytes)
curcomp = COMPRESSION_PARENT_SELF;
Expand Down Expand Up @@ -2921,9 +2920,9 @@ std::error_condition chd_file_compressor::compress_continue(double &progress, do
osd_work_item_release(item.m_osd);
item.m_osd = nullptr;

// for parent walking, just add to the hashmap
if (m_walking_parent)
{
// for parent walking, just add to the hashmap
uint32_t uph = hunk_bytes() / unit_bytes();
uint32_t units = uph;
if (item.m_hunknum == hunk_count() - 1 || !compressed())
Expand All @@ -2932,10 +2931,9 @@ std::error_condition chd_file_compressor::compress_continue(double &progress, do
if (m_parent_map.find(item.m_hash[unit].m_crc16, item.m_hash[unit].m_sha1) == hashmap::NOT_FOUND)
m_parent_map.add(item.m_hunknum * uph + unit, item.m_hash[unit].m_crc16, item.m_hash[unit].m_sha1);
}

// if we're uncompressed, use regular writes
else if (!compressed())
{
// if we're uncompressed, use regular writes
std::error_condition err = write_hunk(item.m_hunknum, item.m_data);
if (err)
return err;
Expand All @@ -2947,10 +2945,10 @@ std::error_condition chd_file_compressor::compress_continue(double &progress, do
if (codec == CHD_CODEC_NONE)
m_total_out += m_hunkbytes;
}

// for compressing, process the result
else do
{
// for compressing, process the result

// first see if the hunk is in the parent or self maps
uint64_t selfhunk = m_current_map.find(item.m_hash[0].m_crc16, item.m_hash[0].m_sha1);
if (selfhunk != hashmap::NOT_FOUND)
Expand Down Expand Up @@ -3079,7 +3077,7 @@ void chd_file_compressor::async_walk_parent(work_item &item)

void *chd_file_compressor::async_compress_hunk_static(void *param, int threadid)
{
auto *item = reinterpret_cast<work_item *>(param);
auto *const item = reinterpret_cast<work_item *>(param);
item->m_compressor->async_compress_hunk(*item, threadid);
return nullptr;
}
Expand Down
8 changes: 4 additions & 4 deletions src/lib/util/chd.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -323,10 +323,10 @@ class chd_file
void set_parent_sha1(util::sha1_t parent);

// file create
std::error_condition create(std::string_view filename, uint64_t logicalbytes, uint32_t hunkbytes, uint32_t unitbytes, chd_codec_type compression[4]);
std::error_condition create(util::random_read_write::ptr &&file, uint64_t logicalbytes, uint32_t hunkbytes, uint32_t unitbytes, chd_codec_type compression[4]);
std::error_condition create(std::string_view filename, uint64_t logicalbytes, uint32_t hunkbytes, chd_codec_type compression[4], chd_file &parent);
std::error_condition create(util::random_read_write::ptr &&file, uint64_t logicalbytes, uint32_t hunkbytes, chd_codec_type compression[4], chd_file &parent);
std::error_condition create(std::string_view filename, uint64_t logicalbytes, uint32_t hunkbytes, uint32_t unitbytes, const chd_codec_type (&compression)[4]);
std::error_condition create(util::random_read_write::ptr &&file, uint64_t logicalbytes, uint32_t hunkbytes, uint32_t unitbytes, const chd_codec_type (&compression)[4]);
std::error_condition create(std::string_view filename, uint64_t logicalbytes, uint32_t hunkbytes, const chd_codec_type (&compression)[4], chd_file &parent);
std::error_condition create(util::random_read_write::ptr &&file, uint64_t logicalbytes, uint32_t hunkbytes, const chd_codec_type (&compression)[4], chd_file &parent);

// file open
std::error_condition open(std::string_view filename, bool writeable = false, chd_file *parent = nullptr, const open_parent_func &open_parent = nullptr);
Expand Down
Loading

0 comments on commit 5731492

Please sign in to comment.