Add option to verify ISOs against redump.org?

[hrydgard]

redump.org provides metadata for most of the PSP discs, but of course don't host any downloads.

I didn't actually know that redump.org had a PSP database because it's not visible from the main website's list of platforms, but it's really there:

http://redump.org/discs/system/psp

So if we can get our hands on the actual database, or find an easy way to query it, it should be easy to add a way to check that an ISO has not been modified from its original form.

[ghost]

I have all the games in ISO, so if I would be interested in that new option.

[NABN00B]

What purpose would this serve? Display a warning on top of the screen if the ISO failed the verification or refuse to launch it? Either way, I doubt there's a complete database anywhere so this would be very likely to cause some confusion in the future.

[LunaMoo]

It's more about stopping people from blaming emulation while abusing broken pirated copy or otherwise corrupted disc and wasting people's time, not doing anything at launch. We could add an requirement of validating users copy before posting issues here and also display at reporting site which copies are good/broken as PPSSPP is collecting hashes for a while already.

In theory ISO should not exactly be the most popular container, I for example use PBP for pretty much everything simply for it's best compression, psstore games as well as pre-patched ones or just an umd backups. CSO are also extremely popular and some of them can modify/remove content for better compression which is awful, but used historically by many desperates with small memsticks.
Thankfully pirates share ISO's exclusively and most pirating users don't know we support compressed formats from the PSP or might just be to lazy since it's not that important on devices PPSSPP runs on, so it might still prove useful in practice.

[unknownbrackets]

Well, some PBPs simply just have the UMD image (aka ISO) inside them, as I recall. Not certain how we calculate CRCs today, but we may already do those correctly. Could you compare the CRC of one of your games (as recorded on report.ppsspp.org when you report compatibility) with the ISO's expected CRC?

We already have a database of each game id and version based on reports. So we could just import verified CRCs into this database for cross referencing.

On the downloads page, we can get an XML export of that database, but it lacks IDs. Better would be to scrape it, probably. Not sure we would expect a lot of updates in the future anyway.

Of note, we do already have reported CRCs. For more popular games, it's already pretty obvious / easily detectable which CRC is right:
https://report.ppsspp.org/game/ULES01044_1.00

In fact, we do that already, although the heuristics aren't perfect:
https://github.com/unknownbrackets/ppsspp-report/blob/6cead3ca4d17dd4702cc2b1e9f19b22c9fbc95f7/lib/compat/suggest.js#L170-L171

ppsspp/UI/ReportScreen.cpp

Lines 434 to 435 in c77af18

	} else if (item == "VerifyDisc") {
	suggestion = rp->T("SuggestionVerifyDisc", "Check your ISO is a good copy of your disc");

Of note: because of PSN releases, I'm pretty sure there are games with the same ID, but multiple perfectly valid and correct CRCs. I'm not sure how often this happens, though.

-[Unknown]

[Abbanon]

It looks like they offer mass .txt Datfile downloads, displaying all of their information by platform and game title (size, crc, md5, sha1)
http://redump.org/downloads/
Maybe this could be used?

[Deterous]

Attached is an export of the redump PSP DB with the relevant information to help improve PPSPP.
As of today(!) the redump database now has the complete set of metadata for all PAL PSP titles, as well as all USA PSP titles and a large portion of Japan/Asia titles. Note that there may remain further revisions for PAL/USA, and if anyone wants to help please look at wiki.redump.org for the missing dumps!
In total, this csv contains metadata of 2809 UMDs, including some preproduction and demo UMDs.

redump-psp.csv

Thanks for all the work on PPSSPP!

Final note: redump.org does not contain ISOs, this information is purely to help verify ISOs that you dump yourself :)

[hrydgard]

That is great as mentioned, and now that it has game serials / IDs, we can easily do lookups into it to verify ISOs.

I plan to add support for ISO verification in the near future. I think I'll just bundle the csv as a file in /assets.

I can also think of a bunch of different uses for this, such as specifying compat.ini options by game name instead of by ID, showing the international name on Japanese ISOs in the game listing, etc.

[Deterous]

The "Disc Serials" column is the printed serial on the UMD itself.
Unfortunately redump does not have a complete dataset on the internal serial of each UMD
If desired, I may be able to produce one with internal serials to the best of our knowledge (some disc pages have internal serial in the comments)
Although I am not sure whether PPSSPP uses the internal serial?

[hrydgard]

Printed is fine as it's generally gonna match the internal one, if there are one or two misprints or mistakes it won't take away much from the usefulness. We use the ID found in PARAM.SFO on the ISO file for lookups into our compatibility settings file etc.

[Deterous]

Yes, the ID inside PARAM.SFO is the internal serial I am referencing.
The Disc Serial column is a comma-separated list of all printed serials that match that ISO.
You may have to do some sanitizing to get the ID out of it, such as hyphens vs spaces, removing suffixes, etc.

e.g. one entry is "ULUS-10266, ULUS-17008"
and another is "ULUS-10237, ULUS-10237GH"

[hrydgard]

Yeah, that's fine, doesn't look hard.

[hrydgard]

This is now implemented, checks happen on the game info screen only currently, not during game startup. Let me know how it works for you.