[RFC] Refactor promo sets, add missing sets as listed on Scryfall

[luziferius]

During the conversation in issue #6180 @JayDi85 indicated that refactoring the old sets is possible, because it does not totally destroy already existing deck files. #6180 (comment)

So I started to go through all sets listed on Scryfall and add them to XMage, from old to new.
The intention is to fix all old sets and promotional releases.

• XMage should be able to download images for all official MTG cards from Scryfall.
• All printings of supported cards should be available within XMage.
• Edit: Fix this warning issued by the CI system: Warning: total missing sets: 256, with missing cards: 11596

Progress report

TODO

These tasks contain a bit more work than simply adding a new class describing the set.

• • Prerelease Events contains cards until 2008, newer Prerelease promos are placed in their respective’s set’s promo cards set. Update https://github.com/magefree/mage/blob/master/Mage.Sets/src/mage/sets/PrereleaseEvents.java to reflect that. Also update the Set release date and code.
    • • Updated the set code.
• • Split Arena League into the individual sets as listed on Scryfall.
• • Split Judge Promo into the individual sets as listed on Scryfall.
• • Split Friday Night Magic into the individual sets as listed on Scryfall.
• • Split Magic Player Rewards into the individual sets as listed on Scryfall.
• • Can the booster related settings be removed? The set has no boosters.
    

    mage/Mage.Sets/src/mage/sets/Starter2000.java

    Lines 25 to 29 in 68e5932

    	this.numBoosterLands = 1;
    	this.numBoosterCommon = 10;
    	this.numBoosterUncommon = 3;
    	this.numBoosterRare = 1;
    	this.ratioBoosterMythic = 0;

• • Verify Correctness of the MtgJson translation table located here:
    https://github.com/magefree/mage/blob/master/Mage.Verify/src/main/java/mage/verify/MtgJson.java
    It seems that newer JSON documents now have fully UPPER case set abbreviations.
• • Make sure there are no Unicode characters in collector number strings and card names.
• • Update the sets list at https://github.com/magefree/mage/blob/master/Utils/mtg-sets-data.txt?
• • Update all image downloader plugins to work with the new structure (Not necessary, as indicated in a comment below) JayDi85: scryfall updated, other plugins are outdated and must be reworked
• • Provide a map to convert decks to the new set structure JayDi85: no needs, xmage auto-fixes unknown set codes
    • A static HashMap that provides a lookup-table to update the user decks should be sufficient.
• • Make sure that the Collector numbers of the Junior Series Europe (PJSE) set are not interpreted as numbers in scientific notation. These look like "1E08", which may get converted to large integers by accident.
• • Build the "Supported Sets" set list in the scryfall downloader by iterating over all classes. Extract the required code from the classes instead of hard-coding it in a giant list.
• • Add all ~7 variants of Tamiyo's Journal Both in SOI and PSOI.
• • Add MTG Arena printings?
    If Yes: https://scryfall.com/sets/ana https://scryfall.com/sets/pana https://scryfall.com/sets/ha1
• • Split the WPN and Gateway Promos?

Added sets

All Sets are added up to the recently announced SLU (Secret Lair: Ultimate Edition).

Exceptions

The Celebration Cards (PCEL) set is basically silver-bordered, except for two cards, which have actually functional and implementable rules text. These cards are illegal in all formats.

These non-English sets are not added. Adding these will allow players to force the display of non-English card images. Therefore I opted to omit them for now.

• Foreign Black Border (FBB)
• Fourth Edition Foreign Black Border (4BB)
• Rinascimento (RIN)
• Renaissance (REN)
• Redemption Program (PRED)
• Hobby Japan Promos (PHJ)
• Japan Junior Tournament (PJJT)
• Magic Premiere Shop 2005 (PMPS) and following
• Salvat 2005 (PSAL)
• Salvat 2011(PS11)

Notes

Please review the added classes, as they are mostly automatically generated (see below). I’ve checked them for plausibility, but I can’t guarantee that the automated process works in all cases. There are some bits I’m modifying in each class, like setting the Expansion type, booster content, and fixing all missing/wrongly generated references IntelliJ finds.

Adding all cards of a set by hand is tiresome and error prone (I tired out after 10 or so cards), so I’ve written a small script that can automatically generate Java class source files for XMage sets. It uses the card database code I’ve written for my project https://github.com/luziferius/MTGDeckConverter to build the class code. The script code used is included below for reference.
(Side note: Although the code I use is licensed under GPLv3+, the output of a GPL-licensed program, even if it is source code, is not covered by the GPL, so it is OK to include the generated code files in XMage.)

# Copyright (C) 2020 Thomas Hess <[email protected]>

# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.

# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.

# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.

import argparse
import datetime
import pathlib
import re
import string
import textwrap
from typing import NamedTuple, Iterable, Union, Tuple

import MTGDeckConverter.card_db.db


class Card(NamedTuple):
    name: str
    collector_number: str
    rarity: str

    def format_as_source_line(self, has_duplicates: bool) -> str:
        """Formats this card as a java source line that adds this card to the generated set."""
        package_name = "basiclands" \
            if self.name in basic_lands \
            else self.name[0].lower()
        class_name = name_to_camel_case(self.name)
        return card_line_template.substitute(
            card_name=self.name,
            collector_number=self.collector_number,
            rarity=self.rarity,
            card_package=package_name,
            card_class_name=class_name,
            # Default to assume regular basic lands with interchangeable artworks.
            # Or drop the token for non-"basic land" cards
            artwork_specifier=", NON_FULL_USE_VARIOUS" if has_duplicates else ""
        )


class Namespace(NamedTuple):
    """Mocks parsed arguments, used for type checking / hinting purposes."""
    database: pathlib.Path
    set_abbreviation: str
    output_path: pathlib.Path


def parse_args() -> Namespace:
    """Generate an argument parser and parse the command line arguments."""
    argument_parser = argparse.ArgumentParser(description="Generate XMage Sets")
    argument_parser.add_argument(
        "--database",
        action="store", type=pathlib.Path, default=pathlib.Path("./mtg.sqlite3"),
        help="Path to database storing the card data."
    )
    argument_parser.add_argument(
        "-o", "--output-path",
        action="store", type=pathlib.Path,
        help="Path to XMage Sets source Package. If given, a .java source will will be written there."
    )
    argument_parser.add_argument(
        "set_abbreviation", action="store",
        help="XMage java class to generate."
    )
    return argument_parser.parse_args()


basic_lands = ("Plains", "Island", "Swamp", "Mountain", "Forest")
alpha_re = re.compile('[^a-zA-Z]')
java_template = string.Template("""\
package mage.sets;

import mage.cards.ExpansionSet;
import mage.constants.Rarity;
import mage.constants.SetType;

/**
 * https://scryfall.com/sets/${set_abbr_lower}
 */
public class ${set_class_name} extends ExpansionSet {

    private static final ${set_class_name} instance = new ${set_class_name}();

    public static ${set_class_name} getInstance() {
        return instance;
    }

    private ${set_class_name}() {
        super("${set_name}", "${set_abbr_upper}", ExpansionSet.buildDate(${set_release_date}), SetType.);
        // this.hasBoosters = false;
        this.hasBasicLands = ${set_has_basics};

${card_list}
     }
}
""")
card_line_template = string.Template(
    'cards.add(new SetCardInfo('
    '"${card_name}", ${collector_number}, Rarity.${rarity}, mage.cards.${card_package}.${card_class_name}.class${artwork_specifier})'
    ');'
)


def validate_input_set(args: Namespace, db: MTGDeckConverter.card_db.db.CardDatabase):
    """Raise an Exception, if the given set is not known. Prevents generation of broken Java source files."""
    if not db.is_set_abbreviation_known(args.set_abbreviation.lower()):
        raise ValueError("Given set abbreviation is unknown!")


def word_to_title_case(word: str) -> str:
    if len(word) > 1:
        return word.title()
    else:
        return word


def name_to_camel_case(set_name: str) -> str:
    """Converts a string containing a name of something (Card or Set name) to a valid Java CamelCase identifier"""
    filtered_name = alpha_re.sub(" ", set_name)
    words = filtered_name.split(" ")
    
    camel_case = "".join(word_to_title_case(word) for word in words)
    return camel_case


def format_all_card_lines(args: Namespace, db: MTGDeckConverter.card_db.db.CardDatabase) -> str:
    """
    Queries the database for all files in the given set, and formats the card-adding source lines.
    Sorts the cards by the English name in ascending order.
    """
    cards_query: Iterable[Tuple[str, Union[str, int], str, int]] = db.db.execute(
        "SELECT P1.English_Name, P1.Collector_Number, P1.Rarity, Count(*) > 1 AS has_duplicates "
        "FROM Printings_View AS P1 "
        "INNER JOIN Printings_View AS P2 USING (English_Name, Abbreviation) "
        "WHERE Abbreviation = ? "
        "GROUP BY P1.English_Name, P1.Collector_Number, P1.Rarity "
        "ORDER BY P1.English_Name ASC, P1.Collector_Number ASC",
        (args.set_abbreviation.lower(),)
    )
    cards = (
        (Card(name, format_collector_number(collector_number), format_rarity(rarity)), bool(has_duplicates))
        for name, collector_number, rarity, has_duplicates in cards_query)

    card_list = "\n".join(card.format_as_source_line(has_duplicates) for (card, has_duplicates) in cards)
    return textwrap.indent(card_list, " "*8)


def format_rarity(rarity: str) -> str:
    """Converts the internal rarity name to an XMage Rarity enum member name."""
    return rarity.upper()


def format_collector_number(collector_number: Union[str, int]) -> str:
    """Collectors number may be an int or string. Convert both types to str for formatting purposes."""
    if isinstance(collector_number, int):
        return str(collector_number)
    else:
        # Put quotation marks around the collector number. Converts this into a Java string literal.
        return f'"{collector_number}"'


def format_release_date(date_str: str) -> str:
    """Format the release date in a format suitable to be put into the ExpansionSet.buildDate() constructor."""
    date = datetime.date.fromisoformat(date_str)
    # Format manually, because date.strftime() will create zero-padded numbers, which won’t work in Java code,
    # as they will be interpreted as octal literals.
    date_str = f"{date.year}, {date.month}, {date.day}"
    return date_str


def set_has_basics(args: Namespace, db: MTGDeckConverter.card_db.db.CardDatabase) -> str:
    """Returns a Java boolean literal true/false indicating if the set has basic lands."""
    # Use that the tuple string representation is a valid SQL statement for the IN operator.
    basic_lands_set = str(tuple(basic_lands))
    has_basics = bool(db.db.execute(
        "SELECT EXISTS( SELECT * "
        "FROM Printings_View "
        "WHERE Abbreviation = ? "
        f"AND English_Name IN {basic_lands_set} )",
        (args.set_abbreviation.lower(),)
    ).fetchone()[0])
    return str(has_basics).lower()


def main():
    args = parse_args()
    db = MTGDeckConverter.card_db.db.CardDatabase(args.database)
    validate_input_set(args, db)
    set_abbr_upper = args.set_abbreviation.upper()
    set_abbr_lower = args.set_abbreviation.lower()
    set_name, set_release_date = db.db.execute(
        "SELECT English_Name, Release_date "
        "FROM Card_Set "
        "WHERE Abbreviation = ?",
        (set_abbr_lower,)).fetchone()  # type: str, str
    set_release_date = format_release_date(set_release_date)
    class_name = name_to_camel_case(set_name)
    card_list = format_all_card_lines(args, db)
    has_basics = set_has_basics(args, db)
    java_class = java_template.substitute(
        set_abbr_lower=set_abbr_lower,
        set_abbr_upper=set_abbr_upper,
        set_release_date=set_release_date,
        set_name=set_name,
        set_class_name=class_name,
        card_list=card_list,
        set_has_basics=has_basics
    )
    print(java_class)
    if args.output_path is not None and args.output_path.is_dir():
        file_path = args.output_path / f"{class_name}.java"
        if file_path.exists():
            raise ValueError("Java file already exists!")
        else:
            file_path.write_text(java_class, encoding="utf-8")


if __name__ == '__main__':
    main()

The database has to be seeded once, using a snippet like this:

import MTGDeckConverter.card_db.db 
from pathlib import Path 
db = MTGDeckConverter.card_db.db.CardDatabase(Path("./mtg.sqlite3")) 
db.populate_database() 
db.db.commit()

[JayDi85]

1. Is it possible to add cards that do not appear in boosters to a set with boosters? (Mostly relevant for 8ED/9ED starter packs.)

If set have non booster cards then you must setup maxCardNumberInBooster (all cards with number after that will be ignored in booster generation). See example ThroneOfEldraine. If you can't split it by number then make two different set.

2. Is it sufficient to add a .java source file to Mage.Sets/src/mage/sets/ to add basic support for a new set?

Yes, you must create one file per set. Java files loaded automaticity by client/server on startup and generates full sets/cards database. Database is basic source info about sets and cards. Moreover each set have different settings for boosters and other special code for booster generating (like special lands, partners, etc).

3. Should I add gold-bordered sets? (E.g. https://scryfall.com/sets/ptc) Those aren’t legal in sanctioned tournaments, but are reprints of otherwise legal cards.

If scryfall marks it as standlone set then yes -- you can add it. All sets have SetType settings. That's uses for legality check in main formats. If you setup it with SetType.PROMOTIONAL then only freeform game modes will be available for it.

4. Should NON_FULL_USE_VARIOUS to any non-full-art basic land, even if the set only has one of a kind? This is used to generate the list of available basic lands for the "add random basic lands to a deck" functionality built into the deck builder, right?

Nope. Use that settings if set contains cards with SAME name (like multiple lands or normal and bonus versions of the card). You will get error in tests (MageVerify) if something wrong. MageVerify checks many settings for cards in sets. I strongly recommends to use it (IntelliJ can run that test from IDE -- just click on mage-verify project and choose run tests).

5. I’m getting NPEs in the deck editor when running the client within IntelliJ and the card display pop-up window does not work. And the card image fetcher indefinitely han

If you run by IDE then check java version in project settings. It must be 1.8 (java 8) only. Newer version do not supports.

[theelk801]

I'm pretty sure that as long as the expansion has the right SetType then it won't affect legality but its cards will still be legal in the correct places based on name.

[JayDi85]

Ok. The cards in question have collectors number S1 to S7. So probably it has to remain split as it is.

If scryfall split it then xmage must do same (I known there are little amount of cards in that sets, but that's easy to support, e.g. for auto-generators, checks, images downloader, etc). When thats sets will be completed then I can modify sets search dialog to add filter (shows all or only basic sets).

What about tournament legal promotional sets?

As theelk801 above: xmage collects card names for legality checks. If any legal set contains card then you can use same card from any other set.

For Example, Battlebond only has 1 artwork per basic land and still set this flag. That’s why I asked.

That's not critical. If one card in set then you can remove NON_FULL_USE_VARIOUS (e.g. you can remove it from battlebond's land). BTW: there are possible another settings like full art cards or special BFZ frame (that's info for M15 render mode).

I’ve read about it and configured my IDE accordingly when I set up the environment.

Well, try to clean up:

• delete all cards.h2.* files from all project folder (client and server);
• delete images folder Mage.Client\plugins\images;
• if you want then reset all xmage settings (just delete all settings with that instructions)

That's error is system related. Maybe something wrong with java or paths.

[theelk801]

Battlebond has the tag because the automated script I used to generate the file always adds it to basics.

[JayDi85]

ERROR 2020-01-20 14:21:23,554 java.lang.IllegalArgumentException: Set code PTC already exists.

You can remove old cumulative PTC set (for pre-release events and sets) from xmage and replace it by new multiple sets.

[JayDi85]

You can find error logs by github, without local run:

[luziferius]

I’ve seen the CI links, but normally I run them locally, because it’s faster.

But now something else:
I’m currently splitting the Arena League promos and a search for ARENA led me to this:

mage/Mage.Verify/src/main/java/mage/verify/MtgJson.java

Lines 28 to 50 in 5000020

	static {
	mtgJsonToXMageCodes.put("pWCQ", "WMCQ");
	mtgJsonToXMageCodes.put("pSUS", "SUS");
	mtgJsonToXMageCodes.put("pPRE", "PTC");
	mtgJsonToXMageCodes.put("pMPR", "MPRP");
	mtgJsonToXMageCodes.put("pMEI", "MBP");
	mtgJsonToXMageCodes.put("pGTW", "GRC"); // pGTW - Gateway = GRC (WPN + Gateway in one inner set)
	mtgJsonToXMageCodes.put("pWPN", "GRC"); // pWPN - Wizards Play Network = GRC (WPN + Gateway in one inner set)
	mtgJsonToXMageCodes.put("pGRU", "GUR");
	mtgJsonToXMageCodes.put("pGPX", "GPX");
	mtgJsonToXMageCodes.put("pFNM", "FNMP");
	mtgJsonToXMageCodes.put("pELP", "EURO");
	mtgJsonToXMageCodes.put("pARL", "ARENA");
	mtgJsonToXMageCodes.put("pALP", "APAC");
	mtgJsonToXMageCodes.put("PO2", "P02");
	mtgJsonToXMageCodes.put("DD3_JVC", "DD3JVC");
	mtgJsonToXMageCodes.put("DD3_GVL", "DDD");
	mtgJsonToXMageCodes.put("DD3_EVG", "DD3EVG");
	mtgJsonToXMageCodes.put("DD3_DVD", "DDC");
	mtgJsonToXMageCodes.put("NMS", "NEM");
	mtgJsonToXMageCodes.put("MPS_AKH", "MPS-AKH");
	mtgJsonToXMageCodes.put("FRF_UGIN", "UGIN");
	mtgJsonToXMageCodes.put("pCMP", "CP");

The map translates pARL to ARENA. Now that ARENA is removed, I wanted to verify how MtgJson handles the other sets (PAL99-PAL06). According to that block, the initial p is lower case.
Then I downloaded the JSON document from https://mtgjson.com/json/AllSets.json.zip and loaded it into Python for inspection. I noticed that all sets are fully UPPER case, thus it seems that the translation is partly outdated.

I’ll add the verification of that code to the TODO list above. Maybe most of those items can be removed later.

>>> import json, pprint, pathlib
>>> mtg_data=json.loads(pathlib.Path("./AllPrintings.json").read_text())

>>> for set_ in mtg_data.keys():
...     if set_.startswith("p"): 
...         print(set_)

>>> pprint.PrettyPrinter(compact=True).pprint(list(mtg_data.keys()))                                                                                                                                                                               
['10E', '2ED', '3ED', '4BB', '4ED', '5DN', '5ED', '6ED', '7ED', '8ED', '9ED',
 'A25', 'AER', 'AKH', 'ALA', 'ALL', 'AMH1', 'ANA', 'APC', 'ARB', 'ARC', 'ARN',
 'ATH', 'ATQ', 'AVR', 'BBD', 'BFZ', 'BNG', 'BOK', 'BRB', 'BTD', 'C13', 'C14',
 'C15', 'C16', 'C17', 'C18', 'C19', 'CED', 'CEI', 'CHK', 'CHR', 'CM1', 'CM2',
 'CMA', 'CMB1', 'CMD', 'CN2', 'CNS', 'CON', 'CP1', 'CP2', 'CP3', 'CSP', 'CST',
 'DD1', 'DD2', 'DDC', 'DDD', 'DDE', 'DDF', 'DDG', 'DDH', 'DDI', 'DDJ', 'DDK',
 'DDL', 'DDM', 'DDN', 'DDO', 'DDP', 'DDQ', 'DDR', 'DDS', 'DDT', 'DDU', 'DGM',
 'DIS', 'DKA', 'DKM', 'DOM', 'DPA', 'DRB', 'DRK', 'DST', 'DTK', 'DVD', 'E01',
 'E02', 'ELD', 'EMA', 'EMN', 'EVE', 'EVG', 'EXO', 'EXP', 'F01', 'F02', 'F03',
 'F04', 'F05', 'F06', 'F07', 'F08', 'F09', 'F10', 'F11', 'F12', 'F13', 'F14',
 'F15', 'F16', 'F17', 'F18', 'FBB', 'FEM', 'FNM', 'FRF', 'FUT', 'G00', 'G01',
 'G02', 'G03', 'G04', 'G05', 'G06', 'G07', 'G08', 'G09', 'G10', 'G11', 'G17',
 'G18', 'G99', 'GK1', 'GK2', 'GN2', 'GNT', 'GPT', 'GRN', 'GS1', 'GTC', 'GVL',
 'H09', 'H17', 'HA1', 'HHO', 'HML', 'HOP', 'HOU', 'HTR', 'HTR17', 'HTR18',
 'ICE', 'IMA', 'INV', 'ISD', 'ITP', 'J12', 'J13', 'J14', 'J15', 'J16', 'J17',
 'J18', 'J19', 'J20', 'JGP', 'JOU', 'JUD', 'JVC', 'KLD', 'KTK', 'L12', 'L13',
 'L14', 'L15', 'L16', 'L17', 'LEA', 'LEB', 'LEG', 'LGN', 'LRW', 'M10', 'M11',
 'M12', 'M13', 'M14', 'M15', 'M19', 'M20', 'MB1', 'MBS', 'MD1', 'ME1', 'ME2',
 'ME3', 'ME4', 'MED', 'MGB', 'MH1', 'MIR', 'MM2', 'MM3', 'MMA', 'MMQ', 'MOR',
 'MP2', 'MPR', 'MPS', 'MRD', 'NEM', 'NPH', 'OARC', 'OC13', 'OC14', 'OC15',
 'OC16', 'OC17', 'OC18', 'OC19', 'OCM1', 'OCMD', 'ODY', 'OE01', 'OGW', 'OHOP',
 'OLGC', 'ONS', 'OPC2', 'OPCA', 'ORI', 'OVNT', 'P02', 'P03', 'P04', 'P05',
 'P06', 'P07', 'P08', 'P09', 'P10', 'P10E', 'P11', 'P15A', 'P2HG', 'PAER',
 'PAKH', 'PAL00', 'PAL01', 'PAL02', 'PAL03', 'PAL04', 'PAL05', 'PAL06', 'PAL99',
 'PALP', 'PANA', 'PARC', 'PARL', 'PAVR', 'PBBD', 'PBFZ', 'PBNG', 'PBOK', 'PC2',
 'PCA', 'PCEL', 'PCMD', 'PCMP', 'PCY', 'PD2', 'PD3', 'PDGM', 'PDKA', 'PDOM',
 'PDP10', 'PDP11', 'PDP12', 'PDP13', 'PDP14', 'PDRC', 'PDTK', 'PDTP', 'PELD',
 'PELP', 'PEMN', 'PF19', 'PF20', 'PFRF', 'PG07', 'PG08', 'PGPX', 'PGRN', 'PGRU',
 'PGTC', 'PGTW', 'PHEL', 'PHJ', 'PHOP', 'PHOU', 'PHPR', 'PHUK', 'PI13', 'PI14',
 'PIDW', 'PISD', 'PJAS', 'PJJT', 'PJOU', 'PJSE', 'PKLD', 'PKTK', 'PLC', 'PLGM',
 'PLNY', 'PLPA', 'PLS', 'PM10', 'PM11', 'PM12', 'PM13', 'PM14', 'PM15', 'PM19',
 'PM20', 'PMBS', 'PMEI', 'PMH1', 'PMOA', 'PMPS', 'PMPS06', 'PMPS07', 'PMPS08',
 'PMPS09', 'PMPS10', 'PMPS11', 'PNAT', 'PNPH', 'POGW', 'POR', 'PORI', 'PPC1',
 'PPOD', 'PPP1', 'PPRE', 'PPRO', 'PR2', 'PRED', 'PREL', 'PRES', 'PRIX', 'PRM',
 'PRNA', 'PROE', 'PRTR', 'PRW2', 'PRWK', 'PS11', 'PS14', 'PS15', 'PS16', 'PS17',
 'PS18', 'PS19', 'PSAL', 'PSDC', 'PSLD', 'PSOI', 'PSOM', 'PSS1', 'PSS2', 'PSS3',
 'PSUM', 'PSUS', 'PTC', 'PTG', 'PTHS', 'PTK', 'PTKDF', 'PUMA', 'PURL', 'PUST',
 'PVAN', 'PWAR', 'PWCQ', 'PWOR', 'PWOS', 'PWP09', 'PWP10', 'PWP11', 'PWP12',
 'PWPN', 'PWWK', 'PXLN', 'PXTC', 'PZ1', 'PZ2', 'PZEN', 'RAV', 'REN', 'RIN',
 'RIX', 'RNA', 'ROE', 'RQS', 'RTR', 'S00', 'S99', 'SCG', 'SHM', 'SLD', 'SOI',
 'SOK', 'SOM', 'SS1', 'SS2', 'STH', 'SUM', 'TBTH', 'TD0', 'TD2', 'TDAG', 'TFTH',
 'THB', 'THP1', 'THP2', 'THP3', 'THS', 'TMP', 'TOR', 'TPR', 'TSB', 'TSP', 'UDS',
 'UGIN', 'UGL', 'ULG', 'UMA', 'UNH', 'USG', 'UST', 'V09', 'V10', 'V11', 'V12',
 'V13', 'V14', 'V15', 'V16', 'V17', 'VIS', 'VMA', 'W16', 'W17', 'WAR', 'WC00',
 'WC01', 'WC02', 'WC03', 'WC04', 'WC97', 'WC98', 'WC99', 'WTH', 'WWK', 'XLN',
 'ZEN']

[JayDi85]

You need to move new graphic info object to constructor:

[JayDi85]

Nope, do not add non english sets.

[JayDi85]

Found some TODOs in start topic:

Can the booster related settings be removed? The set has no boosters

Yes, if set don't have boosters then you don't need that settings.

Update the sets list at https://github.com/magefree/mage/blob/master/Utils/mtg-sets-data.txt?

If you can then yes. But it's not necessary. That's file uses for card code template generating and other perl scripts.

Update all image downloader plugins to work with the new structure

1. What's changes are you talking about?

Provide a map to convert decks to the new set structure
A static HashMap that provides a lookup-table to update the user decks should be sufficient.

2. Are you talking about auto-convert deck files in xmage release folder, user's computer or something else?

[luziferius]

I thought of a backwards compatibility issue with this:

This PR removes and alters some sets. This will be incompatible with previously downloaded images. The removed sets won’t harm much, as their image archive will just linger on disk forever without being used.
But it has to be checked before merging that the updated sets will work correctly with the old image data or if they start to show wrong images or crash the client at worst.

[JayDi85]

Yes, I know. It will be in what's new docs (about images clean up and/or re-download recommendations).

It's ok if some old promo sets will be stores in zips/folders forever (until images folder clean up).

[JayDi85]

Now it finally done. XMage got ~200 new sets and ~6000 new reprints with that PR. Thanks for contribute.