How to add a new dataset?

[JiangLei1012]

That 's a great work!
If I want to train and test on a new dataset, how do I proceed? Do I need to write my own initial configuration file?

[happyxin5]

Have you ever encountered this problem?
(tddpm) D:\Study\code\tab-ddpm-main>wget "https://www.dropbox.com/s/rpckvcs3vx7j605/data.tar?dl=0" -O data.tar
--2024-09-25 21:30:54-- https://www.dropbox.com/s/rpckvcs3vx7j605/data.tar?dl=0
Resolving www.dropbox.com (www.dropbox.com)... 103.228.130.61, 2a03:2880:f10d:83:face:b00c:0:25de
Connecting to www.dropbox.com (www.dropbox.com)|103.228.130.61|:443%7C103.228.130.61%7C:443/)... failed: Unknown error.
Connecting to www.dropbox.com (www.dropbox.com)|2a03:2880:f10d:83:face:b00c:0:25de|:443%7C2a03:2880:f10d:83:face:b00c:0:25de%7C:443/)... failed: Bad file descriptor.

[celsofranssa]

Hi @JiangLei1012,
Could you let me know if you managed to find a way to include a new dataset?

[VinuraD]

Hi, if you still need to know how to include a new dataset for Tabddpm,

1. create a directory for the dataset under the data/ dir.
2. Add data as .npy files, numerical and categorical columns in separate files, and info.json.
3. create a directory for the dataset under the exp/ directory.
4. Create config.toml file (I just edited a toml file from an existing dataset, e.g.- churn2)
5. Also, need to have an optimized catboost/mlp for the dataset inside tuned_models/* dir. As a starter, can use an existing catboost model file (e.g.- from churn2) with a few edits.

The tune/train/eval scripts can be run afterward.

[VinuraD]

Hi, if you still need to know how to include a new dataset for Tabddpm,

1. create a directory for the dataset under the data/ dir.
2. Add data as .npy files, numerical and categorical columns in separate files, and info.json.
3. create a directory for the dataset under the exp/ directory.
4. Create config.toml file (I just edited a toml file from an existing dataset, e.g.- churn2)
5. Also, need to have an optimized catboost/mlp for the dataset inside tuned_models/* dir. As a starter, can use an existing catboost model file (e.g.- from churn2) with a few edits.

The tune/train/eval scripts can be run afterward.

Also make sure that cat columns are of string type.