custom dataset

[maherr13]

Are there procedures/steps/scripts for training the model on a custom dataset?

[ShenhanQian]

You can follow the procedures/scripts of Speech2Gesture .

[maherr13]

Thanks that was helpful, may I ask How to calculate scaling constants for my dataset like in [speakers_stat.py? @ShenhanQian

[ShenhanQian]

The scale_factor is used to resize the keypoints of subjects so that they have equal widths of shoulders.

Therefore, you can first compute the mean shoulder width of your own subject, and obtain its scale_factor by its ratio to the mean shoulder width of either subject in our released data.

[ShenhanQian]

I have just uploaded reference scripts for custom data processing.

You can find the mean shoulder width of Oliver here.

SpeechDrivesTemplates/data_preprocess/2_3_rescale_shoulder_width.py

Line 68 in 1d8182b

oliver_shoulder_dist = 331.0850066245443

Alternatively, you could simply rescale the keypoints with data_preprocess/2_3_rescale_shoulder_width.py so that the new speaker will have the same shoulder width as Oliver, then you can set the new speaker's scale_factor to Oliver's.

[maherr13]

I would like to thank you for the scripts, it's really helpful.

I would like to mention two errors in 2_1_gen_kpts.py :

1. at line 47 opWrapper.emplaceAndPop takes op.VectorDatum object type so I corrected it to opWrapper.emplaceAndPop(op.VectorDatum([datum]))

2. at line 82 I think I missed the bracket location I corrected it to len(sys.argv) == 2

[maherr13]

I tried the pipeline and it worked perfectly fine except that I got the mean and std values for the key points equal to zeros from the last two scripts.

any thoughts why could that happen?

[Garfield-Finch]

Thank you for your support!
Could you please share your ``clips.csv'' (or a segment of it) to help us debug?

[maherr13]

sorry for late response,

the data I ran through the pipeline was 6 videos each 10-sec max for the same person.