Feature/split p2e

[michele-milesi]

Summary

Describe the purpose of the pull request, including:

• Split p2e_dv1 into two separate scripts: p2e_dv1_exploration and p2e_dv1_finetuning
• Split p2e_dv2 into two separate scripts: p2e_dv2_exploration and p2e_dv2_finetuning
• Now the finetuning can start with the same buffer of the exploration or with a completely new one. If the buffer is new, then the user can decide whether to collect the first experiences (until learning_start is reached) with the actor_exploration or the actor_task. After the learning_start, only the actor_task collects experiences.
• Added p2e_dv3 algorithm. It works as the other P2E algorithms, with the difference that this one enables the definition of more than one exploration critic. Each critic learns to predict either the intrinsic or task-related targets. Moreover, for actor learning, each critic has a weight that establishes its importance in the actor loss.

Type of Change

Please select the one relevant option below:

• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist

Please confirm that the following tasks have been completed:

• I have tested my changes locally and they work as expected. (Please describe the tests you performed.)
• I have added unit tests for my changes, or updated existing tests if necessary.
• I have updated the documentation, if applicable.
• I have installed pre-commit and run locally for my code changes.