Skip to content
This repository has been archived by the owner on Jan 25, 2021. It is now read-only.

Challenge submission - Enhance log collection #5

Open
wants to merge 5 commits into
base: enhance-log-analysis
Choose a base branch
from

Conversation

Tommyf
Copy link

@Tommyf Tommyf commented Oct 8, 2019

This new feature brings the following new functionality for integration
with your RoboMaker Simulation jobs. Unfortunately there is no
boto3 support for DeepRacer, so we cannot directly interact with the DeepRacer console.

  • Gets list of RoboMaker simulation jobs
  • Lets user select specific job
  • Gets extended job data, including hyperparameters
  • Job info and hyperparameters are then automatically used elsewhere
    in notebook.

To use:

  • Check "List Robomaker Jobs Directly From AWS"
  • Select your job
  • Load the job summary
  • Continue using the notebook as normal

This change has been submitted as part of the AWS DeepRacer Community Log Analysis Challenge. To join the community go to http://join.deepracing.io

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Racing Name: Tom17

Tom Francis added 5 commits October 8, 2019 15:14
This new feature brings the following new functionality for integration
with your RoboMaker Simulation jobs. Unfortunately there is no
boto3 support for DeepRacer
* Gets list of RoboMaker simulation jobs
* Lets user select specific job
* Gets extended job data, including hyperparameters
* Job info and hyperparameters are then automatically used elsewhere
 in notebook.

To use:
* Check  "List Robomaker Jobs Directly From AWS"
* Select your job
* Load the job summary
* Continue using the notebook as normal
This was hard coded, but the id is different per account.
TODO: Update so it only uses a 'deepracer-simapp' id
Change collection method for hyperparameters.
Instead of finding them in the logs, get the file location from
the job properties and load the json from S3

Use the same method to also get and display the Action Space
@Tommyf
Copy link
Author

Tommyf commented Oct 23, 2019

Some more information about this pull request:

This pull request simplifies log file management when training using the DeepRacer console on AWS. It will:

  • Let you select which simulation job you wish to analyse. Job summary is included in the dropdown for easier selection.
  • Automatically collect extra information from AWS RoboMaker about your job, including episodes per iteration, hyper-params and action space
  • Configure the number of episodes for use further on in the notebook.

This is a nicer alternative to having to enter in the simulation ID, number of episodes per iteration, and not having the hyperparameter and action space info within the notebook.

Here are some screenshots showin the updated functionality.

Selecting to use 'auto parameters'

image

Dropdown box to select simulation

image

Simulation summaries in dropdown

image

Displaying of Simulation Job Details

image
image

Auto selecting and downloading the correct log file from CWL if AutoParams are enabled

image

Automatically loading the correct track from the Simulation Job if AutoParams are enabled

image

Automatically setting EPISODES_PER_ITERATION if AutoParams are enabled

image

TODO:

Currently this only works for training on the AWS DeepRacer console and not local training. If people like this ocncept of automatic parameter gathering, then we could also implement it for the local training setups. Give feedback if you want this!

Also TODO:

I have used widgets in this version of the notebook. There are some annoyances when using widgets.

  • You need to run the cell to display the widget
  • After interacting with a widget, the sel is selected again (Rather than the following cell being selected). You need to make sure to click on the following cell to select it before clicking on 'run' again.
  • If you know a better way to use widgets in Jupyter Notebooks, then please contribute and improve :)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant