Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decide on the models/tasks/datasets to benchmark this technique #1

Open
simi2525 opened this issue Jul 29, 2019 · 0 comments
Open

Decide on the models/tasks/datasets to benchmark this technique #1

simi2525 opened this issue Jul 29, 2019 · 0 comments
Assignees

Comments

@simi2525
Copy link
Member

simi2525 commented Jul 29, 2019

We should decide on what to evaluate the effectiveness of std loss regularization.

After a short online search I found a number of benchmarks that seem to be designed for this purpose:

(this can also help us get some more references in the article, since sadly people care about that)
We can use any of these and we could also use more standard ways of testing.

Since we don't have access to much compute power or funding, we should select as few benchmarks as possible that would still produce results as convincing as possible.

Potential models:

  • VGG11
  • VGG16
  • VGG19
  • ResNet101
  • ResNeXt
  • LeNet
  • AlexNet
  • Simple MLP

Potential datasets:

  • CIFAR10
  • CIFAR100
  • ImageNet 1k
  • COCO
  • Sentiment140
  • Open AI Gym games
CIFAR10 CIFAR100 ImageNet 1k COCO Sentiment140 Open AI Gym games
VGG11 result result result result result result
VGG16 result result result result result result
VGG19 result result result result result result
ResNet101 result result result result result result
ResNeXt result result result result result result
LeNet result result result result result result
AlexNet result result result result result result
Simple MLP result result result result result result

*I suggest we use a table like this and put in the link to the folder/file in the repository where we stored the results of the experiment

A list of popular datasets I found:
https://medium.com/towards-artificial-intelligence/the-50-best-public-datasets-for-machine-learning-d80e9f030279

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants