Add more AutoAugment policies #4753

stiepan · 2023-03-29T13:23:12Z

Category:

New feature (non-breaking change which adds functionality)

Description:

This PR adds 3 more (apart from the "v0" image-net policy) policies to auto_augment module (reduced image net, reduced cifar-10, svhn). A new function, simply called auto_augment, is added to the auto_augment module, as a convienience wrapper for applying AA with one of the predefined policies.

The predifined policies as introduced in the AA paper are a bit over-specified: 1. they specify meanigless magnitude bins for augmentations that do not accept magnitudes, 2. they specify some augmentations to be run with 0 probability, 3. they specify some augmentations to be run with magnitude such that the operation is in fact an identity. This PR adds warnings and adjusts the definition of policies to address the points 1. and 2.

Now, all AA/TA and RA modules use both translation_x and translation_y augmentations, so I removed the _get_translation_y helper from auto_aug and moved the _get_translations from RA to common util used across the three modules.

Additonally, the PR fills some gaps in the documentation (i.e. max_translation_abs/rel) and the docs in utilities.

Additional information:

Affected modules and functionalities:

Key points relevant for the review:

Tests:

Checklist

Documentation

DALI team only

Requirements

Implements new requirements
Affects existing requirements
N/A

REQ IDs: N/A

JIRA TASK: DALI-3299

dali/test/python/auto_aug/test_augmentations.py

stiepan · 2023-03-30T11:31:47Z

dali/test/python/auto_aug/test_auto_augment.py

-@params(*tuple(itertools.product((True, False), (0, 1), ('height', 'width', 'both'))))
-def test_translation(use_shape, offset_fraction, extent):
-    # make sure the translation helper processes the args properly
-    # note, it only uses translate_y (as it is in imagenet policy)
-    shape = [300, 400]
-    fill_value = 217
-    params = {}
-    if use_shape:
-        param = offset_fraction
-        param_name = "max_translate_rel"
-    else:
-        param_name = "max_translate_abs"
-    if extent == 'both':
-        param = shape[0] * offset_fraction
-    elif extent == 'height':
-        param = [shape[0] * offset_fraction, 0]
-    elif extent == 'width':
-        param = [0, shape[1] * offset_fraction]
-    else:
-        assert False, f"Unrecognized extent={extent}"
-    params[param_name] = param
-    translate_y = auto_augment._get_translate_y(use_shape=use_shape, **params)
-    policy = Policy(f"Policy_{use_shape}_{offset_fraction}", num_magnitude_bins=21,
-                    sub_policies=[[(translate_y, 1, 20)]])
-
-    @experimental.pipeline_def(enable_conditionals=True, batch_size=3, num_threads=4, device_id=0,
-                               seed=43)
-    def pipeline():
-        encoded_image, _ = fn.readers.file(name="Reader", file_root=images_dir)
-        image = fn.decoders.image(encoded_image, device="mixed")
-        image = fn.resize(image, size=shape)
-        if use_shape:
-            return auto_augment.apply_auto_augment(policy, image, fill_value=fill_value,
-                                                   shape=shape)
-        else:
-            return auto_augment.apply_auto_augment(policy, image, fill_value=fill_value)
-
-    p = pipeline()
-    p.build()
-    output, = p.run()
-    output = [np.array(sample) for sample in output.as_cpu()]
-    for i, sample in enumerate(output):
-        sample = np.array(sample)
-        if offset_fraction == 1 and extent != "width":
-            assert np.all(sample == fill_value), f"sample_idx: {i}"
-        else:
-            background_count = np.sum(sample == fill_value)
-            assert background_count / sample.size < 0.1, \
-                f"sample_idx: {i}, {background_count / sample.size}"
-
-


It's now common utility for AA/TA/RA modules. Tested in test_augmentations.

stiepan · 2023-03-30T11:32:03Z

dali/test/python/auto_aug/test_rand_augment.py

-@params(*tuple(itertools.product((True, False), (0, 1), ('height', 'width', 'both'))))
-def test_translation(use_shape, offset_fraction, extent):
-    # make sure the translation helper processes the args properly
-    # note, it only uses translate_y (as it is in imagenet policy)
-    shape = [300, 400]
-    fill_value = 105
-    params = {}
-    if use_shape:
-        param = offset_fraction
-        param_name = "max_translate_rel"
-    else:
-        param_name = "max_translate_abs"
-    assert extent in ('height', 'width', 'both'), f"{extent}"
-    if extent == 'both':
-        param = [shape[0] * offset_fraction, shape[1] * offset_fraction]
-    elif extent == 'height':
-        param = [shape[0] * offset_fraction, 0]
-    elif extent == 'width':
-        param = [0, shape[1] * offset_fraction]
-    params[param_name] = param
-    translate_x, translate_y = rand_augment._get_translations(use_shape=use_shape, **params)
-    if extent == 'both':
-        augments = [translate_x, translate_y]
-    elif extent == 'height':
-        augments = [translate_y]
-    elif extent == 'width':
-        augments = [translate_x]
-
-    @experimental.pipeline_def(enable_conditionals=True, batch_size=3, num_threads=4, device_id=0,
-                               seed=43)
-    def pipeline():
-        encoded_image, _ = fn.readers.file(name="Reader", file_root=images_dir)
-        image = fn.decoders.image(encoded_image, device="mixed")
-        image = fn.resize(image, size=shape)
-        if use_shape:
-            return rand_augment.apply_rand_augment(augments, image, n=1, m=30,
-                                                   fill_value=fill_value, shape=shape)
-        else:
-            return rand_augment.apply_rand_augment(augments, image, n=1, m=30,
-                                                   fill_value=fill_value)
-
-    p = pipeline()
-    p.build()
-    output, = p.run()
-    output = [np.array(sample) for sample in output.as_cpu()]
-    for i, sample in enumerate(output):
-        sample = np.array(sample)
-        if offset_fraction == 1:
-            assert np.all(sample == fill_value), f"sample_idx: {i}"
-        else:
-            background_count = np.sum(sample == fill_value)
-            assert background_count / sample.size < 0.1, \
-                f"sample_idx: {i}, {background_count / sample.size}"
-
-


It's now common utility for AA/TA/RA modules. Tested in test_augmentations.

stiepan · 2023-03-30T11:32:09Z

dali/test/python/auto_aug/test_trivial_augment.py

-@params(*tuple(itertools.product((True, False), (0, 1), ('x', 'y'))))
-def test_translation(use_shape, offset_fraction, extent):
-    # make sure the translation helper processes the args properly
-    # note, it only uses translate_y (as it is in imagenet policy)
-    fill_value = 0
-    params = {}
-    if use_shape:
-        param = offset_fraction
-        param_name = "max_translate_rel"
-    else:
-        param = 1000 * offset_fraction
-        param_name = "max_translate_abs"
-    params[param_name] = param
-    translation_x, translation_y = trivial_augment._get_translations(use_shape=use_shape, **params)
-    augment = [translation_x] if extent == 'x' else [translation_y]
-
-    @experimental.pipeline_def(enable_conditionals=True, batch_size=9, num_threads=4, device_id=0,
-                               seed=43)
-    def pipeline():
-        encoded_image, _ = fn.readers.file(name="Reader", file_root=images_dir)
-        image = fn.decoders.image(encoded_image, device="mixed")
-        if use_shape:
-            shape = fn.peek_image_shape(encoded_image)
-            return trivial_augment.apply_trivial_augment(augment, image, num_magnitude_bins=3,
-                                                         fill_value=fill_value, shape=shape)
-        else:
-            return trivial_augment.apply_trivial_augment(augment, image, num_magnitude_bins=3,
-                                                         fill_value=fill_value)
-
-    p = pipeline()
-    p.build()
-    output, = p.run()
-    output = [np.array(sample) for sample in output.as_cpu()]
-    if offset_fraction == 1:
-        # magnitudes are random here, but some should randomly be maximal
-        all_black = 0
-        for i, sample in enumerate(output):
-            sample = np.array(sample)
-            all_black += np.all(sample == fill_value)
-        assert all_black
-    else:
-        for i, sample in enumerate(output):
-            sample = np.array(sample)
-            background_count = np.sum(sample == fill_value)
-            assert background_count / sample.size < 0.1, \
-                f"sample_idx: {i}, {background_count / sample.size}"
-
-


It's now common utility for AA/TA/RA modules. Tested in test_augmentations.

stiepan · 2023-03-30T11:32:17Z

dali/python/nvidia/dali/auto_aug/rand_augment.py

-def _get_translations(use_shape: bool = False, max_translate_abs: Optional[int] = None,
-                      max_translate_rel: Optional[float] = None) -> List[_Augmentation]:
-    max_translate_height, max_translate_width = _parse_validate_offset(
-        use_shape, max_translate_abs=max_translate_abs, max_translate_rel=max_translate_rel,
-        default_translate_abs=100, default_translate_rel=100 / 224)
-    if use_shape:
-        return [
-            a.translate_x.augmentation((0, max_translate_width), True),
-            a.translate_y.augmentation((0, max_translate_height), True),
-        ]
-    else:
-        return [
-            a.translate_x_no_shape.augmentation((0, max_translate_width), True),
-            a.translate_y_no_shape.augmentation((0, max_translate_height), True),
-        ]


It's now common utility for AA/TA/RA modules. Defined in core._utils.

stiepan · 2023-03-30T11:32:40Z

dali/python/nvidia/dali/auto_aug/trivial_augment.py

-
-
-def _get_translations(use_shape: bool = False, max_translate_abs: Optional[int] = None,
-                      max_translate_rel: Optional[float] = None) -> List[_Augmentation]:
-    max_translate_height, max_translate_width = _parse_validate_offset(
-        use_shape, max_translate_abs=max_translate_abs, max_translate_rel=max_translate_rel,
-        default_translate_abs=32, default_translate_rel=1.)
-    if use_shape:
-        return [
-            a.translate_x.augmentation((0, max_translate_width), True),
-            a.translate_y.augmentation((0, max_translate_height), True),
-        ]
-    else:
-        return [
-            a.translate_x_no_shape.augmentation((0, max_translate_width), True),
-            a.translate_y_no_shape.augmentation((0, max_translate_height), True),
-        ]


It's now common utility for AA/TA/RA modules. Defined in core._utils.

stiepan · 2023-03-30T11:34:34Z

!build

dali-automaton · 2023-03-30T11:40:55Z

CI MESSAGE: [7763263]: BUILD STARTED

jantonguirao · 2023-03-30T12:30:50Z

dali/test/python/auto_aug/test_augmentations.py

+            param_shape = shape
+            param_name = "max_translate_abs"
+        if extent == 'both':
+            param = [param_shape[0] * offset_fraction, param_shape[1] * offset_fraction]


nitpick:

Suggested change

param = [param_shape[0] * offset_fraction, param_shape[1] * offset_fraction]

param = offset_fraction * param_shape[:2]

or

Suggested change

param = [param_shape[0] * offset_fraction, param_shape[1] * offset_fraction]

param = offset_fraction * param_shape

I am not sure if that helps - the * works as composition of concatentation not by broadcasting, so I would either get [] or the same list.

dali-automaton · 2023-03-30T14:29:58Z

CI MESSAGE: [7763263]: BUILD PASSED

stiepan · 2023-04-03T15:47:34Z

Rebased onto #4747

stiepan · 2023-04-03T16:03:27Z

!build

dali-automaton · 2023-04-03T16:05:39Z

CI MESSAGE: [7802387]: BUILD STARTED

dali-automaton · 2023-04-03T18:02:18Z

CI MESSAGE: [7802387]: BUILD FAILED

dali-automaton · 2023-04-04T10:27:34Z

CI MESSAGE: [7802387]: BUILD PASSED

Signed-off-by: Kamil Tokarski <[email protected]>

…always skipped augmentations Signed-off-by: Kamil Tokarski <[email protected]>

Signed-off-by: Kamil Tokarski <[email protected]>

stiepan · 2023-04-04T15:33:13Z

Rebased onto #4751

stiepan · 2023-04-04T15:33:30Z

!build

dali-automaton · 2023-04-04T15:35:23Z

CI MESSAGE: [7815611]: BUILD STARTED

klecki · 2023-04-04T14:31:05Z

dali/python/nvidia/dali/auto_aug/auto_augment.py

-        augmentations. If tuple is specified, the first component limits height, the second the
-        width.
+        augmentations. If a tuple is specified, the first component limits height, the second the
+        width. Defaults to 250.
    max_translate_rel: float or (float, float), optional


The tuple is not specified in the type annotation, but I am not sure if those are not an overkill. The docs rendering of such long annotations is a bit problematic.

dali-automaton · 2023-04-04T17:56:29Z

CI MESSAGE: [7815611]: BUILD FAILED

stiepan · 2023-04-06T08:25:21Z

!build

dali-automaton · 2023-04-06T08:31:29Z

CI MESSAGE: [7836392]: BUILD STARTED

dali-automaton · 2023-04-06T13:20:30Z

CI MESSAGE: [7836392]: BUILD PASSED

stiepan added the automatic augmentations Automatic augmentations (AutoAugment, RandAugment, TrivialAugment and more) support in DALI. label Mar 29, 2023

jantonguirao assigned jantonguirao and szalpal Mar 30, 2023

github-advanced-security bot found potential problems Mar 30, 2023

View reviewed changes

dali/test/python/auto_aug/test_augmentations.py Fixed Show fixed Hide fixed

stiepan commented Mar 30, 2023

View reviewed changes

jantonguirao reviewed Mar 30, 2023

View reviewed changes

jantonguirao approved these changes Mar 30, 2023

View reviewed changes

klecki assigned klecki and unassigned szalpal Mar 31, 2023

stiepan force-pushed the aa_more_policies branch from 11e0d27 to 13f8614 Compare April 3, 2023 15:47

stiepan mentioned this pull request Apr 4, 2023

Simplify AutoAugment graph #4751

Merged

18 tasks

stiepan added 7 commits April 4, 2023 15:47

Add warning on 0 probability augs, allow mag_range None

4b527db

Signed-off-by: Kamil Tokarski <[email protected]>

Adjust the v0 policy to silence warnings

a877a35

Signed-off-by: Kamil Tokarski <[email protected]>

Adjust error messages and tests accordingly

b15993d

Signed-off-by: Kamil Tokarski <[email protected]>

Add reduced ImageNet and reduced Cifar10 policies

b6e43b5

Signed-off-by: Kamil Tokarski <[email protected]>

More AA policies, document shape and max translation everywhere

10661ae

Signed-off-by: Kamil Tokarski <[email protected]>

Optimize policy specifications: remove unused magnitude bins, remove …

fcc6beb

…always skipped augmentations Signed-off-by: Kamil Tokarski <[email protected]>

Unify get_translation utility for AA/TA/RA

0903582

Signed-off-by: Kamil Tokarski <[email protected]>

stiepan added 3 commits April 4, 2023 15:47

Adjust AA tests to run all predefined policies

319e0c0

Signed-off-by: Kamil Tokarski <[email protected]>

Add tests for new policy format warnings and errors

d2e7da9

Signed-off-by: Kamil Tokarski <[email protected]>

Expand the comment

5fd45e0

Signed-off-by: Kamil Tokarski <[email protected]>

stiepan force-pushed the aa_more_policies branch from 13f8614 to 5fd45e0 Compare April 4, 2023 15:32

klecki approved these changes Apr 4, 2023

View reviewed changes

stiepan merged commit ed89a2a into NVIDIA:main Apr 6, 2023

stiepan mentioned this pull request Apr 6, 2023

Support CPU samples in predefined automatic augmentations #4772

Merged

18 tasks

JanuszL mentioned this pull request Sep 6, 2023

Roadmap 2023 #4578

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add more AutoAugment policies #4753

Add more AutoAugment policies #4753

stiepan commented Mar 29, 2023 •

edited

Loading

stiepan Mar 30, 2023

stiepan Mar 30, 2023

stiepan Mar 30, 2023

stiepan Mar 30, 2023 •

edited

Loading

stiepan Mar 30, 2023

stiepan commented Mar 30, 2023

dali-automaton commented Mar 30, 2023

jantonguirao Mar 30, 2023

stiepan Mar 30, 2023

dali-automaton commented Mar 30, 2023

stiepan commented Apr 3, 2023

stiepan commented Apr 3, 2023

dali-automaton commented Apr 3, 2023

dali-automaton commented Apr 3, 2023

dali-automaton commented Apr 4, 2023

stiepan commented Apr 4, 2023

stiepan commented Apr 4, 2023

dali-automaton commented Apr 4, 2023

klecki Apr 4, 2023

dali-automaton commented Apr 4, 2023

stiepan commented Apr 6, 2023

dali-automaton commented Apr 6, 2023

dali-automaton commented Apr 6, 2023

	param = [param_shape[0] * offset_fraction, param_shape[1] * offset_fraction]
	param = offset_fraction * param_shape[:2]

	param = [param_shape[0] * offset_fraction, param_shape[1] * offset_fraction]
	param = offset_fraction * param_shape

Add more AutoAugment policies #4753

Add more AutoAugment policies #4753

Conversation

stiepan commented Mar 29, 2023 • edited Loading

Category:

Description:

Additional information:

Affected modules and functionalities:

Key points relevant for the review:

Tests:

Checklist

Documentation

DALI team only

Requirements

stiepan Mar 30, 2023

Choose a reason for hiding this comment

stiepan Mar 30, 2023

Choose a reason for hiding this comment

stiepan Mar 30, 2023

Choose a reason for hiding this comment

stiepan Mar 30, 2023 • edited Loading

Choose a reason for hiding this comment

stiepan Mar 30, 2023

Choose a reason for hiding this comment

stiepan commented Mar 30, 2023

dali-automaton commented Mar 30, 2023

jantonguirao Mar 30, 2023

Choose a reason for hiding this comment

stiepan Mar 30, 2023

Choose a reason for hiding this comment

dali-automaton commented Mar 30, 2023

stiepan commented Apr 3, 2023

stiepan commented Apr 3, 2023

dali-automaton commented Apr 3, 2023

dali-automaton commented Apr 3, 2023

dali-automaton commented Apr 4, 2023

stiepan commented Apr 4, 2023

stiepan commented Apr 4, 2023

dali-automaton commented Apr 4, 2023

klecki Apr 4, 2023

Choose a reason for hiding this comment

dali-automaton commented Apr 4, 2023

stiepan commented Apr 6, 2023

dali-automaton commented Apr 6, 2023

dali-automaton commented Apr 6, 2023

stiepan commented Mar 29, 2023 •

edited

Loading

stiepan Mar 30, 2023 •

edited

Loading