Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OptimType.NONE in SplitTBE (defuse bwd and optim) #1819

Closed
wants to merge 1 commit into from
Closed

Add OptimType.NONE in SplitTBE (defuse bwd and optim) #1819

wants to merge 1 commit into from

Commits on Jun 12, 2023

  1. Add OptimType.NONE in SplitTBE (defuse bwd and optim) (pytorch#1819) 

    Summary:
Pull Request resolved: pytorch#1819

This diff is the **backend** part

This diff introduces `OptimType.NONE`.  Unlike other `OptimType`s,
`OptimType.NONE` does not perform the optimizer step during SplitTBE's
backward pass.  With `OptimType.NONE`, SplitTBE deduplicates output
gradients in the backward pass and generates a sparse gradient tensor
(PyTorch's `sparse_coo_tensor`) for the device's weight (FQN:
`weights_dev`).

Currently, `OptimType.NONE` only supports the case where the embedding
dimensions of all embedding tables are identical.

Differential Revision: D44392172

fbshipit-source-id: 52d746963b772f6ddaada7630cdf4b53d1e71ed3
    @sryap @facebook-github-bot
    sryap authored and facebook-github-bot committed Jun 12, 2023
    Configuration menu
    Copy the full SHA
    e14b055 View commit details
    Browse the repository at this point in the history