Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[API 2.0] Add transformer apis #26418

Merged
merged 22 commits into from
Aug 24, 2020
Merged

[API 2.0] Add transformer apis #26418

merged 22 commits into from
Aug 24, 2020

Conversation

guoshengCS
Copy link
Contributor

@guoshengCS guoshengCS commented Aug 18, 2020 •
edited
Loading

PR types

New features

PR changes

APIs

Describe

Add transformer apis, as follows:

    'MultiheadAttention',
    'TransformerEncoderLayer',
    'TransformerEncoder',
    'TransformerDecoderLayer',
    'TransformerDecoder',
    'Transformer'

guoshengCS and others added 19 commits August 15, 2020 19:05
@guoshengCS
Add MultiHeadAttention api.
a348883 
test=develop
@guoshengCS
Add MultiHeadAttention cache type and gen_cache.
b67bd96 
test=develop
@guoshengCS
Add TransformerEncoderLayer and TransformerEncoder.
d3c1709 
test=develop
@guoshengCS
Add Transformer decoder apis.
87e1106 
test=develop
@guoshengCS
Add Transformer api.
c50ad43 
test=develop
@LiuChiachi
add unittests for transformer api
ad7d225
@LiuChiachi
Merge branch 'add-api-transformer' of https://github.com/guoshengCS/P…
44d92ce 
…addle into transformer
@LiuChiachi
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
a3bebcb 
… transformer
@LiuChiachi
add unittests for transformer api
54e9e56
@guoshengCS
Fix some bugs in Transformer apis.
8637eee 
test=develop
@LiuChiachi
add unittests for encoder, decoder and transformer
33741e8
@LiuChiachi
solve conflics
11b5b8f
@LiuChiachi
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
48a7073 
… transformer
@LiuChiachi
clean conflicts infor in code
5578a0d
@LiuChiachi
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
0bb65e0 
… transformer
@LiuChiachi
clean Chinese comments
a7a0003
@guoshengCS
Add TransformerDecoderCell and TransformerBeamSearchDecoder.
d2c25dc 
test=develop
@guoshengCS
Remove TransformerDecoderCell and TransformerBeamSearchDecoder tempor…
c768e6e 
…arily.

test=develop
@guoshengCS
Add import for Transformer apis.
0d01c16 
test=develop
XiaoguangHu01
XiaoguangHu01 previously approved these changes Aug 22, 2020
View reviewed changes
Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

python/paddle/nn/layer/__init__.py

from .activation import *
from .loss import *
from .conv import *
from .extension import *
from .activation import *
from .norm import *
from .transformer import *
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

大写的组网类，在paddle.nn下建别名，推荐用paddle.nn.Transformer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Thanks

python/paddle/nn/layer/transformer.py Outdated Show resolved Hide resolved
python/paddle/nn/layer/transformer.py Outdated

return (q, k, v) if cache is None else (q, k, v, cache)

def cal_kv(self, key, value):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calculate 的缩写用 calc 更多些

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Rename as compute_kv. Thanks

python/paddle/nn/layer/transformer.py Outdated

return (q, k, v) if cache is None else (q, k, v, cache)

def cal_kv(self, key, value):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个API用户是否需要使用？如果需要的话，建议修改下。
calculate 的缩写用 calc 更多些，或者这里用compute也可以

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Rename as compute_kv. Thanks

iclementine
iclementine previously approved these changes Aug 22, 2020
View reviewed changes
Copy link

@iclementine iclementine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

iclementine
iclementine reviewed Aug 22, 2020
View reviewed changes
python/paddle/nn/layer/transformer.py Outdated
import copy
import collections

import numpy as np
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里其实可以不必 import numpy

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Thanks

@guoshengCS guoshengCS merged commit 317f7ce into PaddlePaddle:develop Aug 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@iclementine iclementine iclementine left review comments

@raindrops2sea raindrops2sea raindrops2sea approved these changes

@XiaoguangHu01 XiaoguangHu01 XiaoguangHu01 left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@guoshengCS @iclementine @raindrops2sea @XiaoguangHu01 @LiuChiachi