Add triplet loss for metric lerning. #11461
Conversation
2. Add python wrapper.
|
@BigFishMaster 你那里有 |
|
@wanghaoshuang 我稍后看看哈 |
There was a problem hiding this comment.
Please complete the doc, especially the loss calculation formula, for conveniently review. Thanks! @wanghaoshuang @BigFishMaster
|
|
||
| template <typename T, int MajorType = Eigen::RowMajor, | ||
| typename IndexType = Eigen::DenseIndex> | ||
| using EigenVector = framework::EigenVector<T, MajorType, IndexType>; |
There was a problem hiding this comment.
EigenVector is not used in following code.
| auto blas = math::GetBlas<DeviceContext, T>(context); | ||
| auto x_mat = math::CreateMatrixDescriptor(x_dims, 0, false); | ||
| auto x_mat_trans = math::CreateMatrixDescriptor(x_dims, 0, true); | ||
| blas.MatMul(*logits, x_mat, *logits, x_mat_trans, T(1), &distances, T(0)); |
There was a problem hiding this comment.
Since the input is 2D tensor, there is no need to use math::CreateMatrixDescriptor, refer the MatMul inferface:
https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/math/blas_impl.h#L176
| using EigenVector = framework::EigenVector<T, MajorType, IndexType>; | ||
|
|
||
| using DIM1 = Eigen::array<int, 1>; | ||
| using DIM2 = Eigen::array<int, 2>; |
There was a problem hiding this comment.
DIM1 -> Array1 ?
DIM2 -> Array2?
…o origin/triplet
| - a negative sample with a different class | ||
| We define the three samples as $a$, $p$, $n$. Then the loss of | ||
| the triplet (a, p, n) is: | ||
| $$L = max(d(a, p) - d(a, n) + margin, 0)$$ |
| "computed with loss together in one call. It is a 2-D Tensor of " | ||
| "the shape [N, feature_len].") | ||
| .AsIntermediate(); | ||
| AddAttr<float>("margin", "(float), The min margin between two sample."); |
| We define the three samples as $a$, $p$, $n$. Then the loss of | ||
| the triplet (a, p, n) is: | ||
| $$L = max(d(a, p) - d(a, n) + margin, 0)$$ | ||
| In which, $d(a, p)$ means the distance between $a$ and $p$. The negative should |
There was a problem hiding this comment.
distance -> L2 distance ?
| @@ -5184,3 +5185,52 @@ def crop(x, shape=None, offsets=None, name=None): | |||
| outputs={'Out': out}, | |||
| attrs=None if len(attrs) == 0 else attrs) | |||
| return out | |||
|
|
|||
|
|
|||
| def dense_triplet_loss(input, label, margin=0.01): | |||
There was a problem hiding this comment.
Add name:
dense_triplet_loss(input, label, margin=0.01, name=None)
| "and K is the feature length in each sample."); | ||
| AddInput("Label", | ||
| "(Tensor) The ground truth which is a 2-D tensor. " | ||
| "Label is a Tensor<int64> with shape [N x 1]. "); |
| self.batch_size = 9 | ||
| self.feature_len = 3 | ||
| self.class_num = 4 | ||
| self.eps = 0.01 |
There was a problem hiding this comment.
self.eps -> self.margin ?
| self.check_grad( | ||
| ["Logits"], | ||
| "Loss", | ||
| max_relative_error=0.05, |
There was a problem hiding this comment.
forward里有relu，这里的gradient check有随机错误吗？
前向的计算过程：https://wanghaoshuang.github.io/2018/07/Dense-Triplet_Loss/