Hello Happy New Year! Answer useful please adopt, click on the right side of the answer can be adopted!
In PyTorch, you can use the torch.topk() function to get the values and subscripts of the first k maximum values in a given tensor. Back propagation can be achieved by propagating the gradient of the output tensor back to the input tensor.
For example:
import torch
x = torch.randn(3, 4)
values, indices = torch.topk(x, 2)
#values 就是前2大的数
#indices 就是前2大的数的下标
x.requiresGrad = True
loss = values.sum()
loss.backward()
#x 的梯度就是前2大的数的梯度
Also, if you want to keep only the gradients of the first k maximum values in the backpropagation, you can use torch.index_select() to set the gradients of the rest to 0.
x.grad[indices] = 0
The
code sets the gradient of x to 0 except for the first k maximum values.