Fix move_metrics_to_cpu with evaluation

[carmocca]

What does this PR do?

Fixes the bug uncovered by the analysis here: #10595 (comment)

• The outputs are no longer moved to CPU with move_metrics_to_cpu=True. This is consistent with how the training loops work.
• The recursive detach() call has been removed, it's not necessary as gradients arent computed during evaluation.
• move_metrics_to_cpu=True now actually moves the metrics to CPU. This is consistent with how the training loops work.

Does your PR introduce any breaking changes? If yes, please list them.

• If the user relies on the outputs being on CPU in {validation,test}_epoch_end.
• If the user memory blows up due to this change. Keep in mind that the user is always able to do return your_tensor_on_gpu.cpu().

Before submitting

• Was this discussed/approved via a GitHub issue? (not for typos and docs)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together?
• [n/a] Did you make sure to update the documentation with your changes? (if necessary)
• Did you write any new necessary tests? (not for typos and docs)
• Did you verify new and existing tests pass locally with your changes?
• Did you list all the breaking changes introduced by this pull request?
• Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

• Is this pull request ready for review? (if not, please submit in draft mode)
• Check that all items from Before submitting are resolved
• Make sure the title is self-explanatory and the description concisely explains the PR
• Add labels and milestones (and optionally projects) to the PR so it can be classified

cc @Borda @carmocca @edward-io @ananthsub @awaelchli

[github-actions]

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

[codecov]

Codecov Report

Merging #10631 (67b8c26) into master (5788789) will decrease coverage by 4%.
The diff coverage is 60%.

@@           Coverage Diff            @@
##           master   #10631    +/-   ##
========================================
- Coverage      92%      88%    -4%     
========================================
  Files         178      178            
  Lines       16266    16303    +37     
========================================
- Hits        14923    14339   -584     
- Misses       1343     1964   +621     

[tchaton]

LGTM !