Fix field validation in VectorReader

[jmazanec15]

Description

This PR fixes the bug when training data index shard is not on training node. It uses the validation logic used in training model request in VectorReader. Additionally, for VectorReader, IndicesService is swapped out with ClusterService.

Lastly, this PR adds a couple error increments in TrainingJob that were not getting picked up.

Issues Resolved

#205

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov-commenter]

Codecov Report

Merging #207 (a8d0afe) into main (3876560) will increase coverage by 0.15%.
The diff coverage is 66.17%.

@@             Coverage Diff              @@
##               main     #207      +/-   ##
============================================
+ Coverage     83.10%   83.26%   +0.15%     
- Complexity      861      864       +3     
============================================
  Files           123      123              
  Lines          3771     3777       +6     
  Branches        361      358       -3     
============================================
+ Hits           3134     3145      +11     
+ Misses          475      471       -4     
+ Partials        162      161       -1     

Impacted Files	Coverage Δ
.../main/java/org/opensearch/knn/index/IndexUtil.java	52.72% <56.25%> (+24.15%)	⬆️
...rch/knn/plugin/transport/TrainingModelRequest.java	83.69% <77.77%> (+13.84%)	⬆️
...rch/knn/index/memory/NativeMemoryEntryContext.java	100.00% <100.00%> (ø)
...rch/knn/index/memory/NativeMemoryLoadStrategy.java	96.61% <100.00%> (ø)
...plugin/transport/TrainingModelTransportAction.java	83.87% <100.00%> (ø)
.../java/org/opensearch/knn/training/TrainingJob.java	87.30% <100.00%> (+0.63%)	⬆️
...java/org/opensearch/knn/training/VectorReader.java	84.00% <100.00%> (-0.22%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3876560...a8d0afe. Read the comment docs.

[vamshin]

LGTM! Thanks