HADOOP-12345: Compute the correct credential length #104
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This has been checked in |
shanthoosh
added a commit
to shanthoosh/hadoop
that referenced
this pull request
Oct 15, 2019
Currently when a job is submitted to Yarn, it is going to hang after AppMaster is created. The log shows that it hangs during bootstrapping from Coordinator stream. Further debugging shows that the jobs hang in the second time of bootstrap while reading locality data from LocalityManager. The sequence is the following: 1. JobModelManager creates CoordinatorStreamConsumer, and bootstrap it, 2. LocalityManager writes locality info into coordinator stream 3. JobModelManager closes CoordinatorStreamConsumer 4. Later localityManager bootstraps CoordinatorStreamConsumer again Step 3 is the problem here. Since CoordinatorStreamConsumer is still held by LocalityManager, it cannot be closed prematurely. Step 3 is introduced in SAMZA-1154, as a refactoring of JobModelManager for task rest end point. To fix this issue, we will revert this change of step 3. Author: Shanthoosh Venkataraman <[email protected]> Reviewers: Xinyu Liu <[email protected]> Closes apache#104 from shanthoosh/master
This was referenced Feb 20, 2024
Closed
This was referenced Jul 17, 2024
This was referenced Jul 26, 2024
This was referenced Aug 2, 2024
This was referenced Aug 12, 2024
This was referenced Aug 20, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I had to discard my earlier pull request as that included fix for the jira HADOOP-11823, hence creating a separate one for each of them.
The fix here addresses the correct credential length to be computed. We need to round of the machine name length to the next multiple of 4, else using such a credential in the NFS RPC request will result in GARBAGE_ARGS from the NFS server. See RFC-5531, page 24 and page 8