Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[LIVY-697] Rsc client cannot resolve the hostname of driver in yarn-cluster mode #246

Closed
wants to merge 1 commit into from

Conversation

runzhiwang
Copy link
Contributor

@runzhiwang runzhiwang commented Oct 14, 2019

What changes were proposed in this pull request?

[LIVY-697] Rsc client cannot resolve the hostname of driver in yarn-cluster mode

  1. The content of Driver in /etc/hosts are as follows:
    127.0.0.1 localhost
    10.10.10.10 test_hostname

  2. The content of Driver in /etc/hostname are as follows:
    test_hostname

  3. The findLocalAddress method in livy cannot return 10.10.10.10, but return test_hostname.
    Because the test_hostname point to 10.10.10.10, which doesn't pass the check
    address.isLoopbackAddress(), so findLocalAddress return test_hostname .

  4. The rsc client cannot resolve the test_hostname, which cause rsc client cannot connect to
    driver.

  5. The findLocalAddress method in livy can return 10.10.10.10 as expected if the content
    of Driver in /etc/hosts are as follows, which is not correct in our environment.
    127.0.0.1 localhost
    127.0.0.1 test_hostname

  6. Though I can modify the findLocalAddress method to return 10.10.10.10, but it maybe cause
    error if the machine has multiple network cards. So, rsc client gets the driver ip from the
    connection.

How was this patch tested?

  1. The content of Driver in /etc/hosts are as follows:
    127.0.0.1 localhost
    127.0.0.1 test_hostname

  2. The content of Driver in /etc/hostname are as follows:
    test_hostname

  3. rsc client can get the driver ip in connection, and connect to driver successfully.

@runzhiwang runzhiwang closed this Oct 14, 2019
@runzhiwang runzhiwang reopened this Oct 14, 2019
@codecov-io
Copy link

codecov-io commented Oct 14, 2019

Codecov Report

Merging #246 into master will increase coverage by 0.04%.
The diff coverage is 100%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #246      +/-   ##
============================================
+ Coverage     68.08%   68.12%   +0.04%     
  Complexity      939      939              
============================================
  Files           101      101              
  Lines          5853     5855       +2     
  Branches        886      886              
============================================
+ Hits           3985     3989       +4     
+ Misses         1296     1295       -1     
+ Partials        572      571       -1
Impacted Files Coverage Δ Complexity Δ
...main/java/org/apache/livy/rsc/ContextLauncher.java 66.34% <100%> (+0.32%) 13 <0> (ø) ⬇️
...ain/java/org/apache/livy/rsc/driver/RSCDriver.java 80.75% <0%> (ø) 45% <0%> (ø) ⬇️
...c/main/scala/org/apache/livy/repl/ReplDriver.scala 33.33% <0%> (+2.56%) 7% <0%> (ø) ⬇️
...cala/org/apache/livy/scalaapi/ScalaJobHandle.scala 55.88% <0%> (+2.94%) 7% <0%> (ø) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2e7d691...4a58338. Read the comment docs.

Copy link
Contributor

@jerryshao jerryshao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants