Complex XPATH is not found by Python Selenium

[GBR-613]

Problem summary:
Selenium does not find elements with "too complex" XPATH

This is the HTML document to reproduce the problem:
`

<script>

function getElementByXpath(path) {
result1 = document.evaluate(path, document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;
try {
evaluator = new XPathEvaluator();
xpath_expr = evaluator.createExpression(path, evaluator.createNSResolver(document)); //fails in MS Edge
result2 = xpath_expr.evaluate(document, XPathResult.FIRST_ORDERED_NODE_TYPE).singleNodeValue;
} catch(e) {
result2 = "";
}

return result1 || result2;

}

function test_xpath() {
path1 = '//div[contains(@Class,"subject company_info")]/div[1]/text()[2]';
path2 = '//*[@id="tabs-info"]/div[4]/div[1]/text()[2]';
e1 = getElementByXpath(path1);
r1 = document.getElementById("i1")
e2 = getElementByXpath(path2);
r2 = document.getElementById("i2")
if (!e1) {
r1.textContent = "... Not found";
} else {
r1.textContent = e1.textContent;
}
if (!e2) {
r2.textContent = "... Not found";
} else {
r2.textContent = e2.textContent;
}

}

</script>

 

 

 

	<div>
		<strong>Company type:</strong> Private Limited Company <br>
		<strong>Company number:</strong> 123456789 <br>
		<strong>Company status:</strong> Active <br>
	</div>
	<div class="subject">
		country of origin: United Kingdom <br>
		incorporation date: 2020.01.01<br>
		has UK establishment:  No   <br>
		has appointments:  Yes   <br>
		in liquidation:  No  <br>
	</div>

	<h4>Nature of Business:</h4>
	<div class="subject">
		<ul>
			<li>82990 - Other business support service activities not elsewhere classified</li>
		</ul>
	</div>			
</div>

`

My aim is to locate element with text "Private Limited Company".
According to both Chrome and Firefox Dev tools, its xpath is the following: //*[@id="tabs-info"]/div[4]/div[1]/text()[1]
Javascript does not find this xpath in runtime, however the very similar xpath is found: //*[@id="tabs-info"]/div[4]/div[1]/text()[2]
I opened tickets against the respective browsers about inconsistency between Dev Tools and Javascript engine.
However Selenium does not find any of them (NoSuchElementException is raised).
The element can be also located by xpath based on a class selector: //div[contains(@class,"subject company_info")]/div[1]/text()[2]
Javascript finds it. Selenium does not find it, too.

This is my Python code to invoke Selenium:
`
from selenium import webdriver

driver = webdriver.PhantomJS()
driver.set_window_size(1120, 550)
driver.get(url="file:///home/user/test.html")
path1 = '//*[@id="tabs-info"]/div[4]/div[1]/text()[2]'
path2 = '//div[contains(@Class,"subject company_info")]/div[1]/text()[2]'
e1 = driver.find_element(By.XPATH, path1)
print("Path #1: " + e1.text)
e2 = driver.find_element(By.XPATH, path2)
print("Path #2: " + e2.text)
driver.quit()
`
Tested with Firefox and Chrome on Linux. Also Selenium IDE playback fails with such kinds of xpath in both Firefox and Chrome on Windows 10.

[ghost]

👋 Hi there! Thank you for creating this issue.

I am the Selenium Assistant Bot 🤖, I triage issues in this repository. If I can't do it, I label it to help maintainers identify issues that need triaging.

I am an Open Source project 🙌, post bugs or ideas here!

[ghost]

❗️ It seems this issue is not using any of the supported templates

💡 Supported issue types are (they start with):

• 🐛 Bug Report (bugs found in a recent release)
• 🚀 Feature Proposal (a useful feature you would like to propose)
• 💥 Regression Report (a supported feature is not working anymore)

Issue templates help this project to stay in shape, please use them and fill them out completely. By doing that you are helping the project because the community and maintainers can provide prompt feedback, and potentially solve the issue.

If you are asking a question, a better way to address this is:

• 📫 Send questions and support requests to the Selenium user group
• 📮 Post them to StackOverflow
• 🗣 Join us in the IRC/Slack channel where the community can help you as well

If you think this is incorrect, please feel free to open a new issue.

Thank you for your contributions.

[AutomatedTester]

This is likely due to phantomJs can’t do that. Since phantomJs is no longer maintained try Chrome or Firefox headless.

…

On Apr 10, 2020, 12:23 PM +0100, selenium-assistant[bot] ***@***.***>, wrote: Closed #8190. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.