[selenium] What is the difference between cssSelector & Xpath and which is better with respect to performance for cross browser testing?

The debate between cssSelector vs XPath would remain as one of the most subjective debate in the Selenium Community. What we already know so far can be summarized as:

  • People in favor of cssSelector say that it is more readable and faster (especially when running against Internet Explorer).
  • While those in favor of XPath tout it's ability to transverse the page (while cssSelector cannot).
  • Traversing the DOM in older browsers like IE8 does not work with cssSelector but is fine with XPath.
  • XPath can walk up the DOM (e.g. from child to parent), whereas cssSelector can only traverse down the DOM (e.g. from parent to child)
  • However not being able to traverse the DOM with cssSelector in older browsers isn't necessarily a bad thing as it is more of an indicator that your page has poor design and could benefit from some helpful markup.
  • Ben Burton mentions you should use cssSelector because that's how applications are built. This makes the tests easier to write, talk about, and have others help maintain.
  • Adam Goucher says to adopt a more hybrid approach -- focusing first on IDs, then cssSelector, and leveraging XPath only when you need it (e.g. walking up the DOM) and that XPath will always be more powerful for advanced locators.

Dave Haeffner carried out a test on a page with two HTML data tables, one table is written without helpful attributes (ID and Class), and the other with them. I have analyzed the test procedure and the outcome of this experiment in details in the discussion Why should I ever use cssSelector selectors as opposed to XPath for automated testing?. While this experiment demonstrated that each Locator Strategy is reasonably equivalent across browsers, it didn't adequately paint the whole picture for us. Dave Haeffner in the other discussion Css Vs. X Path, Under a Microscope mentioned, in an an end-to-end test there were a lot of other variables at play Sauce startup, Browser start up, and latency to and from the application under test. The unfortunate takeaway from that experiment could be that one driver may be faster than the other (e.g. IE vs Firefox), when in fact, that's wasn't the case at all. To get a real taste of what the performance difference is between cssSelector and XPath, we needed to dig deeper. We did that by running everything from a local machine while using a performance benchmarking utility. We also focused on a specific Selenium action rather than the entire test run, and run things numerous times. I have analyzed the specific test procedure and the outcome of this experiment in details in the discussion cssSelector vs XPath for selenium. But the tests were still missing one aspect i.e. more browser coverage (e.g., Internet Explorer 9 and 10) and testing against a larger and deeper page.

Dave Haeffner in another discussion Css Vs. X Path, Under a Microscope (Part 2) mentions, in order to make sure the required benchmarks are covered in the best possible way we need to consider an example that demonstrates a large and deep page.


Test SetUp

To demonstrate this detailed example, a Windows XP virtual machine was setup and Ruby (1.9.3) was installed. All the available browsers and their equivalent browser drivers for Selenium was also installed. For benchmarking, Ruby's standard lib benchmark was used.


Test Code

require_relative 'base'
require 'benchmark'

class LargeDOM < Base

  LOCATORS = {
    nested_sibling_traversal: {
      css: "div#siblings > div:nth-of-type(1) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3) > div:nth-of-type(3)",
      xpath: "//div[@id='siblings']/div[1]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]/div[3]"
    },
    nested_sibling_traversal_by_class: {
      css: "div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1 > div.item-1",
      xpath: "//div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]/div[contains(@class, 'item-1')]"
    },
    table_header_id_and_class: {
      css: "table#large-table thead .column-50",
      xpath: "//table[@id='large-table']//thead//*[@class='column-50']"
    },
    table_header_id_class_and_direct_desc: {
      css: "table#large-table > thead .column-50",
      xpath: "//table[@id='large-table']/thead//*[@class='column-50']"
    },
    table_header_traversing: {
      css: "table#large-table thead tr th:nth-of-type(50)",
      xpath: "//table[@id='large-table']//thead//tr//th[50]"
    },
    table_header_traversing_and_direct_desc: {
      css: "table#large-table > thead > tr > th:nth-of-type(50)",
      xpath: "//table[@id='large-table']/thead/tr/th[50]"
    },
    table_cell_id_and_class: {
      css: "table#large-table tbody .column-50",
      xpath: "//table[@id='large-table']//tbody//*[@class='column-50']"
    },
    table_cell_id_class_and_direct_desc: {
      css: "table#large-table > tbody .column-50",
      xpath: "//table[@id='large-table']/tbody//*[@class='column-50']"
    },
    table_cell_traversing: {
      css: "table#large-table tbody tr td:nth-of-type(50)",
      xpath: "//table[@id='large-table']//tbody//tr//td[50]"
    },
    table_cell_traversing_and_direct_desc: {
      css: "table#large-table > tbody > tr > td:nth-of-type(50)",
      xpath: "//table[@id='large-table']/tbody/tr/td[50]"
    }
  }

  attr_reader :driver

  def initialize(driver)
    @driver = driver
    visit '/large'
    is_displayed?(id: 'siblings')
    super
  end

  # The benchmarking approach was borrowed from
  # http://rubylearning.com/blog/2013/06/19/how-do-i-benchmark-ruby-code/
  def benchmark
    Benchmark.bmbm(27) do |bm|
      LOCATORS.each do |example, data|
    data.each do |strategy, locator|
      bm.report(example.to_s + " using " + strategy.to_s) do
        begin
          ENV['iterations'].to_i.times do |count|
         find(strategy => locator)
          end
        rescue Selenium::WebDriver::Error::NoSuchElementError => error
          puts "( 0.0 )"
        end
      end
    end
      end
    end
  end

end

Results

NOTE: The output is in seconds, and the results are for the total run time of 100 executions.

In Table Form:

css_xpath_under_microscopev2

In Chart Form:

  • Chrome:

chart-chrome

  • Firefox:

chart-firefox

  • Internet Explorer 8:

chart-ie8

  • Internet Explorer 9:

chart-ie9

  • Internet Explorer 10:

chart-ie10

  • Opera:

chart-opera


Analyzing the Results

  • Chrome and Firefox are clearly tuned for faster cssSelector performance.
  • Internet Explorer 8 is a grab bag of cssSelector that won't work, an out of control XPath traversal that takes ~65 seconds, and a 38 second table traversal with no cssSelector result to compare it against.
  • In IE 9 and 10, XPath is faster overall. In Safari, it's a toss up, except for a couple of slower traversal runs with XPath. And across almost all browsers, the nested sibling traversal and table cell traversal done with XPath are an expensive operation.
  • These shouldn't be that surprising since the locators are brittle and inefficient and we need to avoid them.

Summary

  • Overall there are two circumstances where XPath is markedly slower than cssSelector. But they are easily avoidable.
  • The performance difference is slightly in favor of for non-IE browsers and slightly in favor of for IE browsers.

Trivia

You can perform the bench-marking on your own, using this library where Dave Haeffner wrapped up all the code.

Examples related to selenium

SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 81 session not created: This version of ChromeDriver only supports Chrome version 74 error with ChromeDriver Chrome using Selenium Selenium: WebDriverException:Chrome failed to start: crashed as google-chrome is no longer running so ChromeDriver is assuming that Chrome has crashed WebDriverException: unknown error: DevToolsActivePort file doesn't exist while trying to initiate Chrome Browser Class has been compiled by a more recent version of the Java Environment How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium? How to make Firefox headless programmatically in Selenium with Python? element not interactable exception in selenium web automation Selenium Web Driver & Java. Element is not clickable at point (x, y). Other element would receive the click How do you fix the "element not interactable" exception?

Examples related to selenium-webdriver

SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 81 Selenium: WebDriverException:Chrome failed to start: crashed as google-chrome is no longer running so ChromeDriver is assuming that Chrome has crashed WebDriverException: unknown error: DevToolsActivePort file doesn't exist while trying to initiate Chrome Browser How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium? How to make Firefox headless programmatically in Selenium with Python? Selenium Web Driver & Java. Element is not clickable at point (x, y). Other element would receive the click How do you fix the "element not interactable" exception? Scrolling to element using webdriver? Only local connections are allowed Chrome and Selenium webdriver Check if element is clickable in Selenium Java

Examples related to xpath

Xpath: select div that contains class AND whose specific child element contains text XPath: difference between dot and text() How to set "value" to input web element using selenium? How to click a href link using Selenium XPath: Get parent node from child node What is the difference between absolute and relative xpaths? Which is preferred in Selenium automation testing? How to use XPath preceding-sibling correctly Selenium and xPath - locating a link by containing text How to verify an XPath expression in Chrome Developers tool or Firefox's Firebug? Concatenate multiple node values in xpath

Examples related to css-selectors

Angular 2: How to style host element of the component? How to click a href link using Selenium How do I apply a style to all children of an element How to write a CSS hack for IE 11? :last-child not working as expected? Need to find element in selenium by css jQuery select child element by class with unknown path How do I select an element that has a certain class? What is the difference between cssSelector & Xpath and which is better with respect to performance for cross browser testing? What is the mouse down selector in CSS?

Examples related to webdriver

WebDriverException: unknown error: DevToolsActivePort file doesn't exist while trying to initiate Chrome Browser Selenium Web Driver & Java. Element is not clickable at point (x, y). Other element would receive the click How do you fix the "element not interactable" exception? ImportError: No module named 'selenium' how to use List<WebElement> webdriver How to get attribute of element from Selenium? Selenium Finding elements by class name in python Open web in new tab Selenium + Python When running WebDriver with Chrome browser, getting message, "Only local connections are allowed" even though browser launches properly How can I start InternetExplorerDriver using Selenium WebDriver