Get HTML source of WebElement in Selenium WebDriver using Python

Question

I m using the Python bindings to run Selenium WebDriver  from selenium import webdriver wd   webdriver Firefox    I know I can grab a webelement like so  elem   wd find element by css selector   my-id    And I know I can get the full page source with    wd page source  But is there a way to get the  quot element source quot   elem source      lt -- returns the HTML as a string  The Selenium WebDriver documentation for Python are basically non-existent and I don t see anything in the code that seems to enable that functionality  What is the best way to access the HTML of an element  and its children

User · Answer

This works seamlessly for me   element get attribute  innerHTML

User · Answer

Java with Selenium 2 53 0  driver getPageSource

User · Answer

In Ruby  using selenium-webdriver  2 32 1   there is a page source method that contains the entire page source

User · Answer

And in PHPUnit Selenium test it s like this   text    this- gt byCssSelector   some-class-nmae  - gt attribute  innerHTML

User · Answer

Using the attribute method is  in fact  easier and more straightforward  Using Ruby with the Selenium and PageObject gems  to get the class associated with a certain element  the line would be element attribute Class   The same concept applies if you wanted to get other attributes tied to the element  For example  if I wanted the string of an element  element attribute String

User · Answer

You can read the innerHTML attribute to get the source of the content of the element or outerHTML for the source with the current element  Python  element get attribute  innerHTML    Java  elem getAttribute  quot innerHTML quot     C   element GetAttribute  quot innerHTML quot     Ruby  element attribute  quot innerHTML quot    JavaScript  element getAttribute  innerHTML     PHP   element- gt getAttribute  innerHTML     It was tested and worked with the ChromeDriver

User · Answer

There is not really a straightforward way of getting the HTML source code of a webelement  You will have to use JavaScript  I am not too sure about python bindings  but you can easily do like this in Java  I am sure there must be something similar to JavascriptExecutor class in Python   WebElement element   driver findElement By id  quot foo quot      String contents    String   JavascriptExecutor driver  executeScript  quot return arguments 0  innerHTML  quot   element

User · Answer

I hope this could help  http   selenium googlecode com svn trunk docs api java org openqa selenium WebElement html  Here is described Java method   java lang String    getText      But unfortunately it s not available in Python  So you can translate the method names to Python from Java and try another logic using present methods without getting the whole page source     E g    my id   elem 0  get attribute  my-id

User · Answer

InnerHTML will return the element inside the selected element and outerHTML will return the inside HTML along with the element you have selected Example  Now suppose your Element is as below  lt tr id  quot myRow quot  gt  lt td gt A lt  td gt  lt td gt B lt  td gt  lt  tr gt   innerHTML element output  lt td gt A lt  td gt  lt td gt B lt  td gt   outerHTML element output  lt tr id  quot myRow quot  gt  lt td gt A lt  td gt  lt td gt B lt  td gt  lt  tr gt   Live Example  http   www java2s com Tutorials JavascriptDemo f find out the difference between innerhtml and outerhtml in javascript example htm Below you will find the syntax which require as per different binding  Change the innerHTML to outerHTML as per required  Python  element get attribute  innerHTML    Java  elem getAttribute  quot innerHTML quot     If you want whole page HTML  use the below code  driver getPageSource

User · Answer

If you are interested in a solution for Selenium Remote Control in Python  here is how to get innerHTML  innerHTML   sel get eval  quot window document getElementById  prodid   innerHTML quot

User · Answer

It looks outdated  but let it be here anyway  The correct way to do it in your case  elem   wd find element by css selector   my-id   html   wd execute script  quot return arguments 0  innerHTML  quot   elem   or html   elem get attribute  innerHTML    Both are working for me  selenium-server-standalone-2 35 0

User · Answer

WebElement element   driver findElement By id  quot foo quot     String contents    String   JavascriptExecutor driver  executeScript  quot return arguments 0  innerHTML  quot   element     This code really works to get JavaScript from source as well

User · Answer

Sure we can get all HTML source code with this script below in Selenium Python   elem   driver find element by xpath        source code   elem get attribute  outerHTML     If you you want to save it to file   with open  c  html source code html    w   as f      f write source code encode  utf-8      I suggest saving to a file because source code is very very long

User · Answer

The method to get the rendered HTML I prefer is the following  driver get  quot http   www google com quot   body html   driver find element by xpath  quot  html body quot   print body html text  However  the above method removes all the tags  yes  the nested tags as well  and returns only text content  If you interested in getting the HTML markup as well  then use the method below  print body html getAttribute  quot innerHTML quot

User · Answer

The other answers provide a lot of details about retrieving the markup of a WebElement  However  an important aspect is  modern websites are increasingly implementing JavaScript  ReactJS  jQuery  Ajax  Vue js  Ember js  GWT  etc  to render the dynamic elements within the DOM tree  Hence there is a necessity to wait for the element and its children to completely render before retrieving the markup   Python Hence  ideally you need to induce WebDriverWait for the visibility of element located   and you can use either of the following Locator Strategies   Using get attribute  quot outerHTML quot    element   WebDriverWait driver  20  until EC visibility of element located  By CSS SELECTOR   quot  my-id quot     print element get attribute  quot outerHTML quot      Using execute script    element   WebDriverWait driver  20  until EC visibility of element located  By CSS SELECTOR   quot  my-id quot     print driver execute script  quot return arguments 0  outerHTML  quot   element     Note  You have to add the following imports  from selenium webdriver support ui import WebDriverWait from selenium webdriver common by import By from selenium webdriver support import expected conditions as EC

[python] Get HTML source of WebElement in Selenium WebDriver using Python

Examples related to python

Examples related to selenium

Examples related to selenium-webdriver

Examples related to webdriver

Examples related to automated-tests