Selenium WebDriver Wait for complex page with JavaScript to load

Question

I have a web application to test with Selenium  There is a lot of JavaScript running on page load  This JavaScript code is not so well written but I can t change anything  So waiting for an element to appear in the DOM with findElement   method is not an option  I want to create a generic function in Java to wait for a page to load  a possible solution would be      run a JavaScript script form WebDriver and store the result of document body innerHTML in a string variable body  compare the body variable to the previous version of body  if they are the same then set increment a counter notChangedCount otherwise set notChangedCount to zero  wait for a litte time  50 ms for example   if the page has not changed for some time  500 ms for example  so notChangedCount  gt   10 then exit the loop otherwise loop to the first step    Do you think it s a valid solution

User · Accepted Answer

If anyone actually knew a general and always-applicable answer, it would have been implemented everywhere ages ago and would make our lives SO much easier.

There are many things you can do, but every single one of them has a problem:

As Ashwin Prabhu said, if you know the script well, you can observe its behaviour and track some of its variables on window or document etc. This solution, however, is not for everyone and can be used only by you and only on a limited set of pages.
Your solution by observing the HTML code and whether it has or hasn't been changed for some time is not bad (also, there is a method to get the original and not-edited HTML directly by WebDriver), but:
- It takes a long time to actually assert a page and could prolong the test significantly.
- You never know what the right interval is. The script might be downloading something big that takes more than 500 ms. There are several scripts on our company's internal page that take several seconds in IE. Your computer may be temporarily short on resources - say that an antivirus will make your CPU work fully, then 500 ms may be too short even for a noncomplex scripts.
- Some scripts are never done. They call themselves with some delay (setTimeout()) and work again and again and could possibly change the HTML every time they run. Seriously, every "Web 2.0" page does it. Even Stack Overflow. You could overwrite the most common methods used and consider the scripts that use them as completed, but ... you can't be sure.
- What if the script does something other than changing the HTML? It could do thousands of things, not just some innerHTML fun.
There are tools to help you on this. Namely Progress Listeners together with nsIWebProgressListener and some others. The browser support for this, however, is horrible. Firefox began to try to support it from FF4 onwards (still evolving), IE has basic support in IE9.

And I guess I could come up with another flawed solution soon. The fact is - there's no definite answer on when to say "now the page is complete" because of the everlasting scripts doing their work. Pick the one that serves you best, but beware of its shortcomings.

User · Answer

I had a same issue   This solution works for me from WebDriverDoku   WebDriverWait wait   new WebDriverWait driver  10   WebElement element   wait until ExpectedConditions elementToBeClickable By id  someid        http   www seleniumhq org docs 04 webdriver advanced jsp

User · Answer

Don t know how to do that but in my case  end of page load  amp  rendering match with FAVICON displayed in Firefox tab   So if we can get the favicon image in the webbrowser  the web page is fully loaded   But how perform this

User · Answer

You need to wait for Javascript and jQuery to finish loading  Execute Javascript to check if jQuery active is 0 and document readyState is complete  which means the JS and jQuery load is complete   public boolean waitForJStoLoad          WebDriverWait wait   new WebDriverWait driver  30           wait for jQuery to load     ExpectedCondition lt Boolean gt  jQueryLoad   new ExpectedCondition lt Boolean gt             Override       public Boolean apply WebDriver driver            try             return   Long executeJavaScript  return jQuery active      0                     catch  Exception e              return true                                   wait for Javascript to load     ExpectedCondition lt Boolean gt  jsLoad   new ExpectedCondition lt Boolean gt             Override       public Boolean apply WebDriver driver            return executeJavaScript  return document readyState                toString   equals  complete                      return wait until jQueryLoad   amp  amp  wait until jsLoad

User · Answer

The below code works perfectly in my case - my page contains complex java scripts  public void checkPageIsReady        JavascriptExecutor js    JavascriptExecutor driver        Initially bellow given if condition will check ready state of page    if  js executeScript  return document readyState   toString   equals  complete         System out println  Page Is loaded        return             This loop will rotate for 25 times to check If page Is ready after every 1 second      You can replace your value with 25 If you wants to Increase or decrease wait time    for  int i 0  i lt 25  i         try       Thread sleep 1000        catch  InterruptedException e           To check page ready state     if  js executeScript  return document readyState   toString   equals  complete          break                    Source - How To Wait For Page To Load Ready In Selenium WebDriver

User · Answer

Thanks Ashwin    In my case I should need wait for a jquery plugin execution in some element   specifically  qtip    based in your hint  it worked perfectly for me    wait until  new Predicate lt WebDriver gt                  public boolean apply WebDriver driver                    return   JavascriptExecutor driver  executeScript  return document readyState   equals  complete                                     Note  I m using Webdriver 2

User · Answer

Two conditions can be used to check if the page is loaded before finding any element on the page   WebDriverWait wait   new WebDriverWait driver  50     Using below readyState will wait till page load  wait until  ExpectedCondition lt Boolean gt   wd - gt       JavascriptExecutor  wd  executeScript  return document readyState   equals  complete       Below JQuery will wait till data has not been loaded     int count  0              if  Boolean  executor executeScript  return window jQuery    undefined                     while   Boolean  executor executeScript  return jQuery active    0                         Thread sleep 4000                       if count gt 4                          break                      count                                      After these JavaScriptCode try to findOut webElement   WebElement we   wait until ExpectedConditions presenceOfElementLocated by

User · Answer

If all you need to do is wait for the html on the page to become stable before trying to interact with elements  you can poll the DOM periodically and compare the results  if the DOMs are the same within the given poll time  you re golden  Something like this where you pass in the maximum wait time and the time between page polls before comparing  Simple and effective   public void waitForJavascript int maxWaitMillis  int pollDelimiter        double startTime   System currentTimeMillis        while  System currentTimeMillis    lt  startTime   maxWaitMillis            String prevState   webDriver getPageSource            Thread sleep pollDelimiter       lt -- would need to wrap in a try catch         if  prevState equals webDriver getPageSource                   return

User · Answer

Does the JS library define initialize any well known variable on the window   If so you could wait for the variable to appear  You can use    JavascriptExecutor driver  executeScript String script  Object    args   to test for this condition  something like  window SomeClass  amp  amp  window SomeClass variable    null  and return a boolean true   false   Wrap this in a WebDriverWait  and wait until the script returns true

User · Answer

Using implicit wait works for me   driver Manage   Timeouts   ImplicitWait   TimeSpan FromSeconds 10     Refer to this answer Selenium c  Webdriver  Wait Until Element is Present

User · Answer

To do it properly  you need to handle the exceptions   Here is how I do a wait for an iFrame   This requires that your JUnit test class pass the instance of RemoteWebDriver into the page object    public class IFrame1 extends LoadableComponent lt IFrame1 gt         private RemoteWebDriver driver        FindBy id    iFrame1TextFieldTestInputControlID        public WebElement iFrame1TextFieldInput        FindBy id    iFrame1TextFieldTestProcessButtonID        public WebElement copyButton       public IFrame1  RemoteWebDriver drv             super            this driver   drv          this driver switchTo   defaultContent            waitTimer 1  1000           this driver switchTo   frame  BodyFrame1            LOGGER info  IFrame1 constructor                   Override     protected void isLoaded   throws Error                   LOGGER info  IFrame1 isLoaded                 PageFactory initElements  driver  this            try               assertTrue   Page visible title is not yet available    driver       findElementByCssSelector  body form webDriverUnitiFrame1TestFormID h1                        getText   equals  iFrame1 Test                catch   NoSuchElementException e                LOGGER info  No such element                  assertTrue  No such element    false                         Override     protected void load             LOGGER info  IFrame1 load                 Wait lt WebDriver gt  wait   new FluentWait lt WebDriver gt   driver                    withTimeout 30  TimeUnit SECONDS                   pollingEvery 5  TimeUnit SECONDS                   ignoring  NoSuchElementException class                     ignoring  StaleElementReferenceException class                 wait until  ExpectedConditions presenceOfElementLocated               By cssSelector  body form webDriverUnitiFrame1TestFormID h1                     NOTE  You can see my entire working example here

User · Answer

I asked my developers to create a JavaScript variable  isProcessing  that I can access  in the  ae  object  that they set when things start running and clear when things are done  I then run it in an accumulator that checks it every 100 ms until it gets five in a row for a total of 500 ms without any changes  If 30 seconds pass  I throw an exception because something should have happened by then  This is in C    public static void WaitForDocumentReady this IWebDriver driver        Console WriteLine  Waiting for five instances of document readyState returning  complete  at 100ms intervals         IJavaScriptExecutor jse    IJavaScriptExecutor driver      int i   0     Count of  document readyState     complete   amp  amp   ae isProcessing     false      int j   0     Count of iterations in the while   loop      int k   0     Count of times i was reset to 0      bool readyState   false      while  i  lt  5                System Threading Thread Sleep 100           readyState    bool jse ExecuteScript  return   document readyState      complete    amp  amp   ae isProcessing     false              if  readyState    i              else                       i   0              k                      j            if  j  gt  300    throw new TimeoutException  Timeout waiting for document readyState to be complete                 j    100      Console WriteLine  Waited     j ToString       milliseconds  There were     k     resets

User · Answer

I like your idea of polling the HTML until it s stable  I may add that to my own solution  The following approach is in C  and requires jQuery   I m the developer for a SuccessFactors  SaaS  test project where we have no influence at all over the developers or the characteristics of the DOM behind the web page  The SaaS product can potentially change its underlying DOM design 4 times a year  so the hunt is permanently on for robust  performant ways to test with Selenium  including NOT testing with Selenium where possi ble    Here s what I use for  page ready   It works in all my own tests currently  The same approach also worked for a big in-house Java web app a couple of years ago  and had been robust for over a year at the time I left the project    Driver is the WebDriver instance that communicates with the browser DefaultPageLoadTimeout is a timeout value in ticks  100ns per tick      public IWebDriver Driver   get  private set             const int GlobalPageLoadTimeOutSecs   10  static readonly TimeSpan DefaultPageLoadTimeout       new TimeSpan  long   10 000 000   GlobalPageLoadTimeOutSecs    Driver   new FirefoxDriver      In what follows  note the order of waits in method PageReady  Selenium document ready  Ajax  animations   which makes sense if you think about it    load the page containing the code use the code to load the data from somewhere via Ajax present the data  possibly with animations   Something like your DOM comparison approach could be used between 1 and 2 to add another layer of robustness     public void PageReady         DocumentReady        AjaxReady        AnimationsReady        private void DocumentReady         WaitForJavascript script   return document readyState   result   complete       private void WaitForJavascript string script  string result        new WebDriverWait Driver  DefaultPageLoadTimeout  Until          d   gt    IJavaScriptExecutor  d  ExecuteScript script  Equals result       private void AjaxReady         WaitForJavascript script   return jQuery active toString     result   0       private void AnimationsReady         WaitForJavascript script   return     animated    length toString     result   0

User · Answer

For the nodejs Selenium library  I used the following snippet  In my case  I was looking for two objects that are added to the window  which in this example are  lt SOME PROPERTY gt   10000 is the timeout milliseconds   lt NEXT STEP HERE gt  is what happens after the properties are found on the window   driver wait  driver   gt        return driver executeScript   if window hasOwnProperty  lt SOME PROPERTY gt    amp  amp  window hasOwnProperty  lt SOME PROPERTY gt    return true         10000  then      gt            lt NEXT STEP HERE gt     catch err   gt         console log  looking for window properties   err

User · Answer

You can write some logic to handle this  I have write a method that will return the WebElement and this method will be called three times or you can increase the time and add a null check for WebElement Here is an example  public static void main String   args            WebDriver driver   new FirefoxDriver            driver get  https   www crowdanalytix com  home            WebElement webElement   getWebElement driver   homekkkkkkkkkkkk            int i   1          while  webElement    null  amp  amp  i  lt  4                webElement   getWebElement driver   homessssssssssss                System out println  calling                i                      System out println webElement getTagName             System out println  End            driver close               public static WebElement getWebElement WebDriver driver  String id            WebElement myDynamicElement   null          try               myDynamicElement    new WebDriverWait driver  10                        until ExpectedConditions presenceOfElementLocated By                              id id                 return myDynamicElement            catch  TimeoutException ex                return null

User · Answer

Here s from my own code  Window setTimeout executes only when browser is idle   So calling the function recursively  42 times  will take 100ms if there is no activity in the browser and much more if the browser is busy doing something else       ExpectedCondition lt Boolean gt  javascriptDone   new ExpectedCondition lt Boolean gt              public Boolean apply WebDriver d                try   window setTimeout executes only when browser is idle                    introduces needed wait time when javascript is running in browser                 return    Boolean    JavascriptExecutor  d  executeAsyncScript                              var callback  arguments arguments length - 1                                 var count 42                                setTimeout  collect  0                                function collect                                       if count-- gt 0                                         setTimeout  collect  0                                                                        else  callback                                      true                                                                                                                                                                    catch  Exception e                    return Boolean FALSE                                     WebDriverWait w   new WebDriverWait driver timeOut         w until javascriptDone       w null    As a bonus the counter can be reset on document readyState or on jQuery Ajax calls or if any jQuery animations are running  only if your app uses jQuery for ajax calls            function collect                                       if    typeof jQuery      undefined        jQuery active     0   amp  amp        animated    length     0     amp  amp   document readyState      complete                                         count 42                                      setTimeout  collect  0                                                                        else if count-- gt 0                                         setTimeout  collect  0                                                  EDIT  I notice executeAsyncScript doesn t work well if a new page loads and the test might stop responding indefinetly  better to use this on instead   public static ExpectedCondition lt Boolean gt  documentNotActive final int counter        return new ExpectedCondition lt Boolean gt              boolean resetCount true           Override         public Boolean apply WebDriver d                 if resetCount                     JavascriptExecutor  d  executeScript                              window mssCount   counter    r n                                 window mssJSDelay function mssJSDelay    r n                                     if  typeof jQuery     undefined    amp  amp   jQuery active     0         animated    length     0   r n                                         window mssCount   counter    r n                                     window mssCount-- gt 0  amp  amp  r n                                     setTimeout window mssJSDelay window mssCount 1   r n                                   r n                                 window mssJSDelay                       resetCount false                             boolean ready false              try                  ready -1    Long    JavascriptExecutor  d  executeScript                           if typeof window mssJSDelay    function     r n                                 window mssCount   counter    r n                                 window mssJSDelay function mssJSDelay    r n                                     if  typeof jQuery     undefined    amp  amp   jQuery active     0         animated    length     0   r n                                         window mssCount   counter    r n                                     window mssCount-- gt 0  amp  amp  r n                                     setTimeout window mssJSDelay window mssCount 1   r n                                   r n                                 window mssJSDelay    r n                                r n                              return window mssCount                                catch  NoSuchWindowException a                   a printStackTrace                    return true                            catch  Exception e                    e printStackTrace                    return false                            return ready                     Override         public String toString                 return String format  Timeout waiting for documentNotActive script

User · Answer

Here s how I do it   new WebDriverWait driver  20  until         ExpectedConditions jsReturnsValue                      return document readyState      complete    true   false

[java] Selenium WebDriver: Wait for complex page with JavaScript to load

Examples related to java

Examples related to c#

Examples related to selenium

Examples related to selenium-webdriver

Examples related to webdriver