[selenium] Clicking at coordinates without identifying element

As part of my Selenium test for a login function, I would like to click a button by identifying its coordinates and instructing Selenium to click at those coordinates. This would be done without identifying the element itself (via id, xpath, etc).

I understand there are other more efficient ways to run a click command, but I'm looking to specifically use this approach to best match the user experience. Thanks.

import pyautogui
from selenium import webdriver

driver = webdriver.Chrome(chrome_options=options)
driver.maximize_window() #maximize the browser window
driver.implicitly_wait(30)
driver.get(url)
height=driver.get_window_size()['height']

#get browser navigation panel height
browser_navigation_panel_height = driver.execute_script('return window.outerHeight - window.innerHeight;')

act_y=y%height
scroll_Y=y/height

#scroll down page until y_off is visible
try:
    driver.execute_script("window.scrollTo(0, "+str(scroll_Y*height)+")")
except Exception as e:
    print "Exception"
#pyautogui used to generate click by passing x,y coordinates
pyautogui.FAILSAFE=False
pyautogui.moveTo(x,act_y+browser_navigation_panel_height)
pyautogui.click(x,act_y+browser_navigation_panel_height,clicks=1,interval=0.0,button="left")

This is worked for me. Hope, It will work for you guys :)...

This worked for me in Java for clicking on coordinates irrespective on any elements.

Actions actions = new Actions(driver);
actions.moveToElement(driver.findElement(By.tagName("body")), 0, 0);
actions.moveByOffset(xCoordinate, yCoordinate).click().build().perform();

Second line of code will reset your cursor to the top left corner of the browser view and last line will click on the x,y coordinates provided as parameter.

I first used the JavaScript code, it worked amazingly until a website did not click.

So I've found this solution:

First, import ActionChains for Python & active it:

from selenium.webdriver.common.action_chains import ActionChains
actions = ActionChains(driver)

To click on a specific point in your sessions use this:

actions.move_by_offset(X coordinates, Y coordinates).click().perform()

NOTE: The code above will only work if the mouse has not been touched, to reset the mouse coordinates use this:

actions.move_to_element_with_offset(driver.find_element_by_tag_name('body'), 0,0))

In Full:

actions.move_to_element_with_offset(driver.find_element_by_tag_name('body'), 0,0)
actions.move_by_offset(X coordinates, Y coordinates).click().perform()

If using a commercial add-on to Selenium is an option for you, this is possible: Suppose your button is at coordinates x=123, y=456. Then you can use Helium to click on the element at these coordinates as follows:

from helium.api import *
# Tell Helium about your WebDriver instance:
set_driver(driver)
click(Point(123, 456))

(I am one of Helium's authors.)

In C# API you use actions

var element = driver.FindElement(By...);
new Actions(driver).moveToElement(element).moveByOffset(dx, dy).click().perform();

Although it is best to just use simple Id, CSS, Xpath selectors when possible. But the functionality is there when needed (i.e. clicking elements in certain geographic places for functionality).

This can be done using Actions class in java

Use following code -

new Actions(driver).moveByOffset(x coordinate, y coordinate).click().build().perform(); 

Note: Selenium 3 doesn't support Actions class for geckodriver

Also, note that x and y co-ordinates are relative values from current mouse position. Assuming mouse co-ordinates are at (0,0) to start with, if you want to use absolute values, you can perform the below action immediately after you clicked on it using the above code.

new Actions(driver).moveByOffset(-x coordinate, -y coordinate).perform();

There is a way to do this. Using the ActionChains API you can move the mouse over a containing element, adjust by some offset (relative to the middle of that element) to put the "cursor" over the desired button (or other element), and then click at that location. Here's how to do it using webdriver in Python:

from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
browser = webdriver.Chrome()
elem = browser.find_element_by_selector(".some > selector")
ac = ActionChains(browser)
ac.move_to_element(elem).move_by_offset(x_offset, y_offset).click().perform()

Y'all are much to quick to dismiss the question. There are a number of reasons one might to need to click at a specific location, rather than on an element. In my case I have an SVG bar chart with an overlay element that catches all the clicks. I want to simulate a click over one of the bars, but since the overlay is there Selenium can't click on the element itself. This technique would also be valuable for imagemaps.

    WebElement button = driver.findElement(By.xpath("//button[@type='submit']"));
    int height = button.getSize().getHeight();
    int width = button.getSize().getWidth();
    Actions act = new Actions(driver);
    act.moveToElement(button).moveByOffset((width/2)+2,(height/2)+2).click();

Action chains can be a little finicky. You could also achieve this by executing javascript.

self.driver.execute_script('el = document.elementFromPoint(440, 120); el.click();')

In Selenium Java, you can try it using Javascript:

WebDriver driver = new ChromeDriver();_x000D_
_x000D_
if (driver instanceof JavascriptExecutor) {_x000D_
 ((JavascriptExecutor) driver).executeScript("el = document.elementFromPoint(x-cordinate, y-cordinate); el.click();");_x000D_
}

I used the Actions Class like many listed above, but what I found helpful was if I need find a relative position from the element I used Firefox Add-On Measurit to get the relative coordinates. For example:

        IWebDriver driver = new FirefoxDriver();
        driver.Url = @"https://scm.commerceinterface.com/accounts/login/?next=/remittance_center/";
        var target = driver.FindElement(By.Id("loginAsEU"));
        Actions builder = new Actions(driver);            
        builder.MoveToElement(target , -375  , -436).Click().Build().Perform();

I got the -375, -436 from clicking on an element and then dragging backwards until I reached the point I needed to click. The coordinates that MeasureIT said I just subtracted. In my example above, the only element I had on the page that was clickable was the "loginAsEu" link. So I started from there.

I used AutoIt to do it.

using AutoIt;
AutoItX.MouseClick("LEFT",150,150,1,0);//1: click once, 0: Move instantaneous

1. Pro:
   • simple
   • regardless of mouse movement
2. Con:
   • since coordinate is screen-based, there should be some caution if the app scales.
   • the drive won't know when the app finish with clicking consequence actions. There should be a waiting period.

If you can see the source code of page, its always the best option to refer to the button by its id or NAME attribute. For example you have button "Login" looking like this:

 <input type="submit" name="login" id="login" />

In that case is best way to do

selenium.click(id="login");

Just out of the curiosity - isnt that HTTP basic authentification? In that case maybe look at this: http://code.google.com/p/selenium/issues/detail?id=34