[javascript] How do I Geocode 20 addresses without receiving an OVER_QUERY_LIMIT response?

Using the Google Geocoder v3, if I try to geocode 20 addresses, I get an OVER_QUERY_LIMIT unless I time them to be ~1 second apart, but then it takes 20 seconds before my markers are all placed.

Is there any other way to do it, other than storing the coordinates in advance?

Unfortunately this is a restriction of the Google maps service.

I am currently working on an application using the geocoding feature, and I'm saving each unique address on a per-user basis. I generate the address information (city, street, state, etc) based on the information returned by Google maps, and then save the lat/long information in the database as well. This prevents you from having to re-code things, and gives you nicely formatted addresses.

Another reason you want to do this is because there is a daily limit on the number of addresses that can be geocoded from a particular IP address. You don't want your application to fail for a person for that reason.

I'm facing the same problem trying to geocode 140 addresses.

My workaround was adding usleep(100000) for each loop of next geocoding request. If status of the request is OVER_QUERY_LIMIT, the usleep is increased by 50000 and request is repeated, and so on.

And of cause all received data (lat/long) are stored in XML file not to run request every time the page is loading.

I have just tested Google Geocoder and got the same problem as you have. I noticed I only get the OVER_QUERY_LIMIT status once every 12 requests So I wait for 1 second (that's the minimum delay to wait) It slows down the application but less than waiting 1 second every request

info = getInfos(getLatLng(code)); //In here I call Google API
record(code, info);
generated++; 
if(generated%interval == 0) {
holdOn(delay); // Every x requests, I sleep for 1 second
}

With the basic holdOn method :

private void holdOn(long delay) {
        try {
            Thread.sleep(delay);
        } catch (InterruptedException ex) {
            // ignore
        }
    }

Hope it helps

You actually do not have to wait a full second for each request. I found that if I wait 200 miliseconds between each request I am able to avoid the OVER_QUERY_LIMIT response and the user experience is passable. With this solution you can load 20 items in 4 seconds.

$(items).each(function(i, item){

  setTimeout(function(){

    geoLocate("my address", function(myLatlng){
      ...
    });

  }, 200 * i);

}

EDIT:

Forgot to say that this solution is in pure js, the only thing you need is a browser that supports promises https://developer.mozilla.org/it/docs/Web/JavaScript/Reference/Global_Objects/Promise

For those who still needs to accomplish such, I've written my own solution that combines promises with timeouts.

Code:

/*
    class: Geolocalizer
        - Handles location triangulation and calculations.
        -- Returns various prototypes to fetch position from strings or coords or dragons or whatever.
*/

var Geolocalizer = function () {
    this.queue          = [];     // queue handler..
    this.resolved       = [];
    this.geolocalizer = new google.maps.Geocoder();  
};

Geolocalizer.prototype = {
    /*
        @fn: Localize
        @scope: resolve single or multiple queued requests.
        @params: <array> needles
        @returns: <deferred> object
    */
    Localize: function ( needles ) {
        var that = this;
        // Enqueue the needles.
        for ( var i = 0; i < needles.length; i++ ) {
            this.queue.push(needles[i]);
        }
        // return a promise and resolve it after every element have been fetched (either with success or failure), then reset the queue.
        return new Promise (
            function (resolve, reject) {
                that.resolveQueueElements().then(function(resolved){
                  resolve(resolved);
                  that.queue    = [];
                  that.resolved = [];
                });
            }
        );
    },

    /*
        @fn: resolveQueueElements
        @scope: resolve queue elements.
        @returns: <deferred> object (promise)
    */

    resolveQueueElements: function (callback) {
        var that = this;
        return new Promise(
            function(resolve, reject) {
                // Loop the queue and resolve each element.
                // Prevent QUERY_LIMIT by delaying actions by one second.
                (function loopWithDelay(such, queue, i){
                    console.log("Attempting the resolution of " +queue[i-1]);
                    setTimeout(function(){
                        such.find(queue[i-1], function(res){
                           such.resolved.push(res); 
                        });
                        if (--i) {
                            loopWithDelay(such,queue,i);
                        }
                    }, 1000);
                })(that, that.queue, that.queue.length);

                // Check every second if the queue has been cleared.
                var it = setInterval(function(){
                    if (that.queue.length == that.resolved.length) {
                        resolve(that.resolved);
                        clearInterval(it);
                    }
                }, 1000);
            }
        );
    },

    /*
        @fn: find
        @scope: resolve an address from string
        @params: <string> s, <fn> Callback
    */
    find: function (s, callback) {
        this.geolocalizer.geocode({
            "address": s
        }, function(res, status){
           if (status == google.maps.GeocoderStatus.OK) {
               var r = {
                   originalString:  s,
                   lat: res[0].geometry.location.lat(),
                   lng: res[0].geometry.location.lng()
               };
               callback(r);
           }
            else {
                callback(undefined);
                console.log(status);
                console.log("could not locate " + s);
            }
        });
    }
};

Please note that it's just a part of a bigger library I wrote to handle google maps stuff, hence comments may be confusing.

Usage is quite simple, the approach, however, is slightly different: instead of looping and resolving one address at a time, you will need to pass an array of addresses to the class and it will handle the search by itself, returning a promise which, when resolved, returns an array containing all the resolved (and unresolved) address.

Example:

var myAmazingGeo = new Geolocalizer();
var locations = ["Italy","California","Dragons are thugs...","China","Georgia"];
myAmazingGeo.Localize(locations).then(function(res){ 
   console.log(res); 
});

Console output:

Attempting the resolution of Georgia
Attempting the resolution of China
Attempting the resolution of Dragons are thugs...
Attempting the resolution of California
ZERO_RESULTS
could not locate Dragons are thugs...
Attempting the resolution of Italy

Object returned:

The whole magic happens here:

(function loopWithDelay(such, queue, i){
                    console.log("Attempting the resolution of " +queue[i-1]);
                    setTimeout(function(){
                        such.find(queue[i-1], function(res){
                           such.resolved.push(res); 
                        });
                        if (--i) {
                            loopWithDelay(such,queue,i);
                    }
                }, 750);
            })(that, that.queue, that.queue.length);

Basically, it loops every item with a delay of 750 milliseconds between each of them, hence every 750 milliseconds an address is controlled.

I've made some further testings and I've found out that even at 700 milliseconds I was sometimes getting the QUERY_LIMIT error, while with 750 I haven't had any issue at all.

In any case, feel free to edit the 750 above if you feel you are safe by handling a lower delay.

Hope this helps someone in the near future ;)