[mysql] How to set a maximum execution time for a mysql query?

I would like to set a maximum execution time for sql queries like set_time_limit() in php. How can I do ?

This question is related to mysql sql time limit

The answer is


Please rewrite your query like

select /*+ MAX_EXECUTION_TIME(1000) */ * from table


this statement will kill your query after the specified time


I thought it has been around a little longer, but according to this,

MySQL 5.7.4 introduces the ability to set server side execution time limits, specified in milliseconds, for top level read-only SELECT statements.

SELECT 
/*+ MAX_EXECUTION_TIME(1000) */ --in milliseconds
* 
FROM table;

Note that this only works for read-only SELECT statements.

Update: This variable was added in MySQL 5.7.4 and renamed to max_execution_time in MySQL 5.7.8. (source)


If you're using the mysql native driver (common since php 5.3), and the mysqli extension, you can accomplish this with an asynchronous query:

<?php

// Here's an example query that will take a long time to execute.
$sql = "
    select *
    from information_schema.tables t1
    join information_schema.tables t2
    join information_schema.tables t3
    join information_schema.tables t4
    join information_schema.tables t5
    join information_schema.tables t6
    join information_schema.tables t7
    join information_schema.tables t8
";

$mysqli = mysqli_connect('localhost', 'root', '');
$mysqli->query($sql, MYSQLI_ASYNC | MYSQLI_USE_RESULT);
$links = $errors = $reject = [];
$links[] = $mysqli;

// wait up to 1.5 seconds
$seconds = 1;
$microseconds = 500000;

$timeStart = microtime(true);

if (mysqli_poll($links, $errors, $reject, $seconds, $microseconds) > 0) {
    echo "query finished executing. now we start fetching the data rows over the network...\n";
    $result = $mysqli->reap_async_query();
    if ($result) {
        while ($row = $result->fetch_row()) {
            // print_r($row);
            if (microtime(true) - $timeStart > 1.5) {
                // we exceeded our time limit in the middle of fetching our result set.
                echo "timed out while fetching results\n";
                var_dump($mysqli->close());
                break;
            }
        }
    }
} else {
    echo "timed out while waiting for query to execute\n";
    var_dump($mysqli->close());
}

The flags I'm giving to mysqli_query accomplish important things. It tells the client driver to enable asynchronous mode, while forces us to use more verbose code, but lets us use a timeout(and also issue concurrent queries if you want!). The other flag tells the client not to buffer the entire result set into memory.

By default, php configures its mysql client libraries to fetch the entire result set of your query into memory before it lets your php code start accessing rows in the result. This can take a long time to transfer a large result. We disable it, otherwise we risk that we might time out while waiting for the buffering to complete.

Note that there's two places where we need to check for exceeding a time limit:

  • The actual query execution
  • while fetching the results(data)

You can accomplish similar in the PDO and regular mysql extension. They don't support asynchronous queries, so you can't set a timeout on the query execution time. However, they do support unbuffered result sets, and so you can at least implement a timeout on the fetching of the data.

For many queries, mysql is able to start streaming the results to you almost immediately, and so unbuffered queries alone will allow you to somewhat effectively implement timeouts on certain queries. For example, a

select * from tbl_with_1billion_rows

can start streaming rows right away, but,

select sum(foo) from tbl_with_1billion_rows

needs to process the entire table before it can start returning the first row to you. This latter case is where the timeout on an asynchronous query will save you. It will also save you from plain old deadlocks and other stuff.

ps - I didn't include any timeout logic on the connection itself.


You can find the answer on this other S.O. question:

MySQL - can I limit the maximum time allowed for a query to run?

a cron job that runs every second on your database server, connecting and doing something like this:

  • SHOW PROCESSLIST
  • Find all connections with a query time larger than your maximum desired time
  • Run KILL [process id] for each of those processes

I thought it has been around a little longer, but according to this,

MySQL 5.7.4 introduces the ability to set server side execution time limits, specified in milliseconds, for top level read-only SELECT statements.

SELECT 
/*+ MAX_EXECUTION_TIME(1000) */ --in milliseconds
* 
FROM table;

Note that this only works for read-only SELECT statements.

Update: This variable was added in MySQL 5.7.4 and renamed to max_execution_time in MySQL 5.7.8. (source)


You can find the answer on this other S.O. question:

MySQL - can I limit the maximum time allowed for a query to run?

a cron job that runs every second on your database server, connecting and doing something like this:

  • SHOW PROCESSLIST
  • Find all connections with a query time larger than your maximum desired time
  • Run KILL [process id] for each of those processes

Please rewrite your query like

select /*+ MAX_EXECUTION_TIME(1000) */ * from table


this statement will kill your query after the specified time


pt_kill has an option for such. But it is on-demand, not continually monitoring. It does what @Rafa suggested. However see --sentinel for a hint of how to come close with cron.


If you're using the mysql native driver (common since php 5.3), and the mysqli extension, you can accomplish this with an asynchronous query:

<?php

// Here's an example query that will take a long time to execute.
$sql = "
    select *
    from information_schema.tables t1
    join information_schema.tables t2
    join information_schema.tables t3
    join information_schema.tables t4
    join information_schema.tables t5
    join information_schema.tables t6
    join information_schema.tables t7
    join information_schema.tables t8
";

$mysqli = mysqli_connect('localhost', 'root', '');
$mysqli->query($sql, MYSQLI_ASYNC | MYSQLI_USE_RESULT);
$links = $errors = $reject = [];
$links[] = $mysqli;

// wait up to 1.5 seconds
$seconds = 1;
$microseconds = 500000;

$timeStart = microtime(true);

if (mysqli_poll($links, $errors, $reject, $seconds, $microseconds) > 0) {
    echo "query finished executing. now we start fetching the data rows over the network...\n";
    $result = $mysqli->reap_async_query();
    if ($result) {
        while ($row = $result->fetch_row()) {
            // print_r($row);
            if (microtime(true) - $timeStart > 1.5) {
                // we exceeded our time limit in the middle of fetching our result set.
                echo "timed out while fetching results\n";
                var_dump($mysqli->close());
                break;
            }
        }
    }
} else {
    echo "timed out while waiting for query to execute\n";
    var_dump($mysqli->close());
}

The flags I'm giving to mysqli_query accomplish important things. It tells the client driver to enable asynchronous mode, while forces us to use more verbose code, but lets us use a timeout(and also issue concurrent queries if you want!). The other flag tells the client not to buffer the entire result set into memory.

By default, php configures its mysql client libraries to fetch the entire result set of your query into memory before it lets your php code start accessing rows in the result. This can take a long time to transfer a large result. We disable it, otherwise we risk that we might time out while waiting for the buffering to complete.

Note that there's two places where we need to check for exceeding a time limit:

  • The actual query execution
  • while fetching the results(data)

You can accomplish similar in the PDO and regular mysql extension. They don't support asynchronous queries, so you can't set a timeout on the query execution time. However, they do support unbuffered result sets, and so you can at least implement a timeout on the fetching of the data.

For many queries, mysql is able to start streaming the results to you almost immediately, and so unbuffered queries alone will allow you to somewhat effectively implement timeouts on certain queries. For example, a

select * from tbl_with_1billion_rows

can start streaming rows right away, but,

select sum(foo) from tbl_with_1billion_rows

needs to process the entire table before it can start returning the first row to you. This latter case is where the timeout on an asynchronous query will save you. It will also save you from plain old deadlocks and other stuff.

ps - I didn't include any timeout logic on the connection itself.


Examples related to mysql

Implement specialization in ER diagram How to post query parameters with Axios? PHP with MySQL 8.0+ error: The server requested authentication method unknown to the client Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver' phpMyAdmin - Error > Incorrect format parameter? Authentication plugin 'caching_sha2_password' is not supported How to resolve Unable to load authentication plugin 'caching_sha2_password' issue Connection Java-MySql : Public Key Retrieval is not allowed How to grant all privileges to root user in MySQL 8.0 MySQL 8.0 - Client does not support authentication protocol requested by server; consider upgrading MySQL client

Examples related to sql

Passing multiple values for same variable in stored procedure SQL permissions for roles Generic XSLT Search and Replace template Access And/Or exclusions Pyspark: Filter dataframe based on multiple conditions Subtracting 1 day from a timestamp date PYODBC--Data source name not found and no default driver specified select rows in sql with latest date for each ID repeated multiple times ALTER TABLE DROP COLUMN failed because one or more objects access this column Create Local SQL Server database

Examples related to time

Date to milliseconds and back to date in Swift How to manage Angular2 "expression has changed after it was checked" exception when a component property depends on current datetime how to sort pandas dataframe from one column Convert time.Time to string How to get current time in python and break up into year, month, day, hour, minute? Xcode swift am/pm time to 24 hour format How to add/subtract time (hours, minutes, etc.) from a Pandas DataFrame.Index whos objects are of type datetime.time? What does this format means T00:00:00.000Z? How can I parse / create a date time stamp formatted with fractional seconds UTC timezone (ISO 8601, RFC 3339) in Swift? Extract time from moment js object

Examples related to limit

Laravel Eloquent limit and offset How to increase Neo4j's maximum file open limit (ulimit) in Ubuntu? Pagination using MySQL LIMIT, OFFSET How to fix the "508 Resource Limit is reached" error in WordPress? How to limit the number of dropzone.js files uploaded? Google drive limit number of download How can I list (ls) the 5 last modified files in a directory? Limiting Python input strings to certain characters and lengths MySQL JOIN with LIMIT 1 on joined table MySQL - UPDATE query with LIMIT