[sql] How to speed up insertion performance in PostgreSQL

I encountered this insertion performance problem as well. My solution is spawn some go routines to finish the insertion work. In the meantime, SetMaxOpenConns should be given a proper number otherwise too many open connection error would be alerted.

db, _ := sql.open() 
db.SetMaxOpenConns(SOME CONFIG INTEGER NUMBER) 
var wg sync.WaitGroup
for _, query := range queries {
    wg.Add(1)
    go func(msg string) {
        defer wg.Done()
        _, err := db.Exec(msg)
        if err != nil {
            fmt.Println(err)
        }
    }(query)
}
wg.Wait()

The loading speed is much faster for my project. This code snippet just gave an idea how it works. Readers should be able to modify it easily.

Examples related to sql

Passing multiple values for same variable in stored procedure SQL permissions for roles Generic XSLT Search and Replace template Access And/Or exclusions Pyspark: Filter dataframe based on multiple conditions Subtracting 1 day from a timestamp date PYODBC--Data source name not found and no default driver specified select rows in sql with latest date for each ID repeated multiple times ALTER TABLE DROP COLUMN failed because one or more objects access this column Create Local SQL Server database

Examples related to postgresql

Subtracting 1 day from a timestamp date pgadmin4 : postgresql application server could not be contacted. Psql could not connect to server: No such file or directory, 5432 error? How to persist data in a dockerized postgres database using volumes input file appears to be a text format dump. Please use psql Postgres: check if array field contains value? Add timestamp column with default NOW() for new rows only Can't connect to Postgresql on port 5432 How to insert current datetime in postgresql insert query Connecting to Postgresql in a docker container from outside

Examples related to bulkinsert

Insert Multiple Rows Into Temp Table With SQL Server 2012 PostgreSQL - SQL state: 42601 syntax error Import CSV file into SQL Server Cannot bulk load. Operating system error code 5 (Access is denied.) How to Bulk Insert from XLSX file extension? Bulk Insert Correctly Quoted CSV File in SQL Server How to speed up insertion performance in PostgreSQL BULK INSERT with identity (auto-increment) column How do I temporarily disable triggers in PostgreSQL? mongodb: insert if not exists

Examples related to sql-insert

In Oracle SQL: How do you insert the current date + time into a table? SQL Error: ORA-01861: literal does not match format string 01861 "Insert if not exists" statement in SQLite Column count doesn't match value count at row 1 Inserting data to table (mysqli insert) SQL Insert into table only if record doesn't exist C# with MySQL INSERT parameters How to speed up insertion performance in PostgreSQL "column not allowed here" error in INSERT statement Inserting multiple rows in mysql