Single quotes delimit a string constant or a date/time constant.
Double quotes delimit identifiers for e.g. table names or column names. This is generally only necessary when your identifier doesn't fit the rules for simple identifiers.
See also:
You can make MySQL use double-quotes per the ANSI standard:
SET GLOBAL SQL_MODE=ANSI_QUOTES
You can make Microsoft SQL Server use double-quotes per the ANSI standard:
SET QUOTED_IDENTIFIER ON