The difference lies in their usage. The single quotes are mostly used to refer a string in WHERE, HAVING and also in some built-in SQL functions like CONCAT, STRPOS, POSITION etc.
When you want to use an alias that has space in between then you can use double quotes to refer to that alias.
For example
(select account_id,count(*) "count of" from orders group by 1)sub
Here is a subquery from an orders table having account_id as Foreign key that I am aggregating to know how many orders each account placed. Here I have given one column any random name as "count of" for sake of purpose.
Now let's write an outer query to display the rows where "count of" is greater than 20.
select "count of" from
(select account_id,count(*) "count of" from orders group by 1)sub where "count of" >20;
You can apply the same case to Common Table expressions also.