I think the answer is pretty simple (unless I'm missing something?)
SELECT
CASE
WHEN col1 > col2 THEN SUM(col3*col4)
ELSE 0
END AS some_product
FROM some_table
GROUP BY
CASE
WHEN col1 > col2 THEN SUM(col3*col4)
ELSE 0
END
You can put the CASE STATEMENT in the GROUP BY verbatim (minus the alias column name)