So there's loads of posts on the web that show how to do this, I've found 3 ways, same as pointed out by Johan & Sjoerd. I couldn't get any of these queries to work, well obviously they work fine it's my database that's not working correctly and those queries all ran slow.
So I worked out another way that someone else may find useful:
The basic jist of it is to create a temporary table and fill it with all the information, then remove all the rows that ARE in the other table.
So I did these 3 queries, and it ran quickly (in a couple moments).
CREATE TEMPORARY TABLE
`database1`.`newRows`
SELECT
`t1`.`id` AS `columnID`
FROM
`database2`.`table` AS `t1`
.
CREATE INDEX `columnID` ON `database1`.`newRows`(`columnID`)
.
DELETE FROM `database1`.`newRows`
WHERE
EXISTS(
SELECT `columnID` FROM `database1`.`product_details` WHERE `columnID`=`database1`.`newRows`.`columnID`
)