For those that are using Text::CSV I found this thread and then noticed within the CSV module that you could strip it out via switch:
$csv = Text::CSV->new({allow_whitespace => 1});
The logic is backwards in that if you want to strip then you set to 1. Go figure. Hope this helps anyone.