The guy that did AForge did a fairly good job but it's not commercial quality. It's great to learn from but you can tell he was learning too so he has some pretty serious mistakes like assuming the size of an image instead of using the correct bits per pixel.
I'm not knocking the guy, I respect the heck out of him for learning all that and show us how to do it. I think he's a Ph.D now or at least he's about to be so he's really smart it's just not a commercially usable library.
The Math.Net library has its own weirdness when working with Fourier transforms and complex images/numbers. Like, if I'm not mistaken, it outputs the Fourier transform in human viewable format which is nice for humans if you want to look at a picture of the transform but it's not so good when you are expecting the data to be in a certain format (the normal format). I could be mistaken about that but I just remember there was some weirdness so I actually went to the original code they used for the Fourier stuff and it worked much better. (ExocortexDSP v1.2 http://www.exocortex.org/dsp/)
Math.net also had some other funkyness I didn't like when dealing with the data from the FFT, I can't remember what it was I just know it was much easier to get what I wanted out of the ExoCortex DSP library. I'm not a mathematician or engineer though; to those guys it might make perfect sense.
So! I use the FFT code yanked from ExoCortex, which Math.Net is based on, without anything else and it works great.
And finally, I know it's not C#, but I've started looking at using FFTW (http://www.fftw.org/). And this guy already made a C# wrapper so I was going to check it out but haven't actually used it yet. (http://www.sdss.jhu.edu/~tamas/bytes/fftwcsharp.html)
OH! I don't know if you are doing this for school or work but either way there is a GREAT free lecture series given by a Stanford professor on iTunes University.
https://podcasts.apple.com/us/podcast/the-fourier-transforms-and-its-applications/id384232849