For reference, I just encountered much the same issue on Linux and had a hell of a time figuring it out. I eventually determined that I had a ~/.android directory that was root owned (thanks to running adb as root, while flailing around trying to figure out other connection problems). Deleting that root owned ~/.android directory resolved the missing authorisation dialog next time I reconnected the device.
It's possible that simply changing the ownership would also have fixed the problem - I'm guessing it was a simple lack of access to the adb key file stored inside that was the root of the problem. I didn't verify that, though, and I'm not going to deliberately break my hard-won connectivity just so I can check ;-)