I recently just ran into this issue as well. I had a very large table in the dialog div. It was >15,000 rows. When the .empty() was called on the dialog div, I was getting the error above.
I found a round-about solution where before I call cleaning the dialog box, I would remove every other row from the very large table, then call the .empty(). It seemed to have worked though. It seems that my old version of JQuery can't handle such large elements.