If you're using Modernizr - then you can easily differntiate between IE10 and IE11.
IE10 doesn't support the pointer-events
property. IE11 does. (caniuse)
Now, based on the class which Modernizr inserts you could have the following CSS:
.class
{
/* for IE11 */
}
.no-pointerevents .class
{
/* for IE10 */
}