How does it work if you only are using visible-md at Col4 instead? Do you use the -lg at all? If not this might work.
<div class="container">
<div class="row">
<div class="col-xs-4 col-sm-2 col-md-1" align="center">
Col1
</div>
<div class="col-xs-4 col-sm-2" align="center">
Col2
</div>
<div class="hidden-xs col-sm-6 col-md-5" align="center">
Col3
</div>
<div class="visible-md col-md-3 " align="center">
Col4
</div>
<div class="col-xs-4 col-sm-2 col-md-1" align="center">
Col5
</div>
</div>
</div>