Yes, this is confusing...
According to this blog post, it looks like this is an omission from WPF.
To make it work you need to use a style:
<Border Name="ClearButtonBorder" Grid.Column="1" CornerRadius="0,3,3,0">
<Border.Style>
<Style>
<Setter Property="Border.Background" Value="Blue"/>
<Style.Triggers>
<Trigger Property="Border.IsMouseOver" Value="True">
<Setter Property="Border.Background" Value="Green" />
</Trigger>
</Style.Triggers>
</Style>
</Border.Style>
<TextBlock HorizontalAlignment="Center" VerticalAlignment="Center" Text="X" />
</Border>
I guess this problem isn't that common as most people tend to factor out this sort of thing into a style, so it can be used on multiple controls.