1 You can use a drawable
<menu xmlns:android="http://schemas.android.com/apk/res/android">
<item android:id="@+id/menu_item1"
android:icon="@drawable/my_item_drawable"
android:title="@string/menu_item1"
android:showAsAction="ifRoom" />
</menu>
2 Create a style for the action bar and use a custom background:
<resources>
<!-- the theme applied to the application or activity -->
<style name="CustomActivityTheme" parent="@android:style/Theme.Holo">
<item name="android:actionBarStyle">@style/MyActionBar</item>
<!-- other activity and action bar styles here -->
</style>
<!-- style for the action bar backgrounds -->
<style name="MyActionBar" parent="@android:style/Widget.Holo.ActionBar">
<item name="android:background">@drawable/background</item>
<item name="android:backgroundStacked">@drawable/background</item>
<item name="android:backgroundSplit">@drawable/split_background</item>
</style>
</resources>
3 Style again android:actionBarDivider
The android documentation is very usefull for that.