This is how I do it to return to the right fragment otherwise if you have several fragments on the same level it would return to the first one if you donĀ“t override the toolbar back button behavior.
toolbar.setNavigationOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View view) {
finish();
}
});