From the Android Docs -
From View.OnLongClickListener. This is called when the user either touches and holds the item (when in touch mode), or focuses upon the item with the navigation-keys or trackball and presses and holds the suitable "enter" key or presses and holds down on the trackball (for one second).
From View.OnTouchListener. This is called when the user performs an action qualified as a touch event, including a press, a release, or any movement gesture on the screen (within the bounds of the item).
As for the "moving happens even when I touch" I would set a delta and make sure the View has been moved by at least the delta before kicking in the movement code. If it hasn't been, kick off the touch code.