In my experience of doing physical scenes, half of your energy is spent on trying to get the other actor to enter into it physically with you. Most actors don't want to hurt each other.