The person is still getting hit with all the force, regardless of how fast the one guy is moving. The person he's hitting will move; there's no way for a person to resist that kind of force.
It doesn't matter that he's moving faster than a normal person can or that everything is "slow motion." Outside his body's sphere of influence, for lack of a better term, time still flows at a normal pace.
EDIT: It won't be like hitting a wall. The only reason "hitting a wall" is a term is because the force needed to move said wall is greater than a human can exert. As long as you can exert more force on an object that it can resist based on it's own structure (for a wall, weight of the material and foundation in simplest terms) that object will move.
XI Wiki



