If you do flipping, that will likely take a large part of the displacement field outside of your image. That might be the problem.
How is elastic deformation represented? If using BSpline, maybe you should “flatten” your transforms to a BSpline instead of displacement field.
If my theory is correct, using a different origin and larger size along z axis for displacement field should “resolve” the problem.