Hello @Nick_Rubert,
You can constrain the transformation via the weighting scheme (see SetOptimizerWeights). For the BSpline transform the parameters are the translation deltas of the control points. For n control points you have dx_0,dy_0,dz_0...dx_n,dy_n,dz_n . If you set the weights to 0,0,1...0,0,1 you limit the control point motion to the z axis, which limits deformation to that direction.
Generally, you zero out the axes along which you want to limit the deformation. This is crude, as you are limited to the imaging axes which don’t necessarily correspond to the arbitrary direction you want.