For reference, for a 512^3 16-bit volume with a fairly large mask, numpy provides result in a fraction of a second (around 200msec). Interestingly, np.extract is about 50% slower than direct array indexing image[condition].
So, using numpy for image masking is indeed much slower than ITK but still good enough for many use cases.