Currently, the privacy for custom profile fields are set by field type and I guess that is too generalized. Maybe a better option is to apply Privacy to Field Steps rather.

Thanks! - Jackson