The retained bearing in wheels sets the whole wheel alignment in the frame/forks to facilitate everything relating to it.
The non retained bearing is essentially floating and only then held in alignment by the spacer and then the spindle being tightened, all "slaved" off that retained bearing recess as singular datum.