Hmm, as far as I know,/my experience was that with OSF/1 was fine with shared libs, no reboot needed (at least on the 386 reference -- I never tried it on the PMAX].
I suspect you might be thinking about drivers and kernel modules, which were not nearly as clean - particularly in the pure (Mach 3.0 based) OSF/1 microkernel implementation [which was hardly 'micro' in size]. IIRC Rob's new memory system code for SunOS was one of the first UNIX implementations that supported loadable kernel modules.
Anyway, for the original OSF/1, I really don't think shared libs caused a reboot using the original CMU Mach 2.5 memory code, but adding things to the kernel did. I know that was changed with the MMU rewrite for Tru64, but I think it was also done in later versions of the OSF RI kernel (but I've forgotten).