Well, a computer scientist named Cathy May at IBM Research did this for the S370 / Power PC architecture pair back in the mid-1980s. The work was novel to me at the time ... she and her long-term partner were close friends of mine, so I heard about her work over shared dinners across several years.
The basic technique was to load pages of 360 machine code unmodified into Power PC memory, marking the pages as dirty. If the machine branched into one of these pages the VM trap handler would take the page and translate all of its code from 370 instructions to Power PC instructions. Once that was done the branch, suitably recalculated, was reinstated.
It turned out to be remarkably easy to do the translation. And in practice the resulting Power PC code was only about 10% bigger than the original 370 code. AND, since tons of code is never touched (corner cases on corner cases) the result is quite efficient.
I don't know if IBM ever built a product based on her research, but it seemed remarkably clever to me.
Marc