I have taken off my mufflers several times with no problems.
I take my mufflers off for every tire change. The mufflers are in the way of removing the axle, and also prevent the use of sockets. I also remove them to lube the rear linkage, but I try to do it while changing a tire anyway. After 100k miles, and many tires, I still haven't needed to replace the muffler bearings. I'm careful when putting the mufflers back on. Having said that, Murphy's law means they'll shed next time I pull the mufflers off.