This question was put to the Vilner Gaon, as an intended disproof of the obligation to read the other megillos from a klaf. His answer was that unlike Esther, the other megillos are not an individual obligation, but a chovas hatzibbur, so there is no reason to fear an individual's carrying it to fulfill his obligation. It is no different from a sefer Torah, or a klaf for haftarah, which are read on Shabbos.