[Avodah] Chol Moed Minyan for Those Who Wear Tefillen

[Sholom Simon via Avodah]

>Most posqim do rule that the guys wearing tefillin on ch"m should be in
>a different minyan than those who don't. And barring that, compromises
>evolved like hijacking the ezras nashim for the smaller group.

There's something about this that I've never understood.  Suppose a 
person is having a stomach problem -- isn't he supposed to refrain 
from wearing tefillin?  So, in a shul, a guy is not wearing tefillin 
-- how do we even know what his reason is?  (In fact, I think I 
recall hearing that R Moshe himself didn't wear tefillin many times 
towards the end of his life for this reason).

So, of all the things regarding lo sisgadedu -- why tefillin?  Why 
not all the other things that daveners wear differently?  (E.g., 
different minhagim via-a-vis wearing a tallis before marriage?)

-- Sholom