I think that your "textual digits" only arise in the absence of proper
 markup, just as it is common to have other symbols sprayed into the text,
 in the absence of a proper symbol environments.

of course not. ~2.3 has nothing to do with maths, neither p.~12312.

Up to some extent, I personnally consider that _units_ are not maths
(you can write : I drive my car 3 miles or 12 km; this is text, it
would get italised if the context tells so, etc.) The \textmu in m is
also a textual object. (I would use \text{\normalfont  m} in a math

Thierry Bouche
   Ils vivent pour vivre, et nous, hlas! nous vivons pour savoir. 
                                Charles Baudelaire, Paris.