The Collaborative International Dictionary of English v.0.48:

Ordinarily \Or"di*na*ri*ly\, adv.
   According to established rules or settled method; as a rule;
   commonly; usually; in most cases; as, a winter more than
   ordinarily severe.
   [1913 Webster]

         Those who ordinarily pride themselves not a little upon
         their penetration.                       --I. Taylor.
   [1913 Webster]

WordNet (r) 3.0 (2006):

ordinarily
    adv 1: under normal conditions; "usually she was late" [syn:
           normally, usually, unremarkably, commonly,
           ordinarily] [ant: outstandingly, remarkably,
           unco, unusually]