The Collaborative International Dictionary of English v.0.48:

Alarmedly \A*larm"ed*ly\, adv.
   In an alarmed manner.
   [1913 Webster]