The Collaborative International Dictionary of English v.0.48:

Horseback \Horse"back`\, n.
   1. The back of a horse.
      [1913 Webster]

   2. An extended ridge of sand, gravel, and boulders, in a
      half-stratified condition. --Agassiz.
      [1913 Webster]

   On horseback, on the back of a horse; mounted or riding on
      a horse or horses; in the saddle.
      [1913 Webster]

            The long journey was to be performed on horseback.
                                                  --Prescott.
      [1913 Webster]