At what level can I purchase my first horse?

You are able to purchase a horse at any level, however, by choosing to do the missions offered by the NPCs in the lifenet facility of your starter town, you will receive a Worn Horse Bridle as part of the mission line free of charge. The starting horses are the Old Nag (Worn Horse Bridle), Horse (Horse Bridle), and the Riding Horse (Riding Horse Bridle).

If you choose to skip the first missions, you can purchase the Old Nag, the Horse Bridle, or the Riding Horse Bridle from your local Horse Merchant (usually found at gas stations).  If you're looking for something a little more specialized, you can purchase a crafted mount from other players, or train one yourself using the Nature tradeskill.