Bolton was reportedly a member of the centrist Liberal Democrat party before joining UKIP, and some speculate his leadership could distance UKIP from hard-line anti-Islamic rhetoric.
UKIP had a disastrous General Election - and their membership fell dramatically.
After being unveiled as the new leader, Bolton told the party's annual conference in Torquay: "Today is not only a crucial day for our party, it is a critical day for our country". Ms Waters came second out of seven candidates, taking 21% of the vote.
It had been feared that a victory for Ms Waters would have seen the party lurch towards the far right, prompting up to 18 MEPs and scores of councillors and other members to quit the party's ranks.
Bolton, an ex-soldier, won 30% of the vote, seeing off Anne Marie Waters's challenge.
"When we leave the European Union, that is not the end of the story".
Of course I want to stay, I love the club
Since BATE is now in good shape, and Arsenal has low motivation for the upcoming meeting, the hosts may well not lose. Per Mertesacker won the initial header off a corner and the ball bounced the Holding who shinned/kneed/thighed it in.
Bolton distanced himself from the anti-Islam direction advocated by Waters and to a lesser extent by the early favourite, Peter Whittle, who finished a surprise fifth in the contest.
Speaking immediately after the announcement, Bolton said pressuring Theresa May's government into a hard Brexit will be UKIP's "core task" under his stewardship. I do not see myself as being your leader, I see myself as serving this party.
Mr Bolton said he believes in "old-fashioned, classical liberalism", but feels it has "lost its way a bit".
Mr Goldsmith died soon after the 1997 election, creating political space for Ukip after his Referendum Party was dissolved.
Asked at a press conference after the result was announced whether this made him an "establishment mole", Bolton said his background made him ready for the job.
Why is Ukip on its fourth leader in just over a year? The party won no seats and just 1.8% of the national vote, down from the 12.6% of the vote it won in 2015.
- BAE Systems PLC (BAESY) Raised to Hold at Zacks Investment Research
- Little Girl Steals Prince Harry's Popcorn At Invictus Games
- Shelby County Schools Statement On Students Kneeling During Anthem
- State Department Will Not Recognize Kurdish Independence
- Central Florida leaders working to create reception center for displaced Puerto Ricans
- Kurdish unilateral independence and a deadly impasse
- Interview: US measures against Cuba negative, arbitrary, says analyst
- Demi Lovato couldn't see her sister until she got sober
- US Pulls Some Staff Out of Cuba After Bizarre Health Episodes
- Piers Morgan defends Ben Stokes, he 'was fighting for gay guys'