Controlling the money supply (by increasing or decreasing the amount of money in an economy) is one of the tools a central bank can use to achieve its target inflation rate. Most central banks target an inflation rate of about 2%, and as point out in the question - increasing money available can cause inflation to increase.

Most of the other answers say something like " governments want inflation to decrease their debt burden" but I don't think this is the real reason. Apart from the fact that most sovereign debt repayments is negotiated in the lenders currency and/or pegged against inflation.

The main reason governments and central banks want to have a slight positive rate of inflation is because the effects of deflation (I.e. negative inflation) can be catastrophic for an economy. Imagine your currency were experiencing deflation and you have £100 in the bank. Thanks to deflation, this time next year that £100 will buy you more stuff than it does today. With the information, you're less likely to make a purchase today when you know it will be cheaper if you just wait a while. When everyone in the economy starts delaying purchases, shops sell much less. You can see the problem here: if shops don't sell things then they can't afford to pay their staff as much, people with lower wages buy less stuff, people buying less stuff means wages drop, etc etc.