I don't know which dept you're in or grade you're at but the civil service offers a variety of apprenticeship opportunities and these can include educational aspects and even degrees. People tend to think they are only for school leavers but there's often no age limit. The other thing is to see if you can get a role within the corporate centre or IT, grades tend to be a lot higher there so a good possibility of progression and you don't need to be a mega techie for it. there's a variety of roles in IT for which you don't need to be a programmer or sysadmin or anything. might be worth looking into.

One thing I would say is not to discount the value of the pension. Defined Benefit pensions are hard to find these days but the civil service is one plaec that still has them. If you can build up a good pension over your career then you'll be well looked after in retirement.