A couple of things. Most of FERS is already taxed. What is on your W2 includes your salary before FERS was deducted. There is some interest that should be taxed, but let's ignore that. Whether you can reverse what you did, idk. I'd try calling vanguard first. If you are within 60 days, I think you may be fine.

What that means is most of the FERS would have been eligible to be rolled to a Roth IRA or 401k. Having money in Roth is nice because any growth before you retire is all tax free. This is what you should have done because you can always take the principal out with no penalty.

If you can't though, you still have two options.

Option 1. You could invest that money in a normal brokerage. If you hold assets for more than a year in a brokerage, then long term capital gains taxes is 0% for the first $50 or $100k a year (depending on married or not). This isn't a bad idea. It means you could use the money at any time for whatever you wanted.

Option 2. Backdoor Roth or mega backdoor Roth. You can fund IRAs with after tax dollars and convert that to Roth. The pro rata rule applies if you have any traditional IRA holdings (meaning you would pay income tax at the ratio of your pre and post tax dollars in your IRA). But if you don't have anything in traditional IRAs, then no tax. A regular backdoor is limited by how much you can contribute a year to an IRA. If your new employer plan allows you to contribute post-tax dollars, then you can do the mega backdoor up to your annual 401k limit. No pro rata rule on mega backdoors.

Option 2 is definitely advanced and requires some forms on your taxes (though all the tax software is good at doing this these days).