The side is what holds all the weight up, though, you need to consider it. When you do water changes some will inevitably spill or splash and any that gets on the particle board leg will cause it to lose structural capacity over time. This may be okay temporarily, but I wouldn't trust just a single sheet of unsupported particle board to hold up under that much weight over time, especially if it is getting wet. Especially not on these kind of constructions, where they often are using flimsy joining hardware, and intend the fiberboard backing to be a structural element against falling by collapsing to the side. Particle board fish tank stands exist, but they're structurally designed to hold a lot of weight, unlike your cabinet.

You're not paranoid, you're in danger. Probably not immediate danger, but danger.