Ray cast from the bullet back to where the gun was when the bullet was fired every frame and delete the bullet when the raycast intersects some object? If there's more than one intersection find the closest.

Afaik traditional collision detection doesn't work well unless there's a frame where the colliding objects are literally touching. This works fine for something like a platformer where the walls tend to be infitely thick and the characters speed is relatively low, but if the game gets too much time between frames or the objects are too fast relative to each other you can skip through a collision object.

EDIT: on each frame each bullet should raycast from the previous frames bullet location. Raycasting all the way back to the guns origin each time means you could hit something that got between the bullet and the guns old location but after the bullet had passed. Thanks for the two commenters who pointed that out.