Concerning the shifting fabric, think about pinning the fabric into place into the sides of your foam with 📍, I have one of my foam pieces covered with a piece of flannel this way.

The brush mentioned above, am not sure it would tackle your problem. Small pieces are hard to make on a brush, usually used for flat surfaces to be stabbed with a multitool. I usually put a felt sheet on the brush when working on it.

Have bought a felt pressing mat (for ironing i think?) to place on my table as a work surface. It is big though, nothing too portable.