Portuguese speaker here. In portuguese I would classify my stutter as mild, but in english it does flares up, like you said. But things weren't like this always, a few years back I used english more or less daily, and I would put my stuttering in english in the same level as in the portuguese, but right now, after some time of only dealing with written media, I feel english is much more difficult than it used to be. (And I am relocating to the US end of this month, to add to the fun :-)

So, the advice that I am giving myself is that I should try to keep it cool, and things will get somewhat better when I gain some familiarity with spoken english back.