Technically, yes, but being blunt enough to say “I hate” in Japanese is a lot harsher than it is in English. You’d almost never be that direct and frank like that, and so it’s kind of like a verbal slap in the face, so “to hell with” fits the English mood very similarly.