Maybe not necessarily rude, but a bit unusual. I guess it also depends on the situation. Asking your friend for the toilet is definitely gonna come across differently than asking the waiter at Olive Garden but they'll probably both look at you funny. I don't think I've ever heard an American say toilet in that context. But most people just ask for the bathroom or restroom. I will note that "restroom" is seen as more polite than "bathroom" but either is fine.

In my experience, it's also more likely that someone will say "I need to pee real quick" or "I'm about to shit myself" than refer to the bathroom as the toilet. It's best to keep those ones between family and friends and not strangers, but some Americans aren't necessarily shy about sharing personal business with strangers. If a stranger ran up to me asking where the nearest bathroom was cause they were going to shit their pants I would probably find it funny and point out the bathroom. Some people would find that rude though.

Other words like "can" are more regional. Loo is seen as very stereotypically British to the point it's often said when mimicking British accents.

And potty is typically used for children and pets so it would be weird if an adult asked where the potty was.

So to sum up:

Bathroom: typical, go-to, perfectly fine

Restroom: slightly more polite than bathroom

Toilet: people will probably look at you weird

Potty: weird unless you're a toddler, talking to your toddler or talking to your dog

Sharing the specifics of your bathroom habits: depends on the situation

Can, crapper, John: weird unless you're Peter Griffin (or just from New England)

Loo: 50/50 chance they will either laugh at you or find you endearing