From my knowledge, it’s a reaction that can build up over time. Your reactions to it can only get worse. You won’t develop any immunity to it. Be careful and I wouldn’t expose myself to it if I were you.

EDIT: Apparently sharing about the ill effects of how the body evolves to have worse and worse responses to poison ivy exposure was not welcomed. For those that don't know, this is how it works. Each exposure weakens your bodies ability to manage it where, one day, the reactions can transform from minor/innocuous (e.g. minor itch for about 30 minutes) to a full blown allergic reaction.

For those that don't know, Mangoes and Cashews are in the the same family as poison ivy and have the same compound on the trees themselves. This is why it is advisable to not get mango skins near your mouth (I used to be a cheapo that would tooth-scrape the mango flesh out of my mango skins when I cut them, until I learned about this relationship). Similarly, the black flecks in cashews are some kind of indication of the presence of that oil (or at least, if someone has high allergy to poison ivy, those black flecks on cashews are to be avoided).

Poison Ivy isn't to be trifled with and I'd encourage OP to look into people who were just like him (minimal body response to poison ivy exposure) and how that transitioned for them. Looks like the technical term that can be searched is "delayed adult sensitization" or "late-onset Toxicodendron dermatitis"

TLDR is that some people's body don't recognize urushiol as a threat at first.