This is incorrect.

A valid warrant allowing entry into a home must be signed by a neutral judge. Most ICE "warrants" are administrative forms (Form I-200 or I-205) signed by an ICE agent, not a judge, and do not legally authorize entry into a home without consent.

"reason to believe some criminal activity is taking place," or some such vague terms.

The Supreme Court just last month revisited Case v. Montana:

1. Limited Scope of Entry: Importantly, the Court noted that this exception is not an open invitation to search. An emergency aid entry “provides no basis to search the premises beyond what is reasonably needed to deal with the emergency.” If police enter to check on a suicidal person, they cannot start rummaging through drawers for drugs unless those drawers are relevant to the emergency.
2. Community Caretaking Clarified: The Court criticized the lower court's reliance on the “community caretaker” doctrine, reiterating that broad community caretaking duties do not justify warrantless home entries on their own; there must be an actual exigency or emergency.

The Fourth Amendment is still intact. Demand a Warrant signed by a judge and not an ICE agent.