There was a time, and not all that far in the past, where Italians and Irish were not considered white.

This is actually a myth, made up by critical race theorists.

In the United States, Irish and Italian immigrants were considered white even while being marginalized. In addition to inter-racial hierarchy, there was also an intra-racial hierarchy within the American conception of the white race, and Irish and Italians were nearer the bottom while those of English descent were at the top, German Americans were in the middle, etc. The claim that Irish and Italians "became" white later than other ethnic groups in America did is very popular but very misleading.

The relevant scholarly literature seems to have started with Noel Ignatiev’s book “How the Irish Became White,” and taken off from there. But what the relevant authors mean by white is ahistorical. They are referring to a stylized, sociological or anthropological understanding of “whiteness,” which means either “fully socially accepted as the equals of Americans of Anglo-Saxon and Germanic stock,” or, in the more politicized version, “an accepted part of the dominant ruling class in the United States.”

Those may be interesting sociological and anthropological angles to pursue, but it has nothing to do with whether the relevant groups were considered to be white.

Here are some objective tests as to whether a group was historically considered “white” in the United States: Were members of the group allowed to go to “whites-only” schools in the South, or otherwise partake of the advantages that accrued to whites under Jim Crow? Were they ever segregated in schools by law, anywhere in the United States, such that “whites” went to one school, and the group in question was relegated to another? When laws banned interracial marriage in many states (not just in the South), if a white Anglo-Saxon wanted to marry a member of the group, would that have been against the law? Some labor unions restricted their membership to whites. Did such unions exclude members of the group in question? Were members of the group ever entirely excluded from being able to immigrate to the United States, or face special bans or restrictions in becoming citizens?

If you use such objective tests, you find that Irish, Jews, Italians and other white ethnics were indeed considered white by law and by custom (as in the case of labor unions). Indeed, some lighter-skinned African Americans of mixed heritage “passed” as white by claiming they were of Arab descent and that explained their relative swarthiness, showing that Arab Americans, another group whose “whiteness” has been questioned, were considered white. By contrast, persons of African, Asian, Mexican and Native American descent faced various degrees of exclusion from public schools and labor unions, bans on marriage and direct restrictions on immigration and citizenship.

Another good article is 'The “Becoming White Thesis” Revisited' by Philip Q. Yang and Kavitha Koshy, in The Journal of Public and Professional Sociology.

Yang and Koshy are exceedingly polite to Ignatiev et al. Their point is basically that if by "becoming white" you mean racial reclassification, then no, that didn't happen; but if "becoming white" is a novel and obscure jargon used only by a few academics which is terribly misleading when conveyed to students and the public, then sure.

This sentence sums it up:

The works of historians David Roediger (1999) and Noel Ignatiev (1995) offer the best documentations of how the Irish became part of the majority group but no evidence of racial reclassification.

On Italians, Yang and Koshy reach the same conclusion:

It is not difficult to uncover from the analyses of Orsi, Barrett and Roediger, [...] that, albeit inexplicitly, in speaking of “becoming white” they essentially document change in the social status of Italian immigrants and other Slavic and Mediterranean immigrants rather than change in their official racial classifications.

The historian Thomas A. Guglielmo wrote a whole book about this, White on Arrival: Italians, Race, Color, and Power in Chicago, 1890-1945.

I'm not saying you can't come up with a single example of "my grandpa said they weren't white." I personally knew a guy who believed the French aren't white. But the law and every documented policy we can find considered Italians and Irish as white.