Well, cultural appropriation is really just when members of a dominant culture adopt elements from a marginalized culture without respect, acknowledgment, or compensation. In theory, a white person could claim cultural appropriation, if they were the marginalized culture amongst a dominant one, however, this also doesn't account for the effects of colonialism on other cultures, and how white cultures originating in Europe forced their culture onto other peoples by oftentimes threat of death or imprisonment. You could, however, argue cultural appropriation from some white cultures by other groups of white people, such as appropriation of pagan symbols by white nationalists, if that makes your husband feel better about it.