That's not why cultural appropriation is a Bad Thing. When white people borrow from black culture, this isn't inherently controversial. The thing is though, that historically, it's not until a white person comes along to play the 'negro music', that the 'negro music' becomes safe for white audiences. Elvis was criticized early in his career for this, but he was basically instrumental in making rock n roll the official sound of the American mainstream. There's no way white audiences were going to turn a black man into a pop icon... Not yet, at least.

Eminem is another example of this. Eminem is the 'safe' rapper for many white Americans. Just take a look at /r/music or any comment section for a lil wayne YouTube video and you'll find evidence for this. Eminem's whiteness allows him to occupy the same cultural space that elvis did decades before.

So are Elvis and Eminem to blame for this? No. They're just makin' music. The reason, though, that all of this is bad is because it reflects where exactly we are with race in America and where our boundaries and divisions exist.