Racist attitudes were often much stronger among the "poor whites" than the actual aristocracy. It's not like the rich plantation owners were always the ones carrying out lynchings and other horrors against non-whites. You can't just exonerate all those whites simply because they weren't wealthy, that's a misreading of how racism actually manifested itself in this country.
The idea that poor whites between 1850 and 1920 didn't have "the power to oppress other groups" is false.
I agree that the poor whites were often terrible bigots, but don't think the Tim Wise types vilifying them works, as the GOP has shown since Nixon and Agnew introduced the "Southern strategy" in 1968 (as a milder form of Goldwater running against the 1964 Civil Rights Act). The Jesse Helms ad below (from his successful 1990 re-election campaign in NC) was extremely effective:
Indeed the wealthy whites during the era of huge European immigration; from the Irish fleeing the potato famines of the 1840's thru Jews, Poles, Italians, etc. later in the 19th Century, until the US Government started "closing the gate" in the early 20th Century (especially as xenophobia soared with massive casualties in WW I and the Russian Revolution created the "Red Scares") played off the various "have nots" against eachother.
Psychologists have done experiments I won't bore you with the details of here, but confirm that no group wants 2B "in last place" in the social pecking order. So indeed the white aristocracy were able to divert anger properly directed @ them to "the other", a role they cast blacks in for decades (and more recently Latinos, of course). I don't disagree with you, I just find ppl like Tim Wise counterproductive. And I'm sure he realizes this about his inyaface approach, but makes a very nice living giving paid speeches about it.