"I'm speciesist with regard to humans and animals. I'd save a human over an animal. I wouldn't let a cow have a driver's license. So speciesism is a dumb term."
Let's start with what "speciesism" isn't: the treatment of animals differently based on emergent properties. So for instance, Betsy is a cow. Cows are unable to operate motor vehicles. Now, the fact that 100% of cows (as far as I'm aware) have this set of properties is a coincidence, and therefore we don't grant Betsy this set of rights not on the basis that she is a cow, but on the basis that the individuals we know as cows have some set of properties shared by all cows.
You will notice this is consistent and easy to follow because vegan ethics then more or less translate directly to humans, for instance. If a human is unable to operate a motor vehicle, then we do not grant them that right (we may have some heuristics like age limits but these are ultimately still derived from these emergent properties). If a human, like a cow, is unable to do the same evaluation required for freedom of association, then we don't grant them that right either - for instance, toddlers aren't allowed to pick their legal guardians, or humans with developmental disabilities or age-related mental degeneration sometimes aren't granted (or have revoked) their driver's licenses. Similarly, if we met aliens and assessed them as being intelligent enough to integrate into our society in certain ways we would well make those accommodations based on those characteristics.
Speciesism is, specifically, the inclusion of species as an arbitrary criterion of ethical responsibility toward them. So like how Saudi Arabia didn't grant women driver's licenses based on nothing other than their sex, not based on emergent properties such as their capacity to operate a motor vehicle. We recognize that something like that is not only unethical but also kind of stupid, and the same applies to animals.
Notice two huge aggravating factors here: First, compare a human with the intelligence of a typical cow (whether mentally handicapped, an infant, or suffering from some degenerative mental disease) to such a typical cow itself: on the one hand, it's illegal to kill any such human, and you will be severely punished for doing so, and if you were to stop someone from killing that human you would be commended. On the other hand, if you stop someone from killing the animal you are not only the weird one but also might, in fact, do hard time. Second, recognize that no emergent property is at work here - it is purely one's taxonomic classification that affords one this protection. This is plain to see how this right we grant humans is granted to dogs, for instance, whereas one can engage in deliberate acts of animal cruelty with other species as long as it's "consistent with typical animal agriculture practices", but can land you in jail if done to "good animal" and not a "food animal".
In short, not only is it a huge rift that is purely based on species, but the actual emergent properties here are completely overshadowed by this one property. It's not a heuristic to an end; it's just a completely arbitrary distribution of rights. This is what the term "speciesism" refers to.
So, the main point here about speciesism is that it is unethical for the same reason that sexism and racism are unethical. Ask yourself what is morally wrong with enslaving people of a different genetic heritage than you, or denying rights such as driver's licenses to people of a different sex than you - specifically (and this is important) not based on any emergent property that these individuals have, but purely on the basis that they belong to this different class which you happen to have drawn a circle around and stuck a label in. And then ask yourself if that is sincerely any different from drawing a circle around different species and making moral determinations as drastic as "OK to murder; reprehensible to stop the murder of" and "reprehensible to murder; OK to stop the murder of" based on this one label and only on this one label which otherwise carries no moral weight.