Indigenous people have their own culture and way of life. You're destroying that by trying to convince them to go vegan
One sign that the people making this argument don't actually buy it is all the outrage against the Yulin Dog Meat Festival. So, wait... so are we on board with this idea fully or is it just that we make an exception on the basis of "food" animals or "good" animals? If the Aztecs were around would human sacrifice be ok? It doesn't make sense as a way of determining ethics and no carnist applies it as such in any other circumstance.
For reference slavery has been (and in some cases continues to be) practiced by "indigenous" peoples on nearly every continent. Should we also be against abolitionism in these areas on the basis that the first people in a location happen to continue on in an unethical practice? And why does the fact that it's the first person to set foot in an area matter? If we colonize another planet (or even Antarctica), then do the first people to set foot in a particular acre get carte blanche to establish any ethical system they want?
This argument is typically held up against vegans discussing carnism as a general topic and is used to silence conversation on the topic at all. So, this isn't an argument against protecting indigenous culture, this is an argument that is preventing indigenous people from even hearing the argument that veganism is moral.
Culture is a poor argument. Arguments for protecting racial demographics suck as well. At best, you're saying "Hey, be understanding that this is the culture they grew up in, and don't be a judgmental douche when you talk to them about your ethical positions", which is a carnist stereotype of vegans to begin with and not a good argument against veganism, and certainly isn't an argument that "we need to protect their delicate ears from being touched by brutish vegan attacks".