This is something that carnists say to get vegans to not challenge their moral beliefs. Vegans should just live, and let carnists live.
If you accept killing animals is wrong, then ultimately you're saying that no matter what you do that's wrong towards some third party, you should be able to do so with impunity. "So what if I stole a car? You live your life and I live mine."
Mention abusing dogs or cats, or running puppy mills, and carnists bring out the pitchforks. What happened to "live and let live, man"? This is an argument that no one would buy if they watched someone cut the fins off of a baby dolphin and throw it back in the water to die slowly. And yet somehow, now we're all in on this philosophy when it's convenient.