Even if you think you are vegan, you are still using plants that were grown using animal manure as fertilizer. True veganism is impossible for individuals now because animal products are used in growing plants
As explained here, animal fertilizer doesn't cover the total fertilizer needed. So if you're vegan, and you're thinking about eating a farmed animal, then all you'd be doing is increasing the amount of animal fertilizer that is used. In other words, by eating fewer animals, you're decreasing the reliance on animal fertilizers to grow crops. So this is an argument for more veganism, not less.
Ultimately, if no one farmed any animals, then we wouldn't be using any animal fertilizer. So in that case the simple principle of "be the change you want to see in the world" would obligate you towards veganism if you believe this argument.
Animal manure is not a high-quality byproduct. If not recycled, it would pollute the waterways more than it does now. So, again, if we all didn't eat animals, this would become a non-issue. This problem only exists because of carnists.
What this argument is really saying is if you're engaged in 1) slitting animals throats and 2) using animal fertilizer, then because you can't stop doing 2, you shouldn't stop doing both 1 and 2. This is just another appeal to futility, and it's founded on "If you can't be an imaginary puritanical vegan that has zero impact on the world in any way then it doesn't make sense to stop deliberately harming animals." We wouldn't accept this argument if we found out that a friend had a dog chained in his basement he was beaten for fun, so we shouldn't accept it here.