The year is 2505. Everyone is vegan. We have a sanctuary with some cows that aren't forcibly interacted with. What's wrong with milking one and selling the milk?
Ok, so here's the plan, you now need to keep cows for at least 4 times their current lifespan - well past when they make milk. You also have at least twice as many animals (males are now not automatically slaughtered). So you're looking at more than an order of magnitude reduction in the amount of milk right off the bat.
So without artificial insemination, you're relying on the natural reproductive cycle of cows to determine when to drink their milk, which the calf is cutting into, and the cow only is producing significant milk for a relatively short period of its life, maybe a quarter - if you're lucky. If you're looking at grass-fed only and you're not artificially inseminating the cows to produce the milk (because you've somehow reasoned yourself out of bestiality), it's even worse on the sustainability.
With no slaughter market to keep such milk profitable, there will be orders of magnitude fewer animals (for reference, there are about 30000 total American Bison alive today, which is how many cows are slaughtered in America in just 8 hours) on average.
Then let's do this calculation: just in the US, and I would be generous and assume 100000 cows, but I'll even be generous and give another order of magnitude. A million cows. So let's say even half their lifespan to 10 years to be more generous (so that's 100000 pregnancies per year). And with all this generosity you're looking at a cow pregnancy rate (and therefore total milk production rate) of slightly less than 0.3% of our current rate, assuming we avoid practices like mother-calf separation or forced milking, etc. which could each reduce the production by an order of magnitude each... but we'll ignore.
So let me get this straight, the plan here is to avoid going 100% vegan by going 99.7% vegan on milk, and 100% vegan on everything else. That's all with the heaviest lowball of all time on that percentage - In reality, you're looking at significantly less. If one relies on this justification, then it covers the drinking of milk once every few years.
I'd get into more arguments about how selective breeding is harmful to the animals and therefore things would remain unethical, but why bother? We're now pretty deep into fantasy land here; this alternate universe has nothing to do with reality. One can't use this as any sort of justification for using animal products in the real world, where it involves the active torture and slaughter of animals.