Emotionally, those are two good points.Subsidies capped at £50,000 would be a start.
Farm houses designated for farmers only would also help
Practically they falter.
The Common Agricultural Policy was conceptualised to harmonise wage rates across a large trading bloc. To define a spurious cap on the level of payment of a grossly distorted legacy of the original subsidy is arbitrary at best, if not mendacious from some viewpoints.
Local councils have no chance defining ab initio if someone intends to farm or not.
Unless farming itself is a justifiable monopoly occupation, you are trying to define the unknowable.