Aled Samuel visits the Vale of Glamorgan to see the lavish gardens at Duffryn and the small and perfectly-formed Cowbridge Physic Garden.