Champ read my statement again, i said ive been to BRowns before and the Broken Bay Fad which is 22km out, i know how far out Browns is as i have been there a crap load of times, in my 5m Quinnie, in Rivvy's, in all sorts of boats! with no trouble at all, it has blown up out there, just took a long time to get home, i would in some conditions rather be out there in my boat than a 40ftr lets just say!
Yes the sea can change, but with fairly accurate forecasts these days even if they sometimes get it wrong, it is very doable to do it, ive seen sub 4.5m boats out there on the calm days, people even solo, so in answer to the question it would be doable with the right forecast, enough petrol and a good captain, although no fish is worth risking ones life for.