Ferry wharves aren't THAT crowded on a Sunday in my experience and you fish on the side away from the ferry anyway.
Just be tidy, take up minimal space and you'll be right. We often get ppl coming for a chat or just a look see.
Be aware some wharves have no fishing before a certain time I think eg Abbotsford is a nogo before 10am
Good luck, hope you and the kids have a good time!