As a relative newcomer to boat ownership, here are my thoughts...
As boats get longer, they get higher and wider too (derr!), which is why, as others have said, what looks like a relatively small incerase in size, actually gets a substantially larger boat.
I wasn't sure if boating was going to stick with me, so around this time lat year, I bought a small (3.75m) tinny figuring that I would get most of my money back when it came to time to sell (to get out of boating or to upgrade). And while I got my purchase price back, I didn't get back the money I spent on improving the boat. I had added a bimini, a cover and had spent time and money on replacing the water pump and re-sealing the lower unit of the outboard. If you decide to enter boat ownership this way, get a newer/more recently serviced outboard than I did!
Towards the end of last summer I upgraded to a 4.9m fibreglass cuddy cab, which has room for SWMBO to escape the sun if she chooses. It's big enough to have four people comfortably on board. The 60 hp donk is adequate. More hp might have been nice, but I enjoy trolling so it's not really a bid deal.
The bigger boat is still small enough that I can launch and retrieve single handed with ease. If anything it's easier than the little tinny. Go figure!
Your budget will give you a great choice of boats, and I would definitely go larger rather than smaller.
As with buying anything from a private seller, buy the seller as much as you buy the boat.