That's something that I have always struggled with. What seems to work the best for me is multiple smaller boxes that accommodate specific tackle.
I have one medium size box that I keep hooks, sinkers, corks, swivels, pliers, line, etc... in.
I have a few compartment style boxes for jigs, crankbaits, topwaters, soft plastics,etc...
My medium size box above goes on every trip but I have species-specific boxes that I take when I target different fish. One for crappie, a few for bass, another for trout, another few for stripers, one for catfish.
Then I have the boat bag which is a catch-all for the bigger stuff that goes on every trip.