From your text it is one of the two pivot, rather than the four clamping, bolts which is the problem?
How old are the seals – they deteriorate with age. I could send you a single replacement that I have in the garage if you want? (Not a set). I don’t think grease / oakum would help but a new seal should.
You should be able to remove the nut and seal when afloat with only a small amount of water ingress. However if the bolt is out of place due to mating with the ‘stopper’ on the plates, it might leak more.