The problem was that the connector to one of the battery terminals had cracked and as a result wasn't connecting well enough to complete the circuit.
I've been meaning to join RACQ for a while, so I decided this was enough of a reason to. Unfortunately the free 24 hour road-side assistance they provide to members doesn't kick in until 24 hours after you join. I rang them anyway and for a measly $28 one of their service persons came out and fixed the problem.
A quick look and he said it would be easy to fix. The connector does a loop around the terminal, and it was part of that loop that was cracked. All he did was unscrew the bolt holding the loop tight, and screw it directly on to the terminal bypassing the loop completely.
He then checked the level of the water in the radiator (fine) and my oil (not fine) so I went and got the oil and he filled it up for me.
Jeff now starts perfectly, and I'm feeling a lot better.