I told the vet about it, and he did some research and all indications seem to point that this ant dust is the culprit. The active ingredient is called Permethrin, an insecticide. It's save for use with dogs (it's actually used as a flea treatment for dogs) but not for cats. It's a neural toxin, and results in muscle tremors, weakness, convulsions and death. Thankfully Spitz is only showing the first two, so he should be fine.
There is no direct treatment. Recommended treatment is a drip to flush his system, and give medicine to treat the tremors, and he should be fine in about 3 to 5 days, so that's what the vet is doing. This means Xavier is going to be lonely for a while.
When I went and grabbed the cat carrier from the back room yesterday morning, I noticed the door to the room was open. Usually it's kept closed (I keep all the stuff I haven't gotten organised in there). I ignored it at the time, but what I suspect happened is that the last time I was in there the door didn't close 100% (easy to do, it's an old house and none of the internal doors close easily) and one of the cats managed to push it open. Spitz wandered in Wednesday sometime and found some of the ant dust remaining and did the things that all cats do: he tasted it.
The permethrin entered his system and and currently causing problems, however it should all flush out without any long term issues.
Now I've just got to wait. I'm going to visit Spitz this evening on the way home, and probably every day until I can take him home.