I was trying different clock speeds to try and save power on a project, and I found that changing the clock speed to anything less than 600 kHz rendered the device unable to program through the IDE. I tried using both the avrisp and avrisp slow but they mode no difference.
In order to use the device again, I had to reset to the default fuse from the command line with
avrdude -c avrisp2 -pt13 -B 100 -U lfuse:w:0x6a:m