Okay, I'll weigh in.
I find fanatics to be my new secret weapon, like fire they are a terrible foe and a fearsome servant (thanks George Washington). I think they would be troublesome to use as skirmisher defense because you have essentially no control over them. If you were able to set them up in an ideal position they would kick butt, but so does the goblin bomb chucker if you roll well.
My advice for fanatics is always buy lots, if your going to gamble you might as well go all in. I like to throw a troll in their midst every once in awhile too, Its like arming your trolls with pinchers. I also like the poetic cadence of the strongest and weakest working in scary symmetry.
Wolf riders would be my premo choice against skirmishers because they move faster so they do more damage and at 185 points they are one of the more reasonable anti-skirmisher units. Historically, I have rarely taken wolf riders, so James has usually had his wolfkin way with me, but if I were going up against Rome or Carthage I would definitely get me some wolfriders to ply their toothy trade upon the skirmishers.