Informal fallacies seem to me to be, at best, fuzzy guides to logic because of the role they often play as inductive and/or meta-arguments.
A central difficulty in the analysis of informal arguments similar to the ad hominem is the problem of formulating a criterion of relevance. Under what conditions is a conclusion irrelevant to its premises?
For example, if someone argues that Bob's argument is incorrect because Bob is an idiot, then that argument is plausible if, indeed, Bob is unintelligent.
It does not help to point out that this ad hominem is mistaken because unintelligent people can make good arguments just as well as intelligent persons can. The ability to argue well is, c.p. one condition of being intelligent.
One seldom noticed point in this regard is the problem arising when an ad hominem fallacy is not a personal attack, an insult, or a negative name-calling. For example, to argue that what someone states is correct because she is brilliant would be the same sort of logical proceeding because it involves an appeal to the person's character or circumstances.
Even the identification of this argument as an ad verecundiam fallacy would be a mistake because the quality of intelligence is c.p. relevant to an ability to argue well.
The open texture of language and the nature of inductive argument makes a precise identification of informal fallacies in many cases difficult. Sometimes it's best to see informal fallacies as a failure in deduction rather than a failure of logic.
For example, if a Nuclear Engineer who is a expert in the field of environmental safety argues in favor of the efficacy of nuclear reactors, do we discount what is said on the basis that he might be prejudiced, given his background, or do we endorse what he says because he seems to be an expert in the field, or do we ignore his background because we should assume that anyone can argue equally well about nuclear energy?
Again, I think sometimes it's best to see informal fallacies as a failure in deductive logic rather than a failure in (inductive) logic. Character and circumstance described in premises may be often probabilistically related (or causally) to the states of affairs described by statements.
Even so, I'm not claiming that there are no clear cases of ad hominem fallacies.