If the consequentialist says, "You are morally responsible for your the causal consequences of your actions (or inactions)", my response is what do you mean by "morally"? If the consequentialist says, "In situation Y when choosing between A which would lead to X or B which would lead to Z, one ought to choose A", then I would respond what do you mean by "ought"? The same goes for other morals terms like immoral, unethical, duty and should. I think at best these words are equivocal, referring to any one of numerous possible things that could be more clearly described amorally. Alternatively, they may refer to some vague mystical claim regarding some insentient valuing system apparently exiting on some other realm of metaphysical existence -- nothing I see any reason to believe. My objections here apply not only to consequentialism but to all moral claims.
However, in amoral terms, I think in situations in which people do things that we feel some need to make judgements, such as a corporation setting policies for its employees or members of a political democracy creating laws which they will violently enforce, then we generally find it necessary and most effective to use methods amorally corresponding to all classifications of so-called moral theories: consequentialism, deontology, and virtue ethics. For example, in the legal system of most countries today such as the USA, for a person to be convicted of a crime they must (1) break a strict rule (i.e. law) to do or not do something specific which I think corresponds to deontology, (2) in court, prosecution must show the actus reus which I think corresponds to consequentialism, and (3) in court, prosecution must show the mens rea and intent which I think corresponds to virtue ethics. In practice, we make judgements based on all 3 ways.
Normative moral claims seem to claim there is some other mystical objective way in which things actually are valued or matter and that only one of those three forms of such judgement (or some other form of amoral judgement) matters in that mystical objective way. Other times when people use the moral terms equivocally to refer to things that can be described more clearly with amoral terms, they may simply be using the terms to refer to a specific form of action, but in such case there is no debate about whether morality is consequential or deontological rather it is simply a matter of equivocation like using the word lightness to refer to non-heaviness as opposed to non-darkness.