The precise wording for the idiomatic expression is "I can't agree with you 【more】," and it means precisely that you agree completely with his opinion or idea. It's a very emphatic way of saying that you agree with someone. However, if you use "anymore" instead of "more"-- "I can't agree with you anymore" --- the meaning is entirely different. It means that although you had once agreed with his opinion or idea, something happened that changed the mind of you into disagreeing instead.