He is shutting you out. You aren't crazy!
It sounds to me as though your husband has a fragile ego. He is avoiding emotional discussions with you because he expects to be told that he is failing you in some way (why else would you be asking to talk about the relationship). Since his ego can't handle the blow, he avoids the discussion all together.
You feel unresponded to, hurt and anry, which makes you reapproach him with more intensity.Your annoyance and demands convince him that he's going to get reamed out, and so he avoids the talks even more. I can tell you that this vicious cycle is common and easy to solve.
You need to read my book, Till Death Do Us Part (Unless I Kill You First) which will show you, step-by-step, how to approach him so that he will won't feel threatened and will want to listen. If I could lay this all out to you in one short column, I wouldn't have had to write the book! Take the time to read the book and put the steps into practice. if you follow the proven plan I outline in the book, he will be welcoming your talks in no time.