No wonder you're confused!
This guy came on really strong in the beginning. He wanted you to come over every day, and when you didn't come he pursued you.
I'm afraid that this guy is conflicted about becoming too intimate with another person. He actually said this very thing to you. He said he needs space and that your visitations and calls crowded him. Don't think you did anything wrong. He asked you to make the contact, then he freaked out! I'm so sorry for you.
If you are still talking to him, you might want to tell him that you discussed the problem with me. Tell him that he's obviously scared of too much intimacy and closeness. Tell him that breaking up will ease his feeling of anxiety, but it won't solve his problem.
It's simply running away rather than facing his fears and resolving them. You might also tell him that he knows very well how right you are for each other and that this is a gift that doesn't come along every day. He can spend his life running, or, if he's interested, he can face the fear and heal it.
The choice is his. If he's interested in healing himself, then have him visit my site and do the Personality Profile. The profile will identify the causes of his intimacy phobia and give him steps to take to heal it. I hope that he wants to work on himself.
Meanwhile, in the future, I would recommend that you 'check out' the next guy you're interested in. I have another consult called, 'Is He Ready for Love,' which will help you recognize an intimacy phobic (as well as other problems that would make a relationship impossible) before you become deeply involved and your poor heart gets broken.
I would also recommend your waiting to have sex until you are more sure that the person you care for is a suitable candidate for a relationship. I say this because sex intensifies your level of attachment. I don't want you to become attached to someone who isn't capable of loving you back.
Take care of yourself.
You are a lovely woman who is capable of giving and receiving love. Protect the gift that you are.