You're afraid to admit how much you like this guy, since you aren't sure whether he likes you as more than a friend. The result is your trying to be a psychic and divine what he feels for you based upon how he behaves during and after sex. Because we aren't psychics, we can't begin to guess what's in his head and heart.
If you want to know where he's coming from (no pun intended) you need to bite the bullet and ask him. Before you ask, you need to identify what you're afraid of. Are you afraid that he doesn't reciprocate your feelings and that you will feel hurt and rejected by asking him outright. Are you sensing that this guy is afraid to be pinned down, hence your reluctance to ask him where he stands?
When you discern where your fear arises from, then formulate a question that addresses the fear. For example, you could say, 'Why am I afraid that you will run for the hills if I ask you to tell me how you feel about me and the relationship.' Asking the right question opens up the discussion. Then when his cards are on the table, you can talk openly about his fears and resistance to admitting that you're more than friends.
Talking this out is the path to resolution. Let me know what happens.