Can a person eat only one potato chip?
In other words, sleeping with your brother-in-law, even once, will not rid you of your craving for him. It will make you want him even more. As the French saying goes: The best way to stimulate your appetite is to eat.
Having a crush, even a HUGE crush is fine, so long as you know that you will never, ever, give in to your urges. To do so would be an act of murder/suicide: you will emotionally devastate your husband and destroy your marriage.
Since you apparently want to keep your marriage, then, I think you need to ask yourself some questions. First, why am I so obsessed with another man? What isn't working for me in the marriage? What do I think I will obtain/experience by sleeping with someone else? Why do I want my husband's brother? If I were to act on this urge, how would my husband feel? Hurt, angry, betrayed? Why would I want to impose such suffering on my husband? Am I angry at him and not aware of it?
After you ask all these questions and obtain a clearer picture of what is operating inside you, I think you will be ready to move to the next step: To fix what is wrong in your marriage, and to stop'massaging' your fantasy. By this I mean, you are tantalizing yourself by dogging your brother-in-law's path (spying on his house, etc.). And, the more energy you expend in pursuing your fantasy, the stronger that fantasy becomes.
So, if you want your marriage, focus on it, and stop allowing yourself to, literally, go down the road that leads to your brother-in-law.