I fell in love with my best friend and now he's gone, what do I do?

We were best friends for four years. We went everywhere together, did everything together...but we were just friends. At the time we were both married to other people. Then I could tell that our relationship was beginning to change. He got divorced, then I got divorced. He was a minister so we weren't intimate. I totally respected his position and would have never done anything to hurt his reputation. Many people umed that we were dating. We both would say that we were just friends, but there definitely was something a lot deeper happening between us. One day someone from his church came to him to say that there were rumours among the congregation that we were dating. That night he ended our relationship. He said he had to figure things out. He decided to leave his ministry and then left town and told me not to contact him. I haven't, but I hurt so much. I don't understand how he could change so quickly. I think he was afraid of his feelings for me. What do I do?