The healing process for something like this will not involve only specific individuals. It will involve everyone in the entire church, split and all. It is your place to forgive him in the part where it involves you, but for all the rest, it would need everyone else to settle it for sure. Those specific individuals especially, and the pastor himself, would have a lot of prayer ahead of them, individually and corporately. Not only that, he'll have to accept responsibility if he really is responsible. He admits to making mistakes. Now he has to correct them.
You're going to need more than handshakes and apologies for this one. You're going to need hours of prayer and worship, a touch from the Holy Spirit on every individual specifically involved, and you're going to have to face especially repentance from that pastor of yours, not only to the church, but to God. That is the price of leadership. He will face the hardest burdens, because it was he who was placed over the people. A hard task, but probably necessary. It may take a few years, but it can be done.