I wouldn't say kick him out ASAP, but I'd certainly say that to refrain from reporting to the police is unconscionable. Let the justice system take whatever action they can/need/want to take. Perhaps grant absolution upon completion of whatever parole or prison time or whatever the penalty is - and, after all, he may be found not guilty anyway.

False accusations are/have been a problem in nasty divorce cases. In this media-frenzied pedophile-priest environment, while there is certainly evidence that some priests have engaged in terrible behavior and while it can be shown the church has done its best to cover it up, and engaged in unconscionable behavior themselves by moving priests to different parishes rather than going to the police or executing other punishments, I can certainly see the possibilities involved in opportunists deciding to make up a case against a likely priest.