and my last job was doing outsourced pensions.
Those small payroll companies are long gone, out priced by the larger ones. But I can give you one caution in how the problems there were when Mellon owned an outsourced pension company
When I went to work for for an actuarial firm, they were not too efficient, but I saw them grow in client satisfaction as we brought the company into the 21st century with standardization of practices. Then we were bought by Mellon, the first thing they did is bring us back in time by insisting we all use one system, and they chose the one that was back in the dark ages since most of their clients were on that system. Every thing they did after that poor conversion and anger by our existing clients who found the changed system error prone, was toward outsourcing - they violated contracts and sent social security numbers to India and used live data for testing - all real professional no nos.
The odds of identity theft seem to go up when you send personal information out of the country, so you might want to check on that and that does not mean just ask them once. They then gave into client requests and met ridiculous deadlines, by installing the system before it was fully tested - i.e. live testing - and they ended up spending a lot more time fixing data.
So - 1. ask where the programs written/reside - if it is overseas, ask where it is tested and if they have access to live data.
2. Parallel test until you are personally satisfied, do not leave it up to the company, you should have at least one month of parallel testing ,and should get some annual tests in there as well.
3. If you are looking at EDP - they used to keep their dates in code (???) as the number of days since the founding of EDP - this means they had to multiply and divide to figure out the date in processing and they just ignored Leap Year and dates turned out not so accurate. Make sure they can deal with leap year.
Check with other people who use the systems to see how satisfied they are, the company should be able to give you other client names to check,