What to do when you know a sitter is unfit?

It sounds like you've already contacted Rover about the issue. I would hope that they would begin an investigation, but if that doesn't seem to be working I would recommend contacting your local animal control to inform them of the loose dogs. They may not be able to do anything unless the dogs are actually loose at the time you contact them, but if it's happened twice and the problem isn't being addressed, it will probably happen again. If you contact animal control rather than just turning the dogs back over to her, they may cite her for not having her dogs secure and require her to get the fence fixed, especially if there's a history of it happening before.