Can you cancel a booking if the dog is not a good fit for you?

We had an initial meet and greet where we met the owner and the dog. We agreed to book for 3 weeks while they go on vacation (they leave in 2 weeks) Today, we are looking after him so he gets a feel for the place. The owner did not tell us that he is afraid of buses and loud cars. Unfortunately, you can hear all the buses and street while in our backyard and in our house. He seems very anxious and unhappy. He also was very destructive in the first 10 minutes of being here. (starting ripping a pillow, and tore apart a reusable bag when i left him in the house alone for 5 minutes) I'm wondering if it's ok to cancel as a sitter, because I don't want him to be extremely anxious and unhappy here for 3 weeks. :(