In most instances, the first line of defense would be to outline library collection development policy.
I would then refer the challenger to passages from the following ALA resources*:
*I have included additional defense resources for each book entry.
Some of the books in my collection are more likely to be challenged than others. Rather than collect reviews on every book included, I would hear out the complaint of the challenger, have them file a formal complaint if they are not satisfied with my initial response, and then conduct research to find reviews supporting my choice to keep it in the collection.