I believe it’s ok especially when the client is never happy or they may ask you to do something illegal, unethical or immoral in the business setting that may affect your integrity or  business reputation. It all about how you handle it and if you will compromise your values. I don’t, I’ll explain the potential liability and/or outcome so they are aware, however I will only do what’s right for their business and employees. 

Maintain your position, evaluate the potential pushback and make your decision at that point what’s best for all involved. The client will respect you or maybe it will be a reversal of them ending your contract. 

Anyway, review the circumstances, evaluate your position and those involved, don’t make an emotional decision, think it through thoroughly, how will it affect you and your business, is there a compromise, then have a planned agenda of your concerns and have a meeting with your client and see if you can attain an amicable agreement. 

What’s your threshold and how much is it worth to you.