Having built and worked for a small family business for 25 years now, I totally understand were you are coming from.  If your safety is in any way an issue - document what happened, report it, and absolutely leave.  If not - Have you spoken to someone higher in your dept?  Or someone that you have a good relationship with at the company?  Family businesses can be very dysfunctional and still do well as a business.  Family businesses tend to have a different dynamic for sure.
I worked as non-family for other small family businesses before too, and in two cases, the executive family members wanted to live like they were rich, and bankrupted the company in the process.  I was the purchasing agent, and I was blamed when I could not get materials for production in time.  They did not pay the bills to the suppliers, and the suppliers would not send more materials. On one - I was fired for not doing my job properly.  The day they fired me, my immediate supervisor told me what a great job I had done and how much I had improved my dept., then handed me the pink slip saying I was fired for not doing my job.  I found out later it was because they were struggling to pay the health insurance, and I had become a diabetic.  They could not afford that on their plan, so they fired me.  I also found out that the general manager - also non-family - had gone to the owners and told them that if they fought my getting unemployment that she would go to court for my side.  The  point of this is  - that you must have made some friends there during your time.  If you have, you might talk to them, and to any connections you have with the family members.  It could be that family members are slacking in their jobs, and not doing what they are supposed to be doing because they think they will slide by.  We have dealt with that here too, and even though it impacts sales and customer service a great deal at times, it was still tolerated because of who they are.  The biggest problem we seem to have had is that when we do have a bad employee, it is almost impossible to get them fired - even non family - unless there is overwhelming evidence.
If there is someone there that you work directly for, then I would start there. They may not want to lose you as an employee, and may not be aware of the problems you are dealing with.  Document everything, by all means, daily.  If you are fired, you will have proof and also be able to protect yourself in case there is an issue about it with another job.  Leaving with a good reference is always the best, but if that is not possible, having a solid reason why you are leaving is good.  
There are always going to be some problems with every job, and people you don't enjoy working with or for - that is true.  My worst working experiences actually were when I was a Buyer for a very large aircraft corporation, so corporations are not always better.
I wish you the best of luck - whatever you choose.