If the teacher is not interested in listening to your concerns and working out a solution, you should go and speak to the principal. If other parents in the class have the same feelings, then perhaps you can go as a group to voice your concerns. The principal's job is to ensure that every student at the school is being taught to the best of their ability, and if this teacher is not doing that, they should be made aware of it. If the teacher doesn't change, ask the principal about moving your son into another class.
If, after that, the principal refuses to acknowledge the problem, consider taking your son out of that school. At the very least, you are not being respected or heard, and at the worst, it could be a very bad, even dangerous situation for your son. The mid-year change may be difficult on him at first, however you could ease the transition by taking a "sick" day from his old school to introduce him to the new school. Have him meet some of the teachers and students and have a buddy show him around the new school.
If the school is a CCMA accredited school, then you can also go to the association and report the teacher. They assess the school curriculum and teaching philosophy, but this seems to be more of a personality problem, something outside of their realm.