Pratical Solution For Boards Downvote Problem
Just create a point based system that will fix the problem by weighting votes. It's simple, keep tracking of people that upvotes topics very relevant and mark those players. He loses points when he upvotes bad posts or downvotes resonable or good topics. He gain points when he upvotes good posts or downvotes unresonable ones. Make all the "you lost relevance points" very clear so everyone can question and know what is happening. Each "X" (where X is really big) points make his upvote and downvotes worth as 2, and then 3, then 4, then 5, up to a max like 10. This way the small number of good and resonable players will weight more good topics and the balance is restored. Also make very clear who upvoted what (or just a report button), so the community can question abuses of the system.
That's it. Thank you!