My point on difficulty was mainly to point out you can jack around quite a large bit before it negatively impacts performance. Just because someone is obnoxious while bothersome doesn't imply you are carrying their weight. It may be you are piggybacking or their aloofness could be causing you to play worse in which case they may troll harder it may be none or a large sum of these things.