During the ruby fight at spawn the enderman seem to just like to get angry at anyone. As i know i have and other people have died just trying to kill the enderman before the fight. If they could be removed from this end it would not affect the fight as they do not get any power from the ruby dragon. (or if they got ome power or something from the dragon)
This would not affect the wildercraft theme as the goal is to kill a modded dragon.
I dont know how hard it is to remove a mob so like a 3-4.

I honestly love to have enderman in the end, It makes the fight more interesting plus we can get epearls while fighting to teleport to the ground when the dragon makes you levitate. Enderman is essential in the fight.

I guess the biggest thing is that you could be looking at the ground and the enderman get ad at you, dont know if that is just some wierd minecraft thing or just codes getting mixed up