delay cast only in party

so when im playing solo i dont have any lags and the ping is fine. when im playing with a friend in a party i have delay with casting the spells and it sucks. the ping and the fps is the same. when im walking around the map its all good. the only thing is with casting spells. terrible delay that makes the game way more difficult like that. any solutions? thanks!