In defense of the Shadow Isles, they are more organized than the Voidborns and their attacks towards the humans are more habitual than the attacks that the Voidborns do. And by that i mean invasions of a place, not a random creature biting a dude.
After all, the Harrowing occurs almost all the years, and it can happen anywere, anytime, not only in Bilgewater or October as some people think... In other hand, the Void "invaded" a place only once, and it was because they were "invited" by the Icathians... sure, they destroyed the place, but is not something that can happen in any moment.