Well, it's just a cheese build. It does have counterplay, opt for a defensive item such as a Guardian Angel or Stopwatch or Hourglass or something of the sort. Even straight armor would work most of the time. J4 is all about burst, but he has little sustained damage.. If you have a team supporting you, they won't just let a squishy J4 with no cooldowns stand there and auto attack you. Dodge his E Q, it's fairly telegraphed from the longer ranges but it's a huge chunk of his damage, so get good at side-stepping it. Unless he's super super fed, he won't kill you with just an R auto. J4 falls off hard late game in terms of damage, and he's super squishy if he goes full damage. You need to focus him after he E Q's into an aggressive position if he's going a lethality build, because he'll be super fragile. Jarvan is meant to be played as a tank or a bruiser, not an assassin. Subsequently because of this, he's going to have shortcomings in his kit if people are trying to play him that way. Honestly a fed assassin such as Kha'Zix or Rengar would be much scarier than an assassin play style J4. You can flash J4's E Q combo or his Ult right away and it wouldn't even do damage. Just focus him, seriously. After he blows his E Q or Ult on a teammate that's your green flag to unload your combo on him. He is so squishy with a lethality build. You don't see people playing him lethality in higher elos for a reason. It's just a low elo stomp sort of thing. The dude just evaporates under focused fire if he decides to go lethality and teamfight like the champ is supposed to.
I'm a 400k mastery J4 main, feel free to ask me any questions if you need help.