Poor Jungle Yorick and Creature Agro

I went jungle Yorick.

When I pop 1 ghoul and the ghoul dies the jungle creature gets mad and its agro timer starts. The good news is I killed my creature.

The bad news...I eventually had 3 ghouls. When I pop 3 ghouls and tried to get the blue buff on my second pass I could not do enough damage fast enough before the agro timer reset and it started getting life back.

I went around and killed some people in lane bought some items and went back to the jungle and was fine, but I assume this is not intended..unless the rumors of your hatred of all things Yorick is true......