The rift vs my trinket

Hey everyone, I don't post much at all, but here it goes. I am curious about the design decision to put the rift spawn in place of my trinket. It kind of feels like Im penalized for getting rift. Almost every game I get rift, I end up holding rift for a few mins b4 using it. But all this time, I can't ward. It really is kind of frustrating.