FPS problems during stream

I stream using OBS and whenever I stream league it starts fine for the first probably half an hour and then the FPS starts dropping. I can stream Overwatch, R6 and PUBG/Fortnite with no problem but I cant stream league well at all.

Here are my PC specs:

Samsung 840 EVO 250GB SSD ASRock Z97 Extreme6 Intel i7-4790K 4.0GHZ 16GB DDR Ripjaws RAM Zalman CPU cooler SeaSonic 620W Modular PSU MSI GTX 970 4GB

Here are my video and output setting on OBS:

Output mode: simple Bitrate: 2500 Encoder: NVENC Audio Bitrate: 160 Encoder Preset: High Performance

Base resolution: 1920x1080 Output resolution: 1152x648 Downscale filter: Bilinear FPS is set to 20.

any help would be appreciated.

also i set the affinity to OBS to the 5-7 cores and I've changed the settings on my graphics card to high performance.