Adaptive damage, jungling and first clear: it sucks.
There is no way to start with an AP item in the JG in order to shift your adaptive damage to AP. This needs to be addressed for AP junglers or they will continue to suck (assuming they are not overloaded with base damage).
It's probably one of the things holding fid back right now.