Sorry for the late-ish comment but yes, I believe this is "working as intended" even though it's pretty much a straight up gimp to Aatrox when his clear power resides in close 1 on 1 combat and it restricts any forms of use of Blades of Torment up close. I've noticed this awhile back but never loudly voiced my concern seeing as how Aatrox is seemingly forgotten amongst the community and Rioters. The reason Blades of Torment fails to work in your given scenario is as I believe, tied directly to the "updated hit detection" and "missile hitbox" that was brought up not too long ago. If you dig deep enough, you'll see the balance team messing with how projectiles work and as a result, many skills were affected in the process. The reason Aatrox's Blades of Torment does not hit the enemy up close is because the middle section of the indicator is actually blank space and no longer a part of the projectile's hitbox As a result the entire hitbox of the projectile resides solely on the twin lasers that come out of blade on use and hence, why you could not hit Renekton.
Take this with a grain of salt because maybe it is indeed a bug, but from my limited memory the changes to hit detection and projectiles was what made his E as you said, "unreliable" and "worthless".