While I personally completely support having more build diversity on champions, when it comes to diversity through AP and AD, I think the question to ask is: what are the playstyles you actually want to engineer?
When League was originally designed, the AP/AD dichotomy pretty much followed what's described in the OP: abilities scaled pretty much exclusively with AP, attacks scaled with AD, and so there was a clear difference between casters and attackers, or even caster and attacker builds on the same champion (e.g. AP versus AD Yi). However, that model fell apart very quickly: when you balance AD around nothing but autoattacks, which unlike abilities are pretty homogenous across all champions, AD builds become balanced around classes with autoattack steroids, i.e. fighters and marksmen, and are unviable on everyone else, which defeats the purpose of customization. Additionally, many autoattackers also incorporate abilities as part of their damage output or vice versa, so an AD/AP dichotomy generally means half of how they're meant to play inevitably feels weak. Ideally, the solution to that should be to build hybrid, but multiplicative scalings like crit, penetration, etc. means that hybrid builds are weak by themselves, and need to be boosted in such a way that some champions end up forced to build hybrid, or at least rely on one hybrid item (e.g. Akali and Hextech Gunblade).
In addition to AP and AD not being particularly good at enabling build diversity on champions, I also feel trying to enable both AP and AD builds ends up generating only shallow diversity: sure, some champions, like Ezreal or Shaco, can accomodate multiple playstyles, but in the end, unless you're willing to use AD and AP scalings more creatively, making all abilities scale with AP and restricting AD to autoattack power means you're only enabling pretty much the same two builds on every champion, i.e. an attack build and an ability build: even now, in spite of AD-scaling abilities, marksmen tend to blend heavily into each other due to how they all focus so much on autoattacks, which play mostly the same across all members of the class, and generalizing that across more champions risks making them even more homogenous. It's also why some builds like AP Miss Fortune, AP Lucian or AP Aatrox feel shallow, because building them AP basically just makes them do stuff that they can already do with an AD build.
In the end, I think the way forward for more build diversity is going to be to move away from stats, and more towards bespoke bonuses, perhaps even directly add different ways of playing a champion to their kit. Stats were originally a means of giving champions more customization, but now they make builds more rigid, and the whole notion of "stat binding", i.e. tying specific stats to specific classes, is making itemization progress towards a logical end where only a small subset of items will be viable at any given time on any champion. Moving away from more generic bonuses and onto effects that are specifically tailored towards each champion could not only enable greater diversity, but could also make balance and design significantly less costly in the future, in spite of the initial development cost, since there would be significantly fewer variables to account for.