Does ultimate hat reduce the ultimate CD beyond the 40% max CDR cap?

Yep! A little more context on how this works below.

The Ultimate Hat reduces your ultimate's cooldown by up to 15% (also known as multiplicative CDR) - this is notably not the same thing as adding 15% more CDR. Whatever your ultimate's CD reads after applying CDR, multiply that by 0.85 and you've got your post-Ultimate Hat cooldown.

This decision was made so that the hyper-scaling properties of CDR don't impact the decision-making around whether or not you want this rune. Otherwise, the rune would be a worse choice on champions who didn't build much CDR (like Shen), even if reducing their ultimate's cooldown is clearly desirable.