Don't think 'commercial' has anything to do with it, to be honest.
I just don't think he's been anywhere even close to Alien or Blade Runner in nearly 3 decades. You're always going to get a quality finished product from Scott as he's nothing if not a polished visual stylist who's in the enviable position of being able to work with big-name actors, writers and crew but there's precious little in his cannon that lives up to what I think he's capable of. Thelma & Louise maybe, Hannibal certainly (yes, I do like it a great deal on its own merits as it happens
) but Gladiator? That Crusades Thing? Body of Lies? American Gangster?
Well made soma.
EDIT - I'm pretty positive that Sir Ridley will cry himself to sleep for a solid month when he hears of my faint displeasure but, hey, I'm not only here to spread joy.
Hannibal is rich, intelligent and, as you say, darkly funny. It also has Francesca Neri...
Even though I've owned it on DVD since release it was only £6.99 on Blu Ray and it's something I'll watch once a year. He does some of the best commentaries in the business, too, and Hannibal's is one of his finest.