Wouldn't that be even more confusing in other edge cases? If the first path entry acquires something that matches, but you have a matching entry in a later path entry cached, you'll execute that entry no matter what unless you abandon caching altogether (or use inotify or sth to invalidate it, which iirc would introduce races).
@robryk I'm not particularly a fan of path caches in general (for exactly this reason among others), but if a command would otherwise fail entirely, I would like bash to recheck $PATH. Either that will still fail or bash is making me go through extra manual steps with a forced failure.