How significant is that pause before "yesterday"? Doesn't it mean John has almost forgotten, or at least, has to think a bit to remember, that he did meet Sherlock only the previous day?
As if life before Sherlock is a distant, or at least blurry, memory.