LLMs exhibit a 'Lost in the Middle' phenomenon, where the model pays more attention to information at the beginning and end of the context while tending to overlook information in the middle.