Where You Say Matters: A Study of Positional Bias of Small LLMs

Large language models (LLMs) have performed very well across various natural language tasks. These are now increasingly being integrated into handheld and IoT devices. Understanding inherent biases and responses to varying prompting techniques becomes more important. This study examines positional bias using multiple-choice questions and answers using seven small large language models, including models from Microsoft (Phi), Meta (Llama), Google (Gemma), Deepseek and Alibaba Cloud (Qwen). Using the OpenBookQA dataset, our study shows statistically significant results, such as Phi models that demonstrate a preference for D, whereas Deepseek has a significant bias to A. Standard prompting techniques such as Chain of Thought and few-shot prompting do not improve performance and, in some cases, degrade performance. These results highlight the need for bias-aware model selection, especially as these models are increasing in our everyday lives.