Co-text-Aware Croatian Sentiment Annotation Using Open-Source Large Language Models

Sentiment analysis is typically performed at the sentence level in isolation, a practice that ignores the discourse co-text in which utterances occur. We present an evaluation framework for co-text-aware sentiment annotation in which each sentence is classified by an open-source large language model (LLM) under four co-text conditions: full document, co-text window, initial document sentences, and no co-text. We evaluate 9 LLMs on a newly manually annotated Croatian movie-review dataset, using macro and weighted F1. Results reveal that co-text provision does not substantially improve sentiment disambiguation across models, while the model size is a crit- ical factor: google/gemma-4-31B-it achieves the highest score (macro F1: 56.36%; weighted F1: 80.53%), followed by google/gemma-3-27B-it (macro F1:53.97%; weighted F1: 77.57%). Other models struggle to exceed 50% macro F1. Error analysis of the best model shows systematic difficulties with the mixed and other classes, while positive and neutral sentiments are classified more reliably. We further fine-tune the best model using QLoRA, reaching a macro F1 of 57.51%, and show that a concise prompt outperforms definition-heavy and few-shot variants.