Assessing observer reliability for computer vision-based assessment of abdominal contractions during calving

ASSESSING OBSERVER RELIABILITY FOR COMPUTER VISION-BASED ASSESSMENT OF ABDOMINAL CONTRACTIONS DURING CALVING Maria Juliana Pereira1, Michael Iwersen2, Borbala Foris1 1Animal Welfare Unit, University of Veterinary Medicine Vienna, 1210 Vienna, Austria 2Clinic for Ruminants with Ambulatory and Herd Health Services, Ludwig-Maximilians-University, 85764 Munich, Germany BACKGROUND and OBJECTIVE: Effective calving monitoring is critical in dairy farming. Computer vision (CV) systems for calving monitoring use behavioural data from video, but robust algorithms require reliable ground truth from human annotations, an aspect often overlooked during development. Abdominal contractions are important candidate signs for automated calving monitoring, but definitions used for live human observation are hard to apply to video, highlighting the need for video-specific contraction ethograms that take camera perspective into account. We aimed to identify the ethogram–camera view combination that maximizes inter- and intra-observer reliability and is suitable as gold standard for CV. MATERIALS and METHODS: We used two videorecorded calvings to assess contraction-scoring reliability across multiple ethograms. Calvings were recorded with two synchronized cameras (lateral and top view) and we used a 2-hour period from each where three observers annotated contractions using the software BORIS. We evaluated three ethograms for abdominal contractions: (1) individual point events; (2) continuous bouts with start and end time; (3) continuous bouts by grouping contractions separated <10 s as one bout. Observers were equipped with a definition manual and reference clips. We measured inter- and intra-observer reliability using Cohen’s Kappa for each ethogram and camera view. RESULTS: Ethogram 1 resulted in low intra-observer agreement (κ = 0.30–0.57) and very low interobserver agreement (<0.10). Marking the exact beginning of contractions proved difficult and unreliable for annotation. Ethogram 2 resulted in high intra-observer reliability (κ = 0.85–0.94), but inter-observer reliability remained moderate (κ = 0.60), likely due to the lack of a clear definition of when contractions belonged to the same bout. Ethogram 3 was associated with high intra-observer reliability (κ > 0.9). Defining time interval bouts improved inter-observer reliability (κ = 0.71). The two camera views showed a similar reliability pattern with slightly lower Kappa values for lateral view. Top view offered full abdominal visibility and clearer abdominal wall excursion, improving scoring. Clinically, key calving indicators are the onset of rhythmic contractions and intervals between bouts. Based on reliability results, annotating contractions as continuous bouts is feasible for human observers and suitable for automated detection. Future work should test these ethograms across breeds and assess whether counting contractions per bout improves model performance. CONCLUSIONS: Abdominal contractions in cattle are best annotated as continuous bouts rather than point events. Top-view cameras can improve visualization and scoring reliability. Video-specific ethograms may help bridge clinical and computer vision needs. Keywords: Calving, Computer vision, Ethogram, Observer reliability, Precision livestock farming

• VetFarm

Ludwig-Maximilians-Universität München