The tool creates a visualization of the video annotation matrix. It also converts a labeled video matrix into a Fleiss Matrix. Finally, it calculates the Overall Fleiss Kappa Score, Percent Overall Agreement among raters above chance, Confidence Interval of Kappa & Significance Test

Multi-label inter rater agreement using fleiss kappa, krippendorff's alpha and the MASI similarity measure for set simmilarity. Written in R Quarto.

𝗟𝗮𝗯𝗥𝗜 𝗦𝗵𝗶𝗻𝘆 𝗜𝗻𝘀𝘁𝗮𝗹𝗹𝗲𝗿: Tool for estimating reference intervals using the indirect sampling method.

The program implements the calculus of the Fleiss' Kappa in the both the fixed and margin-free version. The data used are a collection of possible results of an analysis of 6 annotators on 100 texts.

Raters inter-rater agreement

💬 Advanced NLP with Spacy Course

Evaluation and agreement scripts for the DISCOSUMO project. Each evaluation script takes both manual annotations as automatic summarization output. The formatting of these files is highly project-specific. However, the evaluation functions for precision, recall, ROUGE, Jaccard, Cohen's kappa and Fleiss' kappa may be applicable to other domains too.

𝗟𝗮𝗯𝗥𝗜 𝗦𝗵𝗶𝗻𝘆 𝗔𝗽𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻: Tool for estimating reference intervals using the indirect sampling method.

Data and codes for analysing an industry model Pacific oyster for sustainable aquaculture

Using information retrieval to help an emerging local artist boost its streams