About the ImageCLEF 2022 Caption - Concept Detection category

The exact dataset is not available anymore, but you can use ROCOv2 or participate in this year’s challenge.

View post 0

Problem with submission separator

Hello Georgios,

you are right, the linked evaluation scripts were not up-to-date, sorry about that. I have now uploaded the actual evaluation code used on aicrowd (you might have to remove the header rows from the label files), please find the new links and explanation how to run the scripts on imageclef.org.

Kind regards,
Johannes

View post 1

Problem with submission separator

The file is being read as CSV, but any plain text file corresponding to the submission format will work, the file ending does not matter.

View post 0

Problem with submission separator

Hello,

please use ‘|’ (pipe) as the separator in your submission files according to the submission instructions.

Kind regards,
Johannes

View post 0

ROUGE-1 score

Yes, that is correct. BLEU is used as the primary score, ROUGE is used as the secondary score.

View post 0

Subtask independence and evaluation process

Hello!

• The tasks are independent from each other, so there could be different teams winning each one
• You do not have to submit to both tasks
• It is not mandatory to use the concepts for the second task

Kind regards,
Johannes

View post 1

ROUGE-1 score

Hello Georgios,

we are using the F-measure.

Kind regards,
Johannes

View post 0