Evaluation, Reproducibility, Benchmarks Meeting 22

Minutes of meeting 22

Date: 26th July 2023

Present: Michela, Lena, Keyvan, Carole

TOP 1: Book chapter

• Carole is interested in joining. Will decide later due to maternity leave

TOP 2: Proposal for further MONAI Benchmarking members

• M. Reyes
• Somebody from grandchallenges
• Decision: Decide on who we should recruit next time with a poll (Annika will create it).
  • Provide vote: Definitely recruit, recruit, don’t recruit, no opinion
  • Sum up votes and invite top 3

Moonshot Benchmarking best practices

• We will go for it!
• Eva will lead it
• We could have 3 stages
  • what you can do with existing results (eg. bootstrapping)
  • what you can do with existing models (i.e. inference time)
  • what you can do with everything (i.e. including training time)
• Next time: Presentation of what already exists (by Eva)