When will the labels for the test set released?

[zubatyuk]

Hi.
When do you plan to release the labels for the test set? Our team got the validation errors significantly lower than one of the winners (https://github.com/zouxiaochuan/molnet_kddcup2021/blob/master/molnet_arxiv.pdf). We want to analyze what went wrong with the test set prediction.

[weihua916]

Great, that's exciting! We will keep the test labels hidden and release the evaluation server for you to submit your model prediction. We will keep the community updated.

[kirayummy]

That is helpful for the graph community! And when will you reopen the evaluation server? Thanks!

[weihua916]

That’s still to be decided. Hopefully in the next month or so

[Fairly]

Hi Weihua, are you still working on the evaluation server? We are so eager to check our new model!

[weihua916]

Yes we are making slow progress! Please give us a bit more time, but we will definitely release the eval server sometime soon.

[cenyk1230]

Hi @zubatyuk,

Would you please tell us the validation errors that your team got?

[zubatyuk]

We had about 0.1 with no signs of serious over fitting.

[weihua916]

As mentioned here, the PCQM4M will be deprecated. The new dataset, called PCQM4Mv2, will be introduced by the end of September. It's taking a lot of effort to update/benchmark/debug the datasets after the KDD Cup. Thank you everyone for the patience!