Service Announcement: Delays in GPU Node Provisioning

If the issue remains unresolved by the end of the day, should we anticipate an extension for the deadline? @mohanty

View post 0

Service Announcement: Delays in GPU Node Provisioning

hi @mohanty
The submission page isn’t reflecting new submissions for Task 1. Could you inform when we can anticipate these submissions to be processed and evaluated?

View post 0

Request for More Comprehensive Error Reports for Recent Submission Failures Task1

Hi @dipam,
Considering this situation, could you offer any advice? Is it wise to keep submitting the same code until we get a result? Given the daily limit of the submissions, can we do something to make sure failed submissions don’t count against our limit?

Also, submissions #247986 and #24795 (by @tomtom) for Task 1 Track 1 failed without any debugging code, even though they didn’t make any OpenAI API calls. Are submissions for Track 1 and Track 2 evaluated on a single server and placed in the same queue, potentially causing the queue to get blocked (and eventually timeout) by previous submissions that make OpenAI API calls?

Thank you.

View post 0

Request for More Comprehensive Error Reports for Recent Submission Failures Task1

(post deleted by author)

View post 0

Request for More Comprehensive Error Reports for Recent Submission Failures Task1

@dipam @mohanty @tomtom

View post 0

Request for More Comprehensive Error Reports for Recent Submission Failures Task1

I’ve noticed that my recent submissions have been failing without detailed log files or explanations, except for submission #247972, which was reported to time out. This lack of clarity makes it difficult to determine if the failures are due to code issues or system errors. Is it possible to receive more comprehensive error reports for submissions #247981, #247983, #247984 and #248051?

Thank you.

View post 0

Updates to Task 1 Metrics

Hi @dipam,
Could you specify the exact number or range of submissions that will undergo human evaluation as stated in your guidelines? It mentions “only the top several systems…will be judged via human evaluation” but it is unclear.
Thank you.

View post 2