UA
Activity
Ratings Progression
Challenge Categories
Challenges Entered
Completed
Sony Group Corporation
Latest submissions
SNCF
SBB
Deutsche Bahn
Multi-Agent Reinforcement Learning on Trains
Latest submissions
OpenSky Network
Cyber-Defence Campus, armasuisse
Multi Agent Reinforcement Learning on Trains.
Latest submissions
See All| graded | 32763 | ||
| graded | 32762 | ||
| graded | 32761 |
DEEP
Help improve humanitarian crisis response through better NLP modeling
Latest submissions
AIcrowd
Multi-Agent Reinforcement Learning on Trains
Latest submissions
| Participant | Rating |
|---|---|
student
|
271 |
| Participant | Rating |
|---|
CYD Campus Aircraft Localization Competition
Data from the future
Over 4 years agoHello @masorx
Just to add the rule to prohibit a future data usage is not enough in general. Always possible to implement a method that solves the offline version of the problem with all future data and then finetune the official online method with predicted data at hidden offline realization as “ground-truth”.
If you really want to solve an online tracking problem, you should invent some method how to strictly hide future data, like at the flatland challenge. But it required something like kernel-competitions at Kaggle.
Evaluation metrics
Over 4 years agoHello @masorx
What units (m or km) of a distance do you use for RMSE calculation?
Evaluation metrics
Over 4 years agoThank you for your answers. I have a few additional questions about metrics.
Why you use 2D distance? We need to predict the height of aircraft, no?
Do you check the submission file fully after each submits? So the score on the leaderboard is final, right?
Evaluation metrics
Over 4 years agoHello,
Can you please describe the evaluation metrics of the competition? What the score and secondary score means?
Flatland
Start of the competition
Over 4 years agoHello,
The link to the competition appears on NeurIPS, but it looks hidden now. When the competition will start?
Flatland Challenge
Publishing the Solutions
Over 4 years agoHere is our (ck.ua team) 2nd place solution:
https://bitbucket.org/roman_chernenko/flatland-final/src/master/
Problems with submissions
Almost 5 years ago(topic withdrawn by author, will be automatically deleted in 24 hours unless flagged)
Solution Codes and Approaches
Almost 5 years agoHello @nilabha,
What results you have with these approaches?
Evaluation process
About 5 years agoHello,
I have a few questions about evaluation process.
- Can you please confirm that our solutions always evaluated on the same test samples? Now looks like the test sequence has shuffled at least.
- How maps are choosen for visualization video? I can see different teams has different maps on the video.
- I seen something strange with score progress during evaluation. The score always start from some low value and next quickly increased. And at the end I always see a big score jump. For example, I had a score 91.6% after 248 simulations. But in the end I have a score 92%. This significant jump at the end is not possible. Looks like the scoring algorithm divided done-agens sum on N+1, where N is number of finished simulations.
Evaluation server hardware specifications
About 5 years agoHello,
What hardware are using for submission evaluation? How much RAM and CPU cores we can use?
Computation budget
About 5 years agoHello @mlerik ,
What hardware are you using for submission evaluation (CPU cores, max allowed RAM etc.)?
[ANNOUNCEMENT] Round 2 Update
About 5 years agoHello @mlerik
Do you have any updates, when round 2 should be start?
Further questions regarding submissions
Over 5 years ago@mlerik, Thank you for the clarification. I’m still confusing how final score will be calculated. Here is a quote from the challenge overview:
Further questions regarding submissions
Over 5 years agoI also have a lot of questions about the challenge regarded with submissions.
- What hardware are using for submission evaluation?
- How score was calculated in the leaderboard?
- How score will combined in Round 1 and Round 2.
- What maximum possible world size and train count?
And it’s a time to update the overview text, because now we can find a lot of useful information only in discussions.
The problem with the round duration
Over 5 years agoCurrently I can see a message " Round 0 : Exploration Round: 3 months" at top of the challenge description". But round 0 has finished yesterday.
Submission format
Over 5 years agoAs I understand from the challenge rules, our models will evaluating on a set of unknown random seed worlds.
But how we will submit our models? Just an archive of python files? Or in a Docker?
Mutli Agent Setup
Over 5 years agoSo, we should expect 10000*10000 world size in next round, right? 
Round 2 is coming!1
Over 4 years agoHello @masorx
When data for round 2 will be available for downloading?