Organization
Location
CN
Badges
Activity
Ratings Progression
Challenge Categories
Challenges Entered
Completed
Meta
Improve RAG with Real-World Benchmarks
Latest submissions
See All| graded | 267130 | ||
| graded | 267129 | ||
| graded | 267099 |
Amazon Search
Amazon KDD Cup 2022
Latest submissions
Testing RAG Systems with Limited Web Pages
Latest submissions
See All| graded | 266952 | ||
| graded | 266951 | ||
| graded | 266273 |
Enhance RAG systems With Multiple Web Sources & Mock API
Latest submissions
See All| graded | 267130 | ||
| graded | 267129 | ||
| failed | 266263 |
| Participant | Rating |
|---|---|
chenghao_shaun
|
0 |
shizueyy
|
0 |
qi_tang
|
0 |
graphway
|
0 |
| Participant | Rating |
|---|
Meta Comprehensive RAG Benchmark: KDD Cup 2-9d1937
🚨 IMP: Phase 2 Announcement
7 months agosame problem, I also email the help@aicrowd.com but no responses
Has phase-2 started?
7 months agoI don’t know, I saw some successful commits and tried to commit and found that it got scores but didn’t update the leaderboard
Has phase-2 started?
7 months agohi bro, I have the same question and have not received any message.
Submission failed
7 months agoSubmission failed : You have exceeded the allowed number of parallel submissions. Please wait until your other submission(s) are graded.
No other submissions but failed.
Expect to return a message that stating whether it was a timeout problem
7 months agoThis doesn’t need to return logs and helps us troubleshoot some issues
📢 Announcements: Phase 1 Extension, New Private Test Set, Batch Prediction Interface, and Updated Baselines
7 months agoAccording to baseline , now each query only has 10s to answer?
- Response Time: Ensure that your model processes and responds to each query within 10 seconds.
📢 Announcement: Addition of `query_time` to the `generate_answer` Interface, and increased Timeouts!
8 months agoWhat is the format of query_time?
Testing time for task 3
8 months agoThe runtime is printed in the log returned by the server and has nothing to do with local testing. This submission id are #253185 #253183 #253142 #253102 #253092
Testing time for task 3
8 months agoI’ve tested this several times inside the generate_answer function and it only takes about 8 seconds to return the output, but the server determines that it has timed out. I suspect that the timing starts when the data is decompressed rather than when the function is called.
About the 'search results' type
8 months agoThe code in line 106 says - search_results (List[str]): Text content from web pages as search results.
But in line 123, it looks like a map soup = BeautifulSoup(html_text['page_result'], features="html.parser")
Hi, where is the baseline?
9 months agoThe web page says
We provide baseline RAG implementations based on llama-2-chat-7b model to help participants on board quickly.
But I couldn’t find it.
Meta KDD Cup 24 - CRAG - Retrieval Summarization
About Test Set Leakage in Round 1
7 months agoIn fact, the test set for round1 is the data set given to us, so there is no leakage problem
How exactly is the number of submissions counted ten times a week?
6 months agoAfter my testing, if the error is reported in the build environment, the submission time will not be deducted, but it will be recorded.