Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Runner: some improvements and fixes #123

Open
4 tasks
siddharth-krishna opened this issue Feb 25, 2025 · 0 comments
Open
4 tasks

Runner: some improvements and fixes #123

siddharth-krishna opened this issue Feb 25, 2025 · 0 comments
Assignees

Comments

@siddharth-krishna
Copy link
Contributor

siddharth-krishna commented Feb 25, 2025

  • Use argparse to parse CLI args more robustly
  • Also save information like timestamp, timeout, machine configuration (numCPUs, memory), hostname to each row of the results CSV, so that when we parallelize running on multiple VMs on the cloud, we can concat the results and still know which benchmark was run where
  • Add CLI arg to upload result to GCP buckets/some other location
  • Add a timeout column to the result CSV files so that when we run different benchmarks with different TOs, the website can display them and calculate SGM appropriately (see Home: SGM on intersection of benchmarks solved by all #114 (comment))

Kristijan, just assigning you for FYI for now as you'll be working on the runner. But feel free to prioritize and work on whatever makes sense, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants