Code

Ray Cluster features#50

Code

Merged

al-rigazzi merged 131 commits intoCrayLabs:developCrayLabs/SmartSim:developfrom al-rigazzi:rlal-rigazzi/SmartSim:rlCopy head branch name to clipboardOct 11, 2021

+1,584-41

Commits

Commits on Apr 13, 2021

• Adds notebook to run head_node.

  al-rigazzi

  committed
  52c3ddc

  Copy full SHA for 52c3ddc

Commits on Apr 16, 2021

• Add Ray notebook which can start cluster head node.

  al-rigazzi

  committed
  34ca15c

  Copy full SHA for 34ca15c

Commits on Apr 23, 2021

• Workarounds for Ray/Jupyter resource problems.

  al-rigazzi

  committed
  4fbd5a1

  Copy full SHA for 4fbd5a1

• Working setup for XC.

  al-rigazzi

  committed
  fa36f4e

  Copy full SHA for fa36f4e

Commits on Apr 28, 2021

• Insert CPU control for SLURM Ray.

  al-rigazzi

  committed
  168b9fe

  Copy full SHA for 168b9fe

Commits on May 5, 2021

• Adds Ray cluster setup.

  al-rigazzi

  committed
  7d73ad5

  Copy full SHA for 7d73ad5

Commits on May 7, 2021

• Refactor RayWorker and RayHead.

  al-rigazzi

  committed
  f0aafa3

  Copy full SHA for f0aafa3

Commits on May 9, 2021

• Adds batch capabilities to Ray Slurm launcher.

  al-rigazzi

  committed
  a65b710

  Copy full SHA for a65b710

Commits on May 10, 2021

• Add preamble to batch_settings

  al-rigazzi

  committed
  0fb644f

  Copy full SHA for 0fb644f

Commits on May 13, 2021

• Fixes some issues on CCM for Ray

  al-rigazzi

  committed
  ab2c6eb

  Copy full SHA for ab2c6eb

• Redirect cmd server output for Ray.

  al-rigazzi

  committed
  a79d143

  Copy full SHA for a79d143

Commits on May 14, 2021

• Avoid Ray workers overlapping with head.

  al-rigazzi

  committed
  4c56512

  Copy full SHA for 4c56512

Commits on May 17, 2021

• Change Slurm configuration for Ray.

  al-rigazzi

  committed
  c47c5ed

  Copy full SHA for c47c5ed

Commits on May 18, 2021

• Remove command server and use Ray client.

  al-rigazzi

  committed
  bead0b2

  Copy full SHA for bead0b2

Commits on May 19, 2021

• Add ray cluster for PBS batch.

  al-rigazzi

  committed
  14f0197

  Copy full SHA for 14f0197

Commits on May 24, 2021

• Adds PBS functionalities for Ray cluster.

  al-rigazzi

  committed
  c4723dc

  Copy full SHA for c4723dc

Commits on May 25, 2021

• Adds Ray starter script.

  al-rigazzi

  committed
  6abb6fa

  Copy full SHA for 6abb6fa

• Merge branch 'rl' of https://github.com/al-rigazzi/SmartSim into rl

  al-rigazzi

  committed
  dc1f8f5

  Copy full SHA for dc1f8f5

• Add new starter for Ray

  al-rigazzi

  committed
  f55aae5

  Copy full SHA for f55aae5

Commits on May 26, 2021

• Fix batch args bug for RayCluster

  al-rigazzi

  committed
  f0d4192

  Copy full SHA for f0d4192

• Merge remote-tracking branch 'CrayLabs/develop' into rl

  al-rigazzi

  committed
  bd248b6

  Copy full SHA for bd248b6

• Add preamble mechanism to BatchSettings.

  al-rigazzi

  committed
  465995a

  Copy full SHA for 465995a

• Remove old ray starter script.

  al-rigazzi

  committed
  376e289

  Copy full SHA for 376e289

• Fixes tests for new interface.

  al-rigazzi

  committed
  705436e

  Copy full SHA for 705436e

• Fix entity utils test.

  al-rigazzi

  committed
  bbeb03a

  Copy full SHA for bbeb03a

• Delete issues.md

  al-rigazzi

  authored
  f7fb211

  Copy full SHA for f7fb211

Commits on Jun 4, 2021

• Add local launcher and test for Ray.

  al-rigazzi

  committed
  397d125

  Copy full SHA for 397d125

• Merge branch 'rl' of https://github.com/al-rigazzi/SmartSim into rl

  al-rigazzi

  committed
  a105693

  Copy full SHA for a105693

• Delete 05_starting_ray.ipynb

  Show description for a3bccfe
  al-rigazzi

  authored
  a3bccfe

  Copy full SHA for a3bccfe

• Delete manual-start.sh

  Show description for c4e0c80
  al-rigazzi

  authored
  c4e0c80

  Copy full SHA for c4e0c80

• Delete start-head.sh

  Show description for 732868d
  al-rigazzi

  authored
  732868d

  Copy full SHA for 732868d

• Delete start-worker.sh

  Show description for 6575a47
  al-rigazzi

  authored
  6575a47

  Copy full SHA for 6575a47

• Update requirements.

  al-rigazzi

  committed
  810f0a3

  Copy full SHA for 810f0a3

Commits on Jul 2, 2021

• Remove check for slurm launcher and rely on exception.

  al-rigazzi

  committed
  d46f73f

  Copy full SHA for d46f73f

• Merge branch 'develop' into rl

  al-rigazzi

  committed
  d0faeae

  Copy full SHA for d0faeae

Commits on Jul 3, 2021

• Address reviewer's comments. Add ray_args.

  al-rigazzi

  committed
  0d30c31

  Copy full SHA for 0d30c31

Commits on Jul 20, 2021

• Merge branch 'develop' into rl

  al-rigazzi

  committed
  2a8088f

  Copy full SHA for 2a8088f

• Modify Ray tutorial.

  al-rigazzi

  committed
  81f9a99

  Copy full SHA for 81f9a99

• Merge branch 'develop' into rl

  al-rigazzi

  committed
  44a147f

  Copy full SHA for 44a147f

• Merge branch 'rl' of https://github.com/al-rigazzi/SmartSim into rl

  al-rigazzi

  committed
  4a7dd91

  Copy full SHA for 4a7dd91

• Merge branch.

  al-rigazzi

  committed
  7d605e4

  Copy full SHA for 7d605e4

• Adds Ray to manifest.

  al-rigazzi

  committed
  ed192d9

  Copy full SHA for ed192d9

• Fix to raystarter.py

  al-rigazzi

  committed
  f2bfb82

  Copy full SHA for f2bfb82

Commits on Jul 22, 2021

• Add manifest.ray_clusters to exp.stop()

  al-rigazzi

  committed
  dcf4827

  Copy full SHA for dcf4827

• Removes egg files.

  al-rigazzi

  committed
  3b57a62

  Copy full SHA for 3b57a62

• Fixes wrong option for Ray and aprun

  al-rigazzi

  committed
  932a563

  Copy full SHA for 932a563

Commits on Jul 27, 2021

• Address review, add flexibility to ray cluster

  al-rigazzi

  committed
  5b4981b

  Copy full SHA for 5b4981b

• Add API functions to RayCluster

  al-rigazzi

  committed
  87d6e29

  Copy full SHA for 87d6e29

• Fix for internal ray args

  al-rigazzi

  committed
  d997cac

  Copy full SHA for d997cac

• Add dashboard port to raystarter args.

  al-rigazzi

  committed
  7731867

  Copy full SHA for 7731867

Commits on Jul 28, 2021

• Apply styling.

  al-rigazzi

  committed
  e22610a

  Copy full SHA for e22610a

Commits on Jul 29, 2021

• Merge remote-tracking branch 'upstream/develop' into rl

  al-rigazzi

  committed
  22f7e39

  Copy full SHA for 22f7e39

• Add tests for ray on slurm

  al-rigazzi

  committed
  f706f27

  Copy full SHA for f706f27

• Fix slurm in alloc ray test

  al-rigazzi

  committed
  0aaa3b3

  Copy full SHA for 0aaa3b3

Commits on Jul 30, 2021

• ADD new information to README and OA tutorial

  Show description for 8a5f9c0
  

  Sam Partee

  committed
  8a5f9c0

  Copy full SHA for 8a5f9c0

• Link Online Analysis example into docs

  Show description for 860f96f
  

  Sam Partee

  committed
  860f96f

  Copy full SHA for 860f96f

• Add visualization to README

  

  Sam Partee

  authored
  3097a26

  Copy full SHA for 3097a26

• Add PBS tests and pass ray_exe to raystarter

  al-rigazzi

  committed
  2b4ea74

  Copy full SHA for 2b4ea74

• Remove expand_exe option

  al-rigazzi

  committed
  4ecffc8

  Copy full SHA for 4ecffc8

• Remove duplicate function

  al-rigazzi

  committed
  59fc8fa

  Copy full SHA for 59fc8fa

• Remove unused ray template

  al-rigazzi

  committed
  b29db44

  Copy full SHA for b29db44

• Merge branch 'develop' of https://github.com/CrayLabs/SmartSim into rl

  al-rigazzi

  committed
  2275fce

  Copy full SHA for 2275fce

• Add egg-info to gitignore

  al-rigazzi

  committed
  47a8869

  Copy full SHA for 47a8869

• Remove expand_exe from RayCluster

  al-rigazzi

  committed
  ada1709

  Copy full SHA for ada1709

• Fix exe_path

  al-rigazzi

  committed
  4629f3d

  Copy full SHA for 4629f3d

• Merge branch 'new-docs' of https://github.com/spartee/SmartSim into ray_docs

  al-rigazzi

  committed
  ff6a95b

  Copy full SHA for ff6a95b

• Adds ray API to docs

  al-rigazzi

  committed
  5bc9794

  Copy full SHA for 5bc9794

Commits on Jul 31, 2021

• Move set_cpus out of launch tests.

  al-rigazzi

  committed
  a5b44c9

  Copy full SHA for a5b44c9

• Fix characters in options for PBSOrchestrator

  al-rigazzi

  committed
  85ba1a5

  Copy full SHA for 85ba1a5

• Fix non utf8 chars in options

  al-rigazzi

  committed
  15e29fb

  Copy full SHA for 15e29fb

• Fix Cobalt options

  al-rigazzi

  committed
  e25ada4

  Copy full SHA for e25ada4

• Fix options in AprunSettings.

  al-rigazzi

  committed
  0cbe771

  Copy full SHA for 0cbe771

Commits on Aug 1, 2021

• Allow multiple trials for Ray tests

  al-rigazzi

  committed
  b1434d5

  Copy full SHA for b1434d5

Commits on Aug 3, 2021

• Fix ray launch summary

  al-rigazzi

  committed
  194c54b

  Copy full SHA for 194c54b

Commits on Aug 5, 2021

• Address local launcher for Ray

  Show description for c28320f
  

  Sam Partee

  committed
  c28320f

  Copy full SHA for c28320f

Commits on Aug 6, 2021

• Make RayCluster closer to SmartSim paradigm

  al-rigazzi

  committed
  9260103

  Copy full SHA for 9260103

• Modifies starter to bind dashboard to all interfaces

  al-rigazzi

  committed
  7fe95ef

  Copy full SHA for 7fe95ef

Commits on Aug 7, 2021

• Revert port for dashboard, update github wf

  al-rigazzi

  committed
  b94be58

  Copy full SHA for b94be58

Commits on Aug 9, 2021

• Add password option to RayCluster

  al-rigazzi

  committed
  2120a84

  Copy full SHA for 2120a84

• Remove output from Notebook

  al-rigazzi

  committed
  a7334f0

  Copy full SHA for a7334f0

• Remove log level env from ray notebook

  al-rigazzi

  committed
  6ee94f4

  Copy full SHA for 6ee94f4

Commits on Aug 10, 2021

• Adapt ray tests.

  al-rigazzi

  committed
  dfd0a85

  Copy full SHA for dfd0a85

• Bump up ray version

  al-rigazzi

  committed
  2186b37

  Copy full SHA for 2186b37

• Merge remote-tracking branch 'upstream/develop' into rl

  al-rigazzi

  committed
  2546172

  Copy full SHA for 2546172

• Remove unused launch branch, fix ray summary

  al-rigazzi

  committed
  b56c669

  Copy full SHA for b56c669

• Remove unused launcher branch

  al-rigazzi

  committed
  677fab6

  Copy full SHA for 677fab6

• Apply styling

  al-rigazzi

  committed
  d3290ea

  Copy full SHA for d3290ea

• Remove notebook output

  al-rigazzi

  committed
  f460fa9

  Copy full SHA for f460fa9

• Remove block_in_batch feature

  al-rigazzi

  committed
  efd7747

  Copy full SHA for efd7747

• Fix ALPS regression introduced in this branch

  al-rigazzi

  committed
  af85bf0

  Copy full SHA for af85bf0

• Fix settings

  al-rigazzi

  committed
  93fcdcf

  Copy full SHA for 93fcdcf

• Fix docstring

  al-rigazzi

  committed
  a958d22

  Copy full SHA for a958d22

• Add ignore flag for ray batch tests.

  al-rigazzi

  committed
  b8d17ba

  Copy full SHA for b8d17ba

• Change ray_started args

  al-rigazzi

  committed
  4fbe440

  Copy full SHA for 4fbe440

• Update docstrings

  al-rigazzi

  committed
  70a492c

  Copy full SHA for 70a492c

• Update TODO list in raycluster.py

  al-rigazzi

  committed
  5fd61e8

  Copy full SHA for 5fd61e8

• Add disclaimer to notebook, license to raycluster

  al-rigazzi

  committed
  11934a7

  Copy full SHA for 11934a7

• Make RayCluster error more useful

  al-rigazzi

  committed
  0b4321f

  Copy full SHA for 0b4321f

• Add ray.shutdown to tests.

  al-rigazzi

  committed
  b857157

  Copy full SHA for b857157

• Add interface to Ray PBS tests. Apply styling

  al-rigazzi

  committed
  2570cc3

  Copy full SHA for 2570cc3

• Remove useless _vars from RayCluster

  al-rigazzi

  committed
  733deb2

  Copy full SHA for 733deb2

• Remove unused attributes from RayCluster

  al-rigazzi

  committed
  217c28f

  Copy full SHA for 217c28f

• Remove ray_head variable

  al-rigazzi

  committed
  4cf8163

  Copy full SHA for 4cf8163

• Fix new variables for RayCluster

  al-rigazzi

  committed
  19c1a2c

  Copy full SHA for 19c1a2c

• Merge branch 'rl_cleanup' into rl

  al-rigazzi

  committed
  d8d7a37

  Copy full SHA for d8d7a37

• Make RayCluster functions static

  al-rigazzi

  committed
  e3db4db

  Copy full SHA for e3db4db

• Modify notebook

  al-rigazzi

  committed
  09ee09e

  Copy full SHA for 09ee09e

Commits on Aug 11, 2021

• Update Ray path to exp

  al-rigazzi

  committed
  aa8ad2e

  Copy full SHA for aa8ad2e

• Update docs, removed unused function

  al-rigazzi

  committed
  8d7fe10

  Copy full SHA for 8d7fe10

• Extend wait time for Ray head log

  al-rigazzi

  committed
  0ca9c0c

  Copy full SHA for 0ca9c0c

Commits on Aug 26, 2021

• Merge branch 'develop' into rl

  al-rigazzi

  committed
  b1628ba

  Copy full SHA for b1628ba

• Remove node retrieval for Ray

  al-rigazzi

  committed
  3e884f3

  Copy full SHA for 3e884f3

• Merge remote-tracking branch 'origin/ray_docs' into rl

  al-rigazzi

  committed
  6fc8281

  Copy full SHA for 6fc8281

Commits on Aug 27, 2021

• Update notebook and summary for RayCluster

  al-rigazzi

  committed
  968a95a

  Copy full SHA for 968a95a

• Fixes to Ray docs

  al-rigazzi

  committed
  2993b7d

  Copy full SHA for 2993b7d

• Add interface to Ray tests

  al-rigazzi

  committed
  191288a

  Copy full SHA for 191288a

• Merge branch 'rl' of https://github.com/al-rigazzi/SmartSim into rl

  al-rigazzi

  committed
  7330d3c

  Copy full SHA for 7330d3c

• Apply styling

  al-rigazzi

  committed
  a239a5a

  Copy full SHA for a239a5a

Commits on Oct 5, 2021

• Address reviewer's comments

  al-rigazzi

  committed
  7674d01

  Copy full SHA for 7674d01

• Merge branch 'develop' of https://github.com/CrayLabs/SmartSim into rl

  al-rigazzi

  committed
  086cb91

  Copy full SHA for 086cb91

• Apply styling

  al-rigazzi

  committed
  84f4069

  Copy full SHA for 84f4069

• Add SSH tunneling instructions to Ray notebook

  al-rigazzi

  committed
  a6fb7f2

  Copy full SHA for a6fb7f2

• Change workers parameter to num_nodes for clarity

  al-rigazzi

  committed
  b694c25

  Copy full SHA for b694c25

• Update Ray tests

  al-rigazzi

  committed
  be4f7e2

  Copy full SHA for be4f7e2

Commits on Oct 11, 2021

• Add review's suggestions and dashboard host fix

  al-rigazzi

  committed
  fb3d899

  Copy full SHA for fb3d899

• Update Ray tutorial

  al-rigazzi

  committed
  2b06c1c

  Copy full SHA for 2b06c1c

• Restrict dashboard option to head only

  al-rigazzi

  committed
  4b325b7

  Copy full SHA for 4b325b7

• Correct typo in Ray notebook

  al-rigazzi

  committed
  0aec403

  Copy full SHA for 0aec403

• Add some info to notebook.

  al-rigazzi

  committed
  ce03a9e

  Copy full SHA for ce03a9e

• Fix typo in notebook

  al-rigazzi

  committed
  5bd5b91

  Copy full SHA for 5bd5b91

• Merge branch 'develop' of https://github.com/CrayLabs/SmartSim into rl

  al-rigazzi

  committed
  149edaf

  Copy full SHA for 149edaf