Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add RemoteEnv and LocalEnv exec tests using SSH test infra #521

Closed
bpkroth opened this issue Oct 3, 2023 · 0 comments · Fixed by #557
Closed

Add RemoteEnv and LocalEnv exec tests using SSH test infra #521

bpkroth opened this issue Oct 3, 2023 · 0 comments · Fixed by #557
Assignees
@bpkroth
Labels
tests Add or fix unit tests

Comments

@bpkroth
Copy link
Contributor

bpkroth commented Oct 3, 2023

Follow on work to #510
@bpkroth bpkroth self-assigned this Oct 3, 2023
@bpkroth bpkroth mentioned this issue Oct 3, 2023
Merged
8 tasks
@bpkroth bpkroth added the tests Add or fix unit tests label Oct 11, 2023
bpkroth added a commit that referenced this issue Oct 26, 2023
@bpkroth @motus
Add SSH Support (#510)
1f0f8ee 
- Adds SSH support for remote exec and file copy operations via
`asyncssy` library. Closes #379
- This requires an event loop thread to operate the async calls in the
background without blocking the rest of our main thread operations or
changing our APIs dramatically. Instead we interact with all operations
as futures after that.
- Add json schema config support for SSH

- Adds test infrastructure for running SSH servers inside containers via
`pytest-docker` which uses `docker compose` to start/stop them as
fixtures. This *should* work in all dev environments (Linux host, WSL
host, Windows host), but is quite sensitive to networking setups.

  - [x] test the background thread cleanup logic
  - [x] test error handling of a broken ssh endpoint
  - [x] test basic exec commands
  - [x] test host operations (e.g., reboot)
  - [x] test basic copy commands
  - [x] test for single connection reuse across services
  - [x] test multi-server support
  - [x] use random ports to avoid conflicts
  - provide some config examples for using this
  - add remote_env exec tests with this (increase code coverage)
    - Will do these in a future PR instead - #521

---------

Co-authored-by: Sergiy Matusevych <sergiym@microsoft.com>
Co-authored-by: Sergiy Matusevych <sergiy.matusevych@gmail.com>
@bpkroth bpkroth mentioned this issue Oct 26, 2023
Merged
bpkroth added a commit that referenced this issue Nov 1, 2023
@bpkroth @motus
Add RemoteEnv tests using SshService testing infrastructure (#557)
b4b5f93 
Adds tests for RemoteEnv `setup`, `run`, `teardown` using SshServices

Fixes some associated bugs:
- convert certain `SFTPError` exceptions to `FileNotFound` in
`SshFileShareService`
This is important for `LocalFileShareEnv` integration especially since
it always calls `download` during calls to `status`, but only handles
`FileNotFound` exceptions when `ignore_missing=True`
- Fixups to #517 for `download`, `upload` to connect via `self._params`
(loaded from the `required_args`) instead of `self.config`

Also included:
- basic configs for SshServices and LocalExec for easy inclusion

Closes #521

---------

Co-authored-by: Sergiy Matusevych <sergiym@microsoft.com>
Co-authored-by: Sergiy Matusevych <sergiy.matusevych@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bpkroth bpkroth

Labels
tests Add or fix unit tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add RemoteEnv tests using SshService testing infrastructure bpkroth/MLOS
1 participant
@bpkroth