Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use uv #392

Merged
merged 6 commits into from
Feb 18, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/csafdemo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ jobs:
- name: Display Python version
run: python -c "import sys; print(sys.version)"
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5
- uses: oras-project/setup-oras@v1
- name: Install dependencies
env:
Expand All @@ -19,7 +21,7 @@ jobs:
python3 -m venv venv
source venv/bin/activate
pip install --upgrade pip setuptools
pip install .[dev]
uv sync --all-extras --dev
pip install check-jsonschema
mkdir -p vuln_spring
cd vuln_spring
Expand Down
46 changes: 28 additions & 18 deletions .github/workflows/dockertests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ jobs:
fail-fast: false
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5
- name: Set up Python
uses: actions/setup-python@v5
with:
Expand All @@ -24,16 +26,16 @@ jobs:
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip setuptools
python3 -m pip install .[dev]
uv sync --all-extras --dev
npm install -g @cyclonedx/cdxgen
mkdir -p repotests
python3 -m pip install -r contrib/requirements.txt
- name: Test container images
run: |
mkdir -p containertests_${{ matrix.os }}_python${{ matrix.python-version }}
# python3 depscan/cli.py --no-banner --cache --no-error --src ghcr.io/owasp-dep-scan/dep-scan -o ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}_python${{ matrix.python-version }}/depscan-scan.json -t docker
python3 depscan/cli.py --no-banner --no-error --src shiftleft/scan-slim -o ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}_python${{ matrix.python-version }}/depscan-slim.json -t docker,license --no-vuln-table
python3 depscan/cli.py --no-banner --no-error --src redmine@sha256:a5c5f8a64a0d9a436a0a6941bc3fb156be0c89996add834fe33b66ebeed2439e -o ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}_python${{ matrix.python-version }}/depscan-redmine.json -t docker --no-vuln-table
# uv run depscan --no-banner --cache --no-error --src ghcr.io/owasp-dep-scan/dep-scan -o ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}_python${{ matrix.python-version }}/depscan-scan.json -t docker
uv run depscan --no-banner --no-error --src shiftleft/scan-slim -o ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}_python${{ matrix.python-version }}/depscan-slim.json -t docker,license --no-vuln-table
uv run depscan --no-banner --no-error --src redmine@sha256:a5c5f8a64a0d9a436a0a6941bc3fb156be0c89996add834fe33b66ebeed2439e -o ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}_python${{ matrix.python-version }}/depscan-redmine.json -t docker --no-vuln-table
env:
PYTHONPATH: "."
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -50,6 +52,8 @@ jobs:
fail-fast: false
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5
- name: Set up Python
uses: actions/setup-python@v5
with:
Expand All @@ -68,7 +72,7 @@ jobs:
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
python3 -m pip install .[dev]
uv sync --all-extras --dev
npm install -g @cyclonedx/cdxgen
mkdir -p repotests
- uses: actions/checkout@v4
Expand All @@ -81,7 +85,7 @@ jobs:
cd ${GITHUB_WORKSPACE}/repotests/java-sec-code
mvn clean compile -DskipTests
cd ${GITHUB_WORKSPACE}
python3 depscan/cli.py --no-banner --no-error --src ${GITHUB_WORKSPACE}/repotests/java-sec-code --reports-dir ${GITHUB_WORKSPACE}/rtests_ubuntu -t java --profile research --explain
uv run depscan --no-banner --no-error --src ${GITHUB_WORKSPACE}/repotests/java-sec-code --reports-dir ${GITHUB_WORKSPACE}/rtests_ubuntu -t java --profile research --explain
env:
PYTHONPATH: "."
PYTHONUTF8: 1
Expand All @@ -99,6 +103,8 @@ jobs:
fail-fast: false
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5
- name: Set up Python
uses: actions/setup-python@v5
with:
Expand All @@ -112,7 +118,7 @@ jobs:
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
python3 -m pip install .[dev]
uv sync --all-extras --dev
npm install -g @cyclonedx/cdxgen
mkdir -p repotests
- uses: actions/checkout@v4
Expand All @@ -129,10 +135,10 @@ jobs:
python3 -m pip install -r contrib/requirements.txt
cp contrib/csaf.toml repotests/microservices-demo/csaf.toml
cp contrib/csaf.toml repotests/NodeGoat/csaf.toml
python3 depscan/cli.py --no-banner --no-error --bom ./test/data/bom-yaml-manifest.json -o ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}/depscan-yaml.json --no-vuln-table
python3 depscan/cli.py --no-banner --no-error -t docker --src ubuntu:latest -o ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}/depscan-rocket.json --no-vuln-table
python3 depscan/cli.py --csaf --no-banner --no-error -t go --src ${GITHUB_WORKSPACE}/repotests/microservices-demo -o ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}/depscan-msd.json --reports-dir ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}/ng-reports
python3 depscan/cli.py --csaf --no-banner --no-error -t js --src ${GITHUB_WORKSPACE}/repotests/NodeGoat --reports-dir ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}/ng-reports
uv run depscan --no-banner --no-error --bom ./test/data/bom-yaml-manifest.json -o ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}/depscan-yaml.json --no-vuln-table
uv run depscan --no-banner --no-error -t docker --src ubuntu:latest -o ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}/depscan-rocket.json --no-vuln-table
uv run depscan --csaf --no-banner --no-error -t go --src ${GITHUB_WORKSPACE}/repotests/microservices-demo -o ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}/depscan-msd.json --reports-dir ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}/ng-reports
uv run depscan --csaf --no-banner --no-error -t js --src ${GITHUB_WORKSPACE}/repotests/NodeGoat --reports-dir ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}/ng-reports
env:
PYTHONPATH: "."
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -151,6 +157,8 @@ jobs:
fail-fast: false
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5
- name: Set up Python
uses: actions/setup-python@v5
with:
Expand All @@ -164,7 +172,7 @@ jobs:
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip setuptools
python3 -m pip install .[dev]
uv sync --all-extras --dev
python3 -m pip install -r contrib/requirements.txt
- name: Get boms generated earlier
uses: actions/download-artifact@v4
Expand All @@ -174,8 +182,8 @@ jobs:
- name: Test container images
run: |
mkdir -p containertests_${{ matrix.os }}_python${{ matrix.python-version }}
python3 depscan/cli.py --no-banner --cache --no-error --bom ${GITHUB_WORKSPACE}/containertests_ubuntu-latest_python3.11/sbom-slim-docker.json -o containertests_${{ matrix.os }}_python${{ matrix.python-version }}/depscan-slim.json --no-vuln-table
python3 depscan/cli.py --no-banner --no-error --bom ${GITHUB_WORKSPACE}/containertests_ubuntu-latest_python3.11/sbom-redmine-docker.json -o containertests_${{ matrix.os }}_python${{ matrix.python-version }}/depscan-redmine.json --no-vuln-table
uv run depscan --no-banner --cache --no-error --bom ${GITHUB_WORKSPACE}/containertests_ubuntu-latest_python3.11/sbom-slim-docker.json -o containertests_${{ matrix.os }}_python${{ matrix.python-version }}/depscan-slim.json --no-vuln-table
uv run depscan --no-banner --no-error --bom ${GITHUB_WORKSPACE}/containertests_ubuntu-latest_python3.11/sbom-redmine-docker.json -o containertests_${{ matrix.os }}_python${{ matrix.python-version }}/depscan-redmine.json --no-vuln-table
env:
PYTHONPATH: "."
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -194,6 +202,8 @@ jobs:
python-version: [ '3.11' ]
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5
- name: Set up Python
uses: actions/setup-python@v5
with:
Expand All @@ -207,7 +217,7 @@ jobs:
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
python3 -m pip install .[dev]
uv sync --all-extras --dev
python3 -m pip install -r contrib/requirements.txt
- name: Get boms generated earlier
uses: actions/download-artifact@v4
Expand All @@ -224,9 +234,9 @@ jobs:
mv containertests_ubuntu-latest/ng-reports/sbom-js.json containertests_ubuntu-latest/nodegoat/sbom-js.json
cp contrib/csaf.toml containertests_ubuntu-latest/microservices/csaf.toml
cp contrib/csaf.toml containertests_ubuntu-latest/nodegoat/csaf.toml
python3 depscan/cli.py --no-banner --no-error --bom ${GITHUB_WORKSPACE}/containertests_ubuntu-latest/sbom-rocket-docker.json -o containertests_${{ matrix.os }}/depscan-rocket.json --no-vuln-table
python3 depscan/cli.py --csaf --no-banner --no-error --bom ${GITHUB_WORKSPACE}/containertests_ubuntu-latest/microservices/sbom-msd-go.json -o containertests_${{ matrix.os }}/depscan-msd.json --reports-dir ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}/reports
python3 depscan/cli.py --csaf --no-banner --no-error --bom ${GITHUB_WORKSPACE}/containertests_ubuntu-latest/nodegoat/sbom-js.json -o containertests_${{ matrix.os }}/depscan-ng.json --reports-dir ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}/ng-reports
uv run depscan --no-banner --no-error --bom ${GITHUB_WORKSPACE}/containertests_ubuntu-latest/sbom-rocket-docker.json -o containertests_${{ matrix.os }}/depscan-rocket.json --no-vuln-table
uv run depscan --csaf --no-banner --no-error --bom ${GITHUB_WORKSPACE}/containertests_ubuntu-latest/microservices/sbom-msd-go.json -o containertests_${{ matrix.os }}/depscan-msd.json --reports-dir ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}/reports
uv run depscan --csaf --no-banner --no-error --bom ${GITHUB_WORKSPACE}/containertests_ubuntu-latest/nodegoat/sbom-js.json -o containertests_${{ matrix.os }}/depscan-ng.json --reports-dir ${GITHUB_WORKSPACE}/containertests_${{ matrix.os }}/ng-reports
env:
PYTHONPATH: "."
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand Down
14 changes: 8 additions & 6 deletions .github/workflows/gobintests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ jobs:
fail-fast: false
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5
- name: Set up Python
uses: actions/setup-python@v5
with:
Expand All @@ -36,7 +38,7 @@ jobs:
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip setuptools
python3 -m pip install .[dev]
uv sync --all-extras --dev
sudo npm install -g @cyclonedx/cdxgen
- name: setup nydus
run: |
Expand All @@ -50,15 +52,15 @@ jobs:
VDB_HOME: vdb_data_nydus
- name: Test with nydus
run: |
python3 depscan/cli.py --no-banner --no-error --src ghcr.io/owasp-dep-scan/depscan:master --reports-dir ${GITHUB_WORKSPACE}/containertests -t docker
uv run depscan --no-banner --no-error --src ghcr.io/owasp-dep-scan/depscan:master --reports-dir ${GITHUB_WORKSPACE}/containertests -t docker
rm -rf vdb_data_nydus
env:
PYTHONPATH: "."
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VDB_HOME: vdb_data_nydus
- name: Test without nydus
run: |
python3 depscan/cli.py --no-banner --no-error --src ghcr.io/owasp-dep-scan/depscan:latest --reports-dir ${GITHUB_WORKSPACE}/containertests -t docker
uv run depscan --no-banner --no-error --src ghcr.io/owasp-dep-scan/depscan:latest --reports-dir ${GITHUB_WORKSPACE}/containertests -t docker
env:
PYTHONPATH: "."
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -75,9 +77,9 @@ jobs:
rm *.zip
cd ..
oras pull ghcr.io/appthreat/vdb:v5 -o $VDB_HOME
python3 depscan/cli.py --no-error --src gobintests/terraform -o gobintests/depscan-terraform.json -t go
python3 depscan/cli.py --no-error --src gobintests/consul -o gobintests/depscan-consul.json -t go
python3 depscan/cli.py --no-error --src gobintests/minikube-linux-amd64 -o gobintests/depscan-minikube.json -t go
uv run depscan --no-error --src gobintests/terraform -o gobintests/depscan-terraform.json -t go
uv run depscan --no-error --src gobintests/consul -o gobintests/depscan-consul.json -t go
uv run depscan --no-error --src gobintests/minikube-linux-amd64 -o gobintests/depscan-minikube.json -t go
env:
PYTHONPATH: "."
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand Down
8 changes: 5 additions & 3 deletions .github/workflows/justbuild.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,20 @@ jobs:

steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
python3 -m pip install setuptools wheel twine build

uv sync --all-extras --dev
- name: Build
run: |
python3 -m build
uv build

- name: Upload a Build Artifact
uses: actions/upload-artifact@v3.1.2
Expand Down
22 changes: 12 additions & 10 deletions .github/workflows/pythonapp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ jobs:
fail-fast: false
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5
- name: Set up Python
uses: actions/setup-python@v5
with:
Expand All @@ -37,22 +39,22 @@ jobs:
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip setuptools
python3 -m pip install ".[dev]"
uv sync --all-extras --dev
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 --exclude test,contrib --exit-zero
uv run flake8 --exclude test,contrib --exit-zero
- name: Test with pytest
run: |
pytest --cov=depscan test
uv run pytest --cov=depscan test
- name: purl tests
run: |
python depscan/cli.py --purl "pkg:pypi/requests@2.32.1"
python depscan/cli.py --purl "pkg:pypi/requests@4.0.0"
python depscan/cli.py --purl "pkg:pypi/reqwestss@0.1.0"
python depscan/cli.py --purl "pkg:npm/%40appthreat/cdxgen@7.0.5"
python depscan/cli.py --purl "pkg:npm/%40appthreat/cdxgen@7.1.0"
python depscan/cli.py --purl "pkg:npm/fsevents@1.2.10"
python depscan/cli.py --purl "pkg:npm/@biomejs/biome@1.8.1"
uv run depscan --purl "pkg:pypi/requests@2.32.1"
uv run depscan --purl "pkg:pypi/requests@4.0.0"
uv run depscan --purl "pkg:pypi/reqwestss@0.1.0"
uv run depscan --purl "pkg:npm/%40appthreat/cdxgen@7.0.5"
uv run depscan --purl "pkg:npm/%40appthreat/cdxgen@7.1.0"
uv run depscan --purl "pkg:npm/fsevents@1.2.10"
uv run depscan --purl "pkg:npm/@biomejs/biome@1.8.1"
env:
PYTHONIOENCODING: utf-8
9 changes: 5 additions & 4 deletions .github/workflows/pythonpublish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,12 @@ jobs:
id-token: write
steps:
- uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v5
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'

- name: Set up Node.js
uses: actions/setup-node@v3
with:
Expand All @@ -51,11 +51,12 @@ jobs:

- name: Build
run: |
python3 -m build
uv build

- name: Publish package distributions to PyPI
if: startsWith(github.ref, 'refs/tags/')
uses: pypa/gh-action-pypi-publish@release/v1
run: |
uv publish

- name: Setup nydus
run: |
Expand Down
9 changes: 4 additions & 5 deletions .github/workflows/snapshot_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ jobs:
steps:

- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5

- name: Setup Python
uses: actions/setup-python@v5
Expand All @@ -29,9 +31,7 @@ jobs:
- name: Install depscan
run: |
python -m pip install --upgrade pip
python -m venv .venv
source .venv/bin/activate
pip install .
uv sync --all-extras --dev

- name: Cache vdb
id: cache-vdb
Expand All @@ -55,9 +55,8 @@ jobs:
DEPSCAN_CSAF_TEMPLATE: "/home/runner/work/dep-scan/dep-scan/contrib/csaf.toml"
INPUT_THANK_YOU: "I have sponsored OWASP-dep-scan."
run: |
source .venv/bin/activate
chmod +x /home/runner/work/dep-scan/dep-scan/test/snapshots.py
/home/runner/work/dep-scan/dep-scan/test/snapshots.py
uv run python /home/runner/work/dep-scan/dep-scan/test/snapshots.py
if test -f /home/runner/work/new_snapshots/diffs.json; then
echo "status=FAILED" >> "$GITHUB_ENV"
fi
Expand Down
12 changes: 6 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -91,13 +91,13 @@ RUN set -e; \
&& php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" && php composer-setup.php \
&& mv composer.phar /usr/local/bin/composer \
&& python3 -m pip install pipenv certifi \
&& curl -LO https://github.com/dragonflyoss/nydus/releases/download/v${NYDUS_VERSION}/nydus-static-v${NYDUS_VERSION}-linux-${GOBIN_VERSION}.tgz \
&& tar -xvf nydus-static-v${NYDUS_VERSION}-linux-${GOBIN_VERSION}.tgz \
&& chmod +x nydus-static/* \
&& mv nydus-static/* /usr/local/bin/ \
&& rm -rf nydus-static-v${NYDUS_VERSION}-linux-${GOBIN_VERSION}.tgz nydus-static \
&& curl -LsSf https://astral.sh/uv/install.sh | sh \
&& cd /opt/dep-scan \
&& python3 -m pip install -e . \
&& uv sync \
&& uv cache clean \
&& rm -r "$(uv python dir)" \
&& rm -r "$(uv tool dir)" \
&& rm ~/.local/bin/uv ~/.local/bin/uvx \
&& chmod a-w -R /opt \
&& rm -rf /var/cache/yum \
&& microdnf clean all
Expand Down
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,15 @@ curl --json '{"url": "https://github.com/HooliCorp/vulnerable-aws-koa-app", "typ
curl -X POST -H 'Content-Type: multipart/form-data' -F 'file=@/tmp/app/sbom_file.json' http://0.0.0.0:7070/scan?type=js
```

## Local development

Setup uv by following the official [documentation](https://docs.astral.sh/uv/).

```shell
uv sync --all-extras --dev
uv run depscan --help
uv run pytest
```

## License

Expand Down
2 changes: 1 addition & 1 deletion depscan/lib/analysis.py
Original file line number Diff line number Diff line change
Expand Up @@ -1176,7 +1176,7 @@ def parse_metrics(metrics):
vector = metric.cvssV4_0.vectorString
method = "CVSSv4"
severity = metric.cvssV4_0.baseSeverity.value
score = metric.cvssV4_0.baseScore.root
score = metric.cvssV4_0.baseScore.value
break
elif method != "CVSSv31" and (m := (metric.cvssV3_1 or metric.cvssV3_0)):
vector = m.vectorString
Expand Down
Loading
Loading