Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(perf): improve validations, recursions, logs and imports #346

Merged
merged 11 commits into from
Jun 9, 2024
1 change: 1 addition & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
/lib
/benchmark
/ci
/CHANGELOG.md
/website
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
website/** linguist-documentation
test/docker/** linguist-vendored
fixtures/** linguist-vendored
benchmark/** linguist-vendored
src/bin/** linguist-language=TypeScript
44 changes: 44 additions & 0 deletions .github/workflows/ci_benchmark.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: '🎖️ CI — Benchmark'

on:
pull_request:
workflow_dispatch:

jobs:
benchmark:
runs-on: ubuntu-latest
timeout-minutes: 5
strategy:
fail-fast: false
name: Compare
steps:
- name: ➕ Actions - Checkout
uses: actions/checkout@v4

- name: ➕ Actions - Setup NodeJS
uses: actions/setup-node@v4
with:
node-version: '22.x'

- name: ➕ Cache dependencies
uses: actions/cache@v4
with:
path: ~/.npm
key: npm-linux-${{ hashFiles('package-lock.json') }}
restore-keys: npm-linux-

- name: ➕ Cache dependencies (Benchmark)
uses: actions/cache@v4
with:
path: ~/.npm
key: npm-benchmark-${{ hashFiles('benchmark/package-lock.json') }}
restore-keys: npm-benchmark-

- name: 📦 Installing Dependencies
run: npm ci

- name: 🚀 Building Poku
run: npm run build

- name: 🎖️ Rock it
run: npm run benchmark
2 changes: 1 addition & 1 deletion .nycrc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"all": true,
"include": ["src/**"],
"exclude": ["ci", "fixtures", "lib", "test", "tools", "website"],
"exclude": ["benchmark", "ci", "fixtures", "lib", "test", "tools", "website"],
"extension": ["ts"],
"reporter": ["text", "lcov", "cobertura"],
"clean": true,
Expand Down
34 changes: 29 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,10 @@ Enjoying **Poku**? Give him a star to show your support ⭐️
<span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><img width="16" height="16" alt="check" src="https://raw.githubusercontent.com/wellwelwel/poku/main/.github/assets/readme/check.svg"> Easily test your server just by running it 🚀<br />
<span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><img width="16" height="16" alt="check" src="https://raw.githubusercontent.com/wellwelwel/poku/main/.github/assets/readme/check.svg"> Run **CJS** (**CommonJS**) files directly with [**Deno**][deno-version-url]<br />
<span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><img width="16" height="16" alt="check" src="https://raw.githubusercontent.com/wellwelwel/poku/main/.github/assets/readme/check.svg"> Easily handle **services**, **servers**, **processes** and **ports**<br />
<span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><img width="16" height="16" alt="check" src="https://raw.githubusercontent.com/wellwelwel/poku/main/.github/assets/readme/check.svg"> [**Node.js**][node-version-url] familiar API<br />

<img width="16" height="16" alt="check" src="https://raw.githubusercontent.com/wellwelwel/poku/main/.github/assets/readme/check.svg"> Safety and Reliability<br />
<span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><img width="16" height="16" alt="check" src="https://raw.githubusercontent.com/wellwelwel/poku/main/.github/assets/readme/check.svg"> Performant and lightweight<br />
<span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><img width="16" height="16" alt="check" src="https://raw.githubusercontent.com/wellwelwel/poku/main/.github/assets/readme/check.svg"> High **isolation** level per file<br />
<span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><img width="16" height="16" alt="check" src="https://raw.githubusercontent.com/wellwelwel/poku/main/.github/assets/readme/check.svg"> Compatible with **Coverage** tools<br />
<span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><img width="16" height="16" alt="check" src="https://raw.githubusercontent.com/wellwelwel/poku/main/.github/assets/readme/check.svg"> **Poku** doesn't use `eval` nor global state 🔐<br />
Expand Down Expand Up @@ -217,16 +219,32 @@ Please check the [**SECURITY.md**](./SECURITY.md) and the section [**Is Poku Saf

---

## Limitations
## Quick comparisons

- **Poku** community is gradually building up 🤝
- Although it has no external dependencies, **Poku** is not _all-in-one_, so it doesn't have features such as _mocks_ and _spies_, where you can use your favorite packages or native solutions.
### Performance

**Poku** is continuously tested ([**CI**](https://github.com/wellwelwel/poku/blob/main/.github/workflows/ci_benchmark.yml)) to ensure the following expectations:

- [x] **~4x** faster than [**Jest**](https://github.com/jestjs/jest) (v29.7.0)
- [x] **~3x** faster than [**Vitest**](https://github.com/vitest-dev/vitest) (v1.6.0)
- [x] **~1x** faster than [**Mocha**](https://github.com/mochajs/mocha) (v10.4.0) + [**Chai**](https://github.com/chaijs/chai) (v5.1.1)

> You can see how the tests are run and compared in the [benchmark](https://github.com/wellwelwel/poku/tree/main/benchmark) directory.

---

## License
### Installation Size

Poku is under the [**MIT License**](./LICENSE).
- [x] ~**175x** lighter than Jest <a href="https://pkg-size.dev/jest"><img src="https://pkg-size.dev/badge/install/21981956" title="Install size for jest"></a>
- [x] ~**302x** lighter than Vitest <a href="https://pkg-size.dev/vitest"><img src="https://pkg-size.dev/badge/install/38368348" title="Install size for vitest"></a>
- [x] ~**44x** lighter than Mocha + Chai <a href="https://pkg-size.dev/mocha chai"><img src="https://pkg-size.dev/badge/install/5548077" title="Install size for mocha, and chai"></a>

---

### Limitations

- **Poku** community is gradually building up 🤝
- Although it has no external dependencies, **Poku** is not _all-in-one_, so it doesn't have features such as _mocks_ and _spies_, where you can use your favorite packages or native solutions.

---

Expand All @@ -235,3 +253,9 @@ Poku is under the [**MIT License**](./LICENSE).
[![Contributors](https://img.shields.io/github/contributors/wellwelwel/poku?style=flat-square)](https://github.com/wellwelwel/poku/graphs/contributors)

[![Contributors](https://opencollective.com/poku/contributors.svg?width=890&button=false)](https://github.com/wellwelwel/poku/graphs/contributors)

---

## License

Poku is under the [**MIT License**](./LICENSE).
1 change: 1 addition & 0 deletions benchmark/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
node_modules
35 changes: 35 additions & 0 deletions benchmark/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Benchmark

The benchmark is performed by comparing a simple success and a failure test, each in a respective file and using parallel/concurrent runs.

> [!important]
>
> **Poku** doesn't intend to be the best, but to offer a balance between high performance, compatibility, lightness and ease of use.

---

The testers to be compared are chosen based on the three most downloaded test runners according to the **npm** weekly statistics _(2024/06/09)_:

- [**Jest**](https://www.npmjs.com/package/jest): 23,549,369
- [**Mocha**](https://www.npmjs.com/package/mocha) + [**Chai**](https://www.npmjs.com/package/chai): 8,053,244 + 11,294,912
- [**Vitest**](https://www.npmjs.com/package/vitest): 4,840,171

---

**Poku** is continuously tested ([**CI**](https://github.com/wellwelwel/poku/blob/main/.github/workflows/ci_benchmark.yml)) to ensure the following expectations:

- [x] **~4x** faster than [**Jest**](https://github.com/jestjs/jest) (v29.7.0)
- [x] **~3x** faster than [**Vitest**](https://github.com/vitest-dev/vitest) (v1.6.0)
- [x] **~1x** faster than [**Mocha**](https://github.com/mochajs/mocha) (v10.4.0) + [**Chai**](https://github.com/chaijs/chai) (v5.1.1)

---

## Running

To run the benchmark tests, follow these steps:

```sh
npm ci
npm run build
npm run benchmark
```
78 changes: 78 additions & 0 deletions benchmark/benchmark.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
import { execSync } from 'node:child_process';
import Benchmark from 'benchmark';

const suite = new Benchmark.Suite();
const results = new Map();

const exec = (command) => {
try {
execSync(command, { stdio: 'ignore' });
} catch {}
};

const test = () => {
const pokuResult = results.get('Poku (Local)');

const tolerancesPerTester = {
Jest: 4,
'Mocha + Chai': 1,
Vitest: 3,
Poku: 0.8,
};

for (const [name, result] of results) {
if (name === 'Poku (Local)') continue;

const expectedRatio = tolerancesPerTester[name];
const actualRatio = pokuResult.opsPerSec / result.opsPerSec;
const isAtLeastExpected = actualRatio >= expectedRatio;

if (!isAtLeastExpected) {
console.error(
`Poku (Local) isn't approximately ${expectedRatio.toFixed(2)}x faster than ${name}.`
);
process.exit(1);
}
}
};

suite
.add('Jest ', () => {
exec(
'node --experimental-vm-modules ./node_modules/jest/bin/jest.js ./test/jest'
);
})
.add('Mocha + Chai ', () => {
exec('./node_modules/mocha/bin/mocha.js --parallel ./test/mocha');
})
.add('Vitest ', () => {
exec('./node_modules/vitest/vitest.mjs run ./test/vitest');
})
.add('Poku ', () => {
exec('./node_modules/poku/lib/bin/index.js --parallel ./test/poku');
})
.add('Poku (Local) ', () => {
exec('../lib/bin/index.js --parallel ./test/poku');
})
.on('cycle', (event) => {
const name = event.target.name.trim();
const result = {
opsPerSec: event.target.hz,
percentage: event.target.stats.rme,
};

results.set(name, result);
console.log(
`${event.target.name} x ${result.opsPerSec.toFixed(2)} ops/sec — ±${result.percentage.toFixed(2)}% (${event.target.stats.sample.length} runs sampled)`
);
})
.on('complete', function () {
const fatest = String(this.filter('fastest').map('name')).trim();

console.log(`\n🚀 Fastest is \x1b[1m${fatest}\x1b[0m\n`);

if (!/^Poku/.test(fatest)) process.exit(1);

test();
})
.run({ async: true });
Loading
Loading