Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented NSIM similarity measurement. #64

Merged
merged 1 commit into from
May 17, 2024
Merged

Implemented NSIM similarity measurement. #64

merged 1 commit into from
May 17, 2024

Conversation

zond
Copy link
Collaborator

@zond zond commented May 15, 2024

  • Implemented NSIM (DOI:10.1016/j.specom.2011.09.004).
  • Replaced the geometric norm calculation across time and frequency axes with '1 - NSIM' when computing Zimtohrli distance.
  • Removed the redundant cpp/zimt/README.md.
  • Refactored the dataset management tools to enable optimization without saving the intermediary results.
  • Implemented simple simulated annealing optimization for some relevant Zimtohrli parameters.
  • Redid the MOS mapping so that Zimtohrli MOS scores are still on the same scale as before the NSIM change.

Before:

Score type MSE Min score Max score Mean score
ViSQOL 0.09 0.52 0.80 0.62
PESQ 0.10 0.48 0.84 0.61
Zimtohrli 0.10 0.56 0.74 0.59
CDPAM 0.12 0.44 0.73 0.57
PARLAQ 0.15 0.47 0.78 0.54
PEAQB 0.18 0.28 0.85 0.51
DPAM 0.28 0.19 0.69 0.42
WARP-Q 0.28 0.07 0.78 0.45
GVPMOS 0.32 0.01 0.78 0.40

After:

Score type MSE Min score Max score Mean score
ViSQOL 0.09 0.52 0.80 0.62
Zimtohrli 0.09 0.59 0.72 0.60
PESQ 0.10 0.48 0.84 0.61
CDPAM 0.12 0.44 0.73 0.57
PARLAQ 0.15 0.47 0.78 0.54
PEAQB 0.18 0.28 0.85 0.51
DPAM 0.28 0.19 0.69 0.42
WARP-Q 0.28 0.07 0.78 0.45
GVPMOS 0.32 0.01 0.78 0.40

See https://github.com/google/zimtohrli/pull/64/files?short_path=4640799#diff-46407991f9df1906835213b977238d969f72c49974a7dda6a9b1276daf199221 for the detailed performance changes.

- Implemented NSIM (DOI:10.1016/j.specom.2011.09.004).
- Replaced the geometric norm calculation across time and frequency axes
  with '1 - NSIM' when computing Zimtohrli distance.
- Removed the redundant cpp/zimt/README.md.
- Refactored the dataset management tools to enable optimization without
  saving the intermediary results.
- Implemented simple simulated annealing optimization for some relevant
  Zimtohrli parameters.
- Redid the MOS mapping so that Zimtohrli MOS scores are still on the
  same scale as before the NSIM change.

Before:

|Score type |MSE  |Min score |Max score |Mean score |
|-----------|-----|----------|----------|-----------|
|ViSQOL     |0.09 |0.52      |0.80      |0.62       |
|PESQ       |0.10 |0.48      |0.84      |0.61       |
|Zimtohrli  |0.10 |0.56      |0.74      |0.59       |
|CDPAM      |0.12 |0.44      |0.73      |0.57       |
|PARLAQ     |0.15 |0.47      |0.78      |0.54       |
|PEAQB      |0.18 |0.28      |0.85      |0.51       |
|DPAM       |0.28 |0.19      |0.69      |0.42       |
|WARP-Q     |0.28 |0.07      |0.78      |0.45       |
|GVPMOS     |0.32 |0.01      |0.78      |0.40       |

After:

|Score type |MSE  |Min score |Max score |Mean score |
|-----------|-----|----------|----------|-----------|
|ViSQOL     |0.09 |0.52      |0.80      |0.62       |
|Zimtohrli  |0.09 |0.59      |0.72      |0.60       |
|PESQ       |0.10 |0.48      |0.84      |0.61       |
|CDPAM      |0.12 |0.44      |0.73      |0.57       |
|PARLAQ     |0.15 |0.47      |0.78      |0.54       |
|PEAQB      |0.18 |0.28      |0.85      |0.51       |
|DPAM       |0.28 |0.19      |0.69      |0.42       |
|WARP-Q     |0.28 |0.07      |0.78      |0.45       |
|GVPMOS     |0.32 |0.01      |0.78      |0.40       |
@zond zond enabled auto-merge (rebase) May 17, 2024 07:14
@zond zond merged commit e92ee29 into main May 17, 2024
2 checks passed
@zond zond deleted the nsim branch May 17, 2024 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants