Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

small differences on the same image captured on headless : true / false mode #156

Closed
andreabisello opened this issue Oct 21, 2019 · 21 comments
Closed

small differences on the same image captured on headless : true / false mode #156

andreabisello opened this issue Oct 21, 2019 · 21 comments

Comments

@andreabisello
Copy link

in some case,
taking screenshot of some images,
i obtain a small differences of the same images changing between headless or not headless mode of puppeteer.

everything updated to latest version.

login-page-test-js-test-delle-funzionalita-della-pagina-di-login-hai-dimenticato-la-password-1-diff

as you can see,

letters in the left image looks a little bulkier then the same letters on the right one ( you can see in the R letter )

the problem can be in jest-image-snapshot or i should move on puppeteer project?

thanks.
@andreabisello
Copy link
Author

there is a font problem in the website.
a required font is not found.
maybe the problem is related with the browser callback chain when font is not found.
now im asking to restore the font,
if the problem is related with the font,
i will move this issue on google puppeteer.

@andreabisello
Copy link
Author

@anescobar1991 the font issue was solved but the problem stay here.
i was able to reproduce the problem in a public environment.
please clone my repository https://github.com/andreabisello/jest-image-snapshot-example
you will find a test in the file headless-differences.test.js called "headless and not headless screenshot should to be the same".
run in headless mode : the screenshot will be generated.
run in not headless mode : the screenshot will be generated, but there is a 1.22% different,
even the page is the same.
as you can see, fonts are rendered in a different way so screenshots are differents.

image

i'm moving to asking support on puppeteer homeproject.

@mdugue
Copy link

mdugue commented May 28, 2020

@andreabisello were you able to solve your issue?

@andreabisello
Copy link
Author

@mdugue i will upgrade puppeteer, jest and jest-images-snapshot to the latest version and i will try , wait a moment

@andreabisello
Copy link
Author

@mdugue i updated my project with latest version of puppeteer, jest and jest-image-snapshot, problem still the same.

@andreabisello
Copy link
Author

@anescobar1991 the problem was not related with the font as i described when the issue was closed because the font issue was solved but the difference still there. can you reopen the issue please?

@mdugue
Copy link

mdugue commented May 28, 2020

@anescobar1991 yes I can confirm that this problem still exists

@anescobar1991
Copy link
Member

The issue is not with jest-image-snapshot sadly, all jest-image-snapshot does is do an image comparison. Have you tried playing with the blur options to see if they help?

Can also watch this issue #201 as when implemented that should solve a lot of these issues.

This is why image comparison is hard, changes that are not visible to the human eye are still caught and cause the comparison to fail.

@mdugue
Copy link

mdugue commented Jun 2, 2020

@anescobar1991 that makes sense. Thanks. SSIM would be 💯. Do you know any good examples with the blurring approach? I'd be curious on good thresholds ets

@omnisip
Copy link
Contributor

omnisip commented Jun 17, 2020

in some case,
taking screenshot of some images,
i obtain a small differences of the same images changing between headless or not headless mode of puppeteer.

everything updated to latest version.

login-page-test-js-test-delle-funzionalita-della-pagina-di-login-hai-dimenticato-la-password-1-diff

as you can see,

letters in the left image looks a little bulkier then the same letters on the right one ( you can see in the R letter )

the problem can be in jest-image-snapshot or i should move on puppeteer project?

thanks.

@andreabisello If you can send me this two images (original and update snapshots) as the original pngs, I can add them to the test suite in PR #220

@andreabisello
Copy link
Author

andreabisello commented Jun 19, 2020 •
edited
Loading

@omnisip thanks

this is the test to obtain the images

the images are the login box container of this url https://legaltest.solutiondocondemand.com , withouth the timepicker at the bottom of the box.

this is the one captured in headless mode

captured in headless mode

and this is the one captured in gui mode

captured_in_gui_mode

@omnisip
Copy link
Contributor

omnisip commented Jun 19, 2020 via email

@andreabisello
Copy link
Author

@omnisip sure, and this is the initial problem : the same images captured on headless / gui mode has small differences : who is generating the images? puppeteer or jest-image-snapshot?

@omnisip
Copy link
Contributor

omnisip commented Jun 19, 2020 via email

@andreabisello
Copy link
Author

@omnisip ok so the issue can be closed, i'll move on puppeteer.
thanks.

@omnisip
Copy link
Contributor

omnisip commented Jun 19, 2020 via email •
edited
Loading

@andreabisello
Copy link
Author

@omnisip the test was already waiting for network activities

await page.goto("https://legaltest.solutiondocondemand.com", {
            timeout: 20000,
            waitUntil: ['load', 'domcontentloaded', 'networkidle0', 'networkidle2']
        })

anyway i added

await page.waitFor(20000)

but nothing changed.

i also tried upgrading everything to latest

package.json

"dependencies": {
    "jest": "latest",
    "jest-image-snapshot": "latest",
    "puppeteer": "latest",
    "puppeteer-extra": "latestnpn"
  }

updating chromium to r756035

and nothing changed.

@andreabisello
Copy link
Author

i attached the two screenshot using page.screenshot api ,
the problem is puppeter / chromium related because you can easily see the difference in the the login button.
i will move on puppeter.
thanks.
headless

gui

@andreabisello
Copy link
Author

i created puppeteer/puppeteer#6101

@omnisip
Copy link
Contributor

omnisip commented Jun 25, 2020 via email

@ychaudhari ychaudhari mentioned this issue Feb 9, 2021
Closed
@andreabisello
Copy link
Author

@omnisip

probably this could explain our issues

https://developer.chrome.com/articles/new-headless/

as i was thinking, chrome headless and chrome gui was not the same chrome.

this could explain the small rendering differences.

i will reactivate the image comparison soon after updating to the new parameter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@omnisip @mdugue @andreabisello @anescobar1991