Cypress doesn't always clean up cookies

[verheyenkoen]

Current behavior:

We have a test that checks if an unauthorized user is redirected to the login page when directly visiting a URL to the "backend" site. This test occasionally fails. From the screenshot it is clear that the user was just sent to the specific backend page without login.

I checked (console props of the cy.visit command & cy.getCookies + cy.writeFile) and in some cases the auth cookie from the previous test is not cleared. For that I'm using a custom login command that only fetches cookies once (per username) and stores them in the global scope. Then I use cy.setCookie() to set the cookie again.

In this case the cookie was set in the beforeEach hook of the previous test.

Obviously this never fails if I just run the latter spec.

Desired behavior:

Obviously, by default (unless Cypress.Cookies.preserveOnce() is used), all cookies should be cleared before a test run starts.

How to reproduce:

This is hard. I couldn't narrow it down to one specific always-reproducable scenario but these are things I can confidently say about it:

• I've seen it fail multiple times in headless runs
• I've seen it fail multiple times in headed runs (cypress run --headed)
• I've seen this fail one time in the UI (cypress open)
• I've seen it fail in all 3 browsers in headed runs

I'm not sure but it seems to fail easier if I change the focus to other apps (or spaces) while running the tests, but even that isn't deterministic for me.

• Operating System: Mac OS X Sierra
• Cypress Version: 1.0.2
• Browser Version: All

[verheyenkoen]

I simplified my login command (because it also produced an application specific side effect and didn't really turn out to save much time), just keeping the cy.request statement and not storing cookies anymore and the problem is still there so it's probably not related to cy.setCookie.

[brian-mann]

Do you think this is related to this issue? #408

[verheyenkoen]

@brian-mann I checked that one. It's not a cross-domain login so I guess not. Am I right that I should never manually call clearCookies in an after hook?

[brian-mann]

Correct, Cypress automatically clears cookies prior to each test.

You can use the cookie debug API to print every single modification to every single cookie.

https://docs.cypress.io/api/cypress-api/cookies.html#Debug

[verheyenkoen]

Is there any way to dump the command log of a cypress run (apart from video recording). Cypress.Cookies.debug() may lead me to the problem indeed but this one is very hard to reproduce in the UI, so I'm looking for a way to review these cookie changes.

[brian-mann]

#700
#448

[pawelgalazka]

Can confirm, Cypress does not clear the cookies between tests, it let it leaks. This not happens all the time though.

[webdevian]

We are having this problem intermittently on Mac OS X Sierra w/ Chrome 63 and Electron 53 on the first test in our spec.

Printing out cy.getCookies() appears to show the cookies are cleared between tests but when we print out the request cookies in php there are some cookies there.

It has been hard to replicate this, there doesn't seem to be a pattern to force it to happen.

[verheyenkoen]

On my end also the first test in the spec that fails. Obviously only if other specs have been run before.

[antonfisher]

In my case it was silly, I had whitelist configuration to keep session id between different cy.location() calls and I forgot about it:

Cypress.Cookies.defaults({whitelist: 'sid'});

So I need to clean up this particular cookie before run whole test spec: before(() => cy.clearCookie('sid'));.
Just in case.

[verheyenkoen]

I'm not using that feature so that's not it.

[webdevian]

Any luck with v2?

[verheyenkoen]

Didn't see this fail anymore. Were there any changes on this part?

[webdevian]

I have noticed it happen still

[verheyenkoen]

Ok now I have too.

[lukemadera]

I'm having this issue as well.

[djalmaoliveira]

I have some problem with cookies/session, my workaround was first select "electron" and run tests that will not work properly, after that select "Chrome" and run tests that work.

Cypress package version: 2.1.0
Cypress binary version: 2.1.0
OS: Ubuntu 17.10

[anurbol]

I have the problem too, cookies persist. That means I am always logged in and my login test fails. Problem is on both Windows 7 and Linux (Debian 9.4), in Chrome, Cypress 2.1.0.

Can this be an issue of Chrome?

I noticed some interesting pattern: cookies are cleared as expected, when I:

1. cypress open
2. run all tests
3. (old cookies are there, user is logged in) stop tests by clicking stop button
4. F12 (open DevTools)
5. restart tests -> everything is OK

[scrat98]

the same problem

[kc-beard]

same problem. cookies are persisting.

[anurbol]

My temporary workaround:

describe('Auth', function () {

    before(() => {

        // remove this when issue will be solved
        // https://github.com/cypress-io/cypress/issues/781
        cy.visit('/')
        cy.clearCookies()
        cy.reload()
    })

...

Place this code at the very beginning of your tests (I guess they are ordered alphabetically).

[kc-beard]

@anurbol we went with something similar, but we have several describe blocks across different files so it's a pain to have to do this everywhere.

[verheyenkoen]

@kc-beard You can also define a beforeEach routine in your support/index.js file, which would be executed before each test in any other file, but that's probably overkill...

[kc-beard]

@verheyenkoen thanks for the tip! I didn't know that. Seems like that's covered in the docs too but i missed it (https://docs.cypress.io/guides/core-concepts/writing-and-organizing-tests.html#Support-file)

[anurbol]

@kc-beard in my experience, placing this in one file (which is starting point of your tests) is enough. verheyenkoen's answer is 2nd option, but as said, overkill (slows down tests execution significally).

[KrisBraun]

This is definitely a significant issue! Feel like something that requires warnings in the docs until it's resolved, as it took us a while to figure out why tests were failing when the documentation claims that cookies are cleared between tests.

[waldobeest]

@verheyenkoen I am upgraded to 8.2.0 and came to this issue due to the problem occurring for me. So this is still an issue.

[WinstonN]

Still the same issue for me too on 8.2.0

[Jacek-fstack]

@waldobeest @WinstonN you guys need to enable the new experimental feature- worked for me

[keatsk]

Like many others have noted in our case the problem was that we wanted to clear a JWT cookie that had a domain set like mycompany.com instead of say localhost so these cookies won't be cleared by cypress. So I had to use this undocumented feature with a fix for TypeScript since our tests use .ts files:

// @ts-ignore
cy.clearCookies({ domain: null })

This worked for me! Specifying the actual domain also works but this is even better. Thanks!

[jennifer-shehane]

We released cy.session() as an experimental command (set experimentalSessionSupport to true in config) in Cypress 8.2.0. This command can be used to cache and restore cookies, localStorage, and sessionStorage. We especially see this command as being useful for testing Auth flows and recommend giving it a try to replace your existing tests around authentication and login.

If you have any feedback about cy.session() for general UX improvements, new features, or unexpected behavior, please leave your feedback in our Cypress Session Experimental Feedback Discussion.

[pgmagno]

Clearing cookies should be delayed.

This one works fine, at least on my side the reason seems like in the application itself, sometimes cookies might be set after some time application boots/loads.

      cy.wait(100)
      cy.clearCookies()

I found something similar to what you describe, although the solution didn't seem to work, sadly. Upon closer inspection, I noticed that the cookies are cleared before each test, and then, for some reason, get reloaded. You can clearly see that the website is resetting to default values every single time, as it is supposed to per described in the documentation, but then the values from a previous session get reloaded. I tried simply disabling the cookies altogether in the web driver (chrome in my case), but this broke cypress, so I had to reset it to allow cookies.

PS: I am noob, just trying to help

[jremmurd]

Shouldn't this be high prio and not backlog? The issue is now open for years and still not resolved, leaving us with flaky tests when dealing with cookies/session.

[samtsai]

@jremmurd have you tried using the experimental session API?

https://docs.cypress.io/api/commands/session

[jremmurd]

@jremmurd have you tried using the experimental session API?

https://docs.cypress.io/api/commands/session

I dont want to share cookies between tests (as the original issue description says) and this is what the new API would provide, right? The thing is that I would expect all cookies to be cleared between separate tests, but they are not always. Also manually clearing them does not work reliably. How would the session API help me with that?

As from my previous post:

cy.clearCookies() // or other workarounds
cy.getCookies().should('be.empty') // succeeds
cy.getCookies().should('be.empty') // fails

[samtsai]

@jremmurd I've adopted the Session api because it:

• The page is cleared (by setting it to about:blank).
• All active session data (cookies, localStorage and sessionStorage) across all domains are cleared.

It's possible that some other domain cookie is set causing the second check to fail, but I cannot be sure.

With Session API you can decide what cookies to share, if your case is none, you should be able to run cy.clearCookies() within your session setup step. The new API is also nice from a performance perspective because it can keep a session "alive" (via caching session context) while being able to switching between sessions if your tests require it.

[samtsai]

@jremmurd I'd also be curious, maybe you can print out the cookies that is returned from cy.getCookies()

[ZachJW34]

I'm going to close this issue because this is solved by enabling the experimentalSessionAndOrigin flag and utilizing cy.session.

[michaelmdresser]

Is there a timeline for moving cy.session out of "experimental" status and into a more stable status?

[michielswaanen]

[emilyrohrbough]

@michaelmdresser We will be releasing Session as GA in the upcoming v12 release. We are expecting to release in early December.

[alestrunda]

cy.clearCookies()
cy.getCookies().should('be.empty') fails

I tried a couple of workarounds above without any help, but strangely enough, calling cy.clearCookies() twice fixed the issue.

[RodrigoLopesRosa]

Like many others have noted in our case the problem was that we wanted to clear a JWT cookie that had a domain set like mycompany.com instead of say localhost so these cookies won't be cleared by cypress. So I had to use this undocumented feature with a fix for TypeScript since our tests use .ts files:

// @ts-ignore
cy.clearCookies({ domain: null })

Thanks bro, your answer solved my problem =)

[chlorophant]

Still an issues as of march 2024

[jennifer-shehane]

@chlorophant Please open a new issue describing it and providing a full example of it. We recommend using cy.session.

[PetrKnedlik]

I just stumbeled on a similar issue right now. I had a full on e2e login & logout test and when I later wanted to just login with a request, upon logout it does not clear the coockie (which does when I do it 'manually' via typing and clicking with cypress (instead of cy.request /login auth endpoint and saving token