Twitter search failed with error 403

[stevenirby]

Error message: Unexpected response from upstream. cUrl error: (0) PHP error:
Query string: action=display&bridge=Twitter&context=By+keyword+or+hashtag&format=Json&q=<some string here>
Version: dev.2021-04-25

I recently upgraded rss-bridge with the latest version I'm getting a ton of exceptions now.

[em92]

Works fine on my instance.
https://feed.eugenemolotov.ru/?action=display&bridge=Twitter&context=By+keyword+or+hashtag&format=Html&q=hello

[berenddeschouwer]

I get this too, and I wonder if this is related to the fix for #2366

[triatic]

I have a couple of keyword searches which work fine after applying #2366 .

[berenddeschouwer]

I have a lot of feeds that work fine.

Every now and then (once a day?) I get a "403 curl (0)", and if I back out #2366 I get a "could not parse guest token" once a day.

I'm playing with the values, but it feels a bit random right now. I need a better testing strategy.

Note that I haven't seen both 403 and "could not parse guest token" simultaneously yet, but it could happen. "Once a day" doesn't make for a quick turnaround in testing.

[triatic]

We know what the guest token expiry time is because it is set by a cookie with a given expiry time.

The maximum number of guest token uses is more of a mystery, it is set at 100 at the moment and I have no issue with 90.

If anything the guest token expiry is more conservative than it needs to be, so I am unsure why some people are seeing 403 forbidden errors.

[berenddeschouwer]

Cool, binary searching between 50 and 100 for now.

[berenddeschouwer]

OK, so:

getApiContents() receives 403 from Twitter: (replaced the guest token with zzz)

HTTP/2 403
set-cookie: guest_id_marketing=v1%3Azzz; Max-Age=63072000; Expires=Fri, 22 Dec 2023 09:40:06 GMT; Path=/; Domain=.twitter.com; Secure; SameSite=None
set-cookie: guest_id_ads=v1%3Azzz; Max-Age=63072000; Expires=Fri, 22 Dec 2023 09:40:06 GMT; Path=/; Domain=.twitter.com; Secure; SameSite=None
set-cookie: guest_id=v1%3Azzz; Max-Age=63072000; Expires=Fri, 22 Dec 2023 09:40:06 GMT; Path=/; Domain=.twitter.com; Secure; SameSite=None
content-type: application/json;charset=utf-8
cache-control: no-cache, no-store, max-age=0
content-length: 79
content-encoding: gzip
x-response-time: 100
x-connection-hash: bdb5dc1...
date: Wed, 22 Dec 2021 09:40:06 GMT
server: tsa_f

There is a guest token in the outgoing headers

The guest token was obtained at 09:20 (local time), was used 3 times successfully, and triggered a 403 at 10:40 (local time)

That then triggers a PHP error in:
lib/error.php:24
#0 lib/contents.php(203): returnError()
#1 bridges/TwitterBridge.php(580): getContents()
#2 bridges/TwitterBridge.php(586): TwitterBridge->getApiContents()
#3 bridges/TwitterBridge.php(221): TwitterBridge->getRestId()
#4 bridges/TwitterBridge.php(241): TwitterBridge->getApiURI()
#5 actions/DisplayAction.php(135): TwitterBridge->collectData()
#6 index.php(40): DisplayAction->execute()
#7 {main}
index.php:40 class DisplayAction->execute - URI must include a scheme, host and path!

[triatic]

"URI must include a scheme, host and path!" doesn't fit with what I see when I try to use an expired token.

I see "Exception: Unexpected response from upstream."

[berenddeschouwer]

"Exception: Unexpected response from upstream." is what's displayed to the rss-bridge client.

"URI must include a scheme, host and path!" is in the debug backtrace.