[API] `getRead` attribute behavior change in 15.4.0 #1328

HorlogeSkynet · 2021-05-02T14:02:20Z

IMPORTANT

Read and tick the following checkbox after you have created the issue or place an x inside the brackets ;)

I have read the CONTRIBUTING.md and followed the provided tips
I accept that the issue will be closed without comment if I do not check here
I accept that the issue will be closed without comment if I do not fill out all items in the issue template.

Explain the Problem

Hello over here ! Actually more of a question than a "bug".
I've noticed an API (breaking) change after the 15.4.0 update. It looks like the getRead parameter is not taken into account anymore.
From CHANGELOG, I thought about a #1056-related change ?
So, is the legacy documentation now outdated or is it more of a regression ?

Thanks for your time 🙏

Steps to Reproduce

Playing with apps/news/api/v1-2/items?getRead=false (legacy) API endpoint
Get the whole items set instead of only the unread ones

System Information

News app version: 15.4.0
Nextcloud version: 21
Cron type: irrelevant
PHP version: 7.3
Database and version: irrelevant
Browser and version: irrelevant
OS and version: irrelevant

The text was updated successfully, but these errors were encountered:

SMillerDev · 2021-05-03T09:25:54Z

getRead wasn't changed so it should still work as expected. What do you get when you pass 0?

HorlogeSkynet · 2021-05-03T11:03:53Z

I may retry if you want but when I tried yesterday it didn't change a thing 😔

HorlogeSkynet · 2021-05-03T18:45:54Z

So I've just tried again, same behavior with 0 : the whole items set is written back...

curl \
	-H "Accept: application/json" \
	-H "Authorization: Basic XXXXXXXXXXXXXX" \
	"https://nextcloud.domain.name/apps/news/api/v1-2/items?getRead=0" \
		| json_pp \
			| tail -n 6

         "unread" : false,
         "updatedDate" : null,
         "url" : "..."
      }
   ]
}

Thanks for your feedback. 👋

captainfodder · 2021-05-04T16:57:57Z

I may not be contributing much diagnostic data, but I can confirm that since the 15.4.0 update that third party apps are just showing all entries as unread such as Fiery Feeds on iOS. That app was last updated 2mo ago and was working properly until the 15.4.0 update went live on my Nextcloud instance. Since then every time the app syncs it pulls all items as unread. I did update to the latest 15.4.2 version and the issue still exists as well.

Based on what I've seen so far, it appears like an issue in the the 15.4.0 update. However if this is due to feature deprecation and the iOS app developer needs to update his app to use new methods please let me know and I'll pass the info along to the iOS dev.

SMillerDev · 2021-05-04T19:21:40Z

could you try applying https://patch-diff.githubusercontent.com/raw/nextcloud/news/pull/1340.patch and see if that fixes it?

HorlogeSkynet · 2021-05-04T19:31:08Z

Sorry but it does not change a thing, still getting the whole items set 🙄

HorlogeSkynet · 2021-05-07T16:36:56Z

Any "news" on this @SMillerDev ? I can do more tests if you want. Bye 👋

SMillerDev · 2021-05-10T06:52:26Z

I'm still thinking what could have caused this, but coming up empty. Maybe it's the boolean/no-boolean magic happening here: #1344 ?

HorlogeSkynet · 2021-05-10T18:33:32Z

Nice catch, definitely seems related. I even wonder whether @tweitzel has already nailed down this issue. As the other OP, I also use PostgreSQL (I thought this detail was irrelevant here, it looks like Web development kidded me again).
I guess you couldn't reproduce this on your (MariaDB) setup ?

tweitzel · 2021-05-10T19:01:59Z

I'm reproducing it on MariaDB. Holler if you have test cases and commands to run that would capture the data you need.

tweitzel · 2021-05-13T20:11:45Z

OK here we go. I turned on the mysql query log on my instance and did a bunch of tests. The web UI appears to be working correctly. The API client (fiery feeds on ios) is not.

Starting with our test object:

original:
MariaDB [owncloud]> select id,title,unread from oc_news_items where id=19095;
+-------+-----------------------------------------------------------------------+--------+
| id    | title                                                                 | unread |
+-------+-----------------------------------------------------------------------+--------+
| 19095 | Patrick Marleau answers question as to whether he’s going to retire   |      1 |
+-------+-----------------------------------------------------------------------+--------+
1 row in set (0.000 sec)

I pull to refresh in the feed that it belongs to:

172.16.1.2 - xarph [13/May/2021:12:40:18 -0700] "GET /nextcloud/index.php/apps/news/api/v1-2/feeds HTTP/1.1" 200 2715
172.16.1.2 - xarph [13/May/2021:12:40:18 -0700] "GET /nextcloud/index.php/apps/news/api/v1-2/folders HTTP/1.1" 200 115
172.16.1.2 - xarph [13/May/2021:12:40:19 -0700] "GET /nextcloud/index.php/apps/news/api/v1-2/items?batchSize=-1&getRead=false&type=3 HTTP/1.1" 200 4923728
172.16.1.2 - xarph [13/May/2021:12:40:20 -0700] "GET /nextcloud/index.php/apps/news/api/v1-2/items?batchSize=-1&getRead=true&type=2 HTTP/1.1" 200 32
		 22453 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) AND (starred = '1')
		 22453 Query	SELECT `feeds`.*, COUNT(`items`.`id`) AS `unreadCount` FROM `oc_news_feeds` `feeds` LEFT JOIN `oc_news_items` `items` ON items.feed_id = feeds.id AND items.unread = 1 WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) GROUP BY `feeds`.`id`
		 22453 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE feeds.user_id = 'xarph' ORDER BY `items`.`last_modified` DESC, `items`.`id` DESC LIMIT 1
		 22455 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) ORDER BY `items`.`id` DESC
		 22456 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) AND (items.starred = '1') ORDER BY `items`.`id` DESC

Opening the item:

172.16.1.2 - xarph [13/May/2021:12:39:39 -0700] "PUT /nextcloud/index.php/apps/news/api/v1-2/items/read/multiple HTTP/1.1" 200 24
		 22457 Query	UPDATE `oc_news_items` SET `unread` = 0, `last_modified` = '1620934779730098' WHERE `id` = 19095

What does it look like now?

MariaDB [owncloud]> select id,title,unread from oc_news_items where id=19095;
+-------+-----------------------------------------------------------------------+--------+
| id    | title                                                                 | unread |
+-------+-----------------------------------------------------------------------+--------+
| 19095 | Patrick Marleau answers question as to whether he’s going to retire   |      0 |
+-------+-----------------------------------------------------------------------+--------+
1 row in set (0.000 sec)

Pull to refresh. The item is still there.

		 22460 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) AND (starred = '1')
		 22460 Query	SELECT `feeds`.*, COUNT(`items`.`id`) AS `unreadCount` FROM `oc_news_feeds` `feeds` LEFT JOIN `oc_news_items` `items` ON items.feed_id = feeds.id AND items.unread = 1 WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) GROUP BY `feeds`.`id`
		 22460 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE feeds.user_id = 'xarph' ORDER BY `items`.`last_modified` DESC, `items`.`id` DESC LIMIT 1
		 22462 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) ORDER BY `items`.`id` DESC
		 22463 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) AND (items.starred = '1') ORDER BY `items`.`id` DESC
		 
172.16.1.2 - xarph [13/May/2021:12:40:18 -0700] "GET /nextcloud/index.php/apps/news/api/v1-2/feeds HTTP/1.1" 200 2715
172.16.1.2 - xarph [13/May/2021:12:40:18 -0700] "GET /nextcloud/index.php/apps/news/api/v1-2/folders HTTP/1.1" 200 115
172.16.1.2 - xarph [13/May/2021:12:40:19 -0700] "GET /nextcloud/index.php/apps/news/api/v1-2/items?batchSize=-1&getRead=false&type=3 HTTP/1.1" 200 4923728
172.16.1.2 - xarph [13/May/2021:12:40:20 -0700] "GET /nextcloud/index.php/apps/news/api/v1-2/items?batchSize=-1&getRead=true&type=2 HTTP/1.1" 200 32

Now, the web UI:
Clicking on a feed:

		 22517 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE feeds.user_id = 'xarph' ORDER BY `items`.`last_modified` DESC, `items`.`id` DESC LIMIT 1
		 22517 Query	SELECT `feeds`.*, COUNT(`items`.`id`) AS `unreadCount` FROM `oc_news_feeds` `feeds` LEFT JOIN `oc_news_items` `items` ON items.feed_id = feeds.id AND items.unread = 1 WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) GROUP BY `feeds`.`id`
		 22517 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) AND (starred = '1')
		 22517 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE (feeds.deleted_at = 0) AND (feeds.user_id = 'xarph') AND (items.feed_id = '233') AND (items.unread = '1') ORDER BY `items`.`id` ASC LIMIT 40
		 
2601:647:4b01:55e0:149:badb:40e5:73aa - - [13/May/2021:12:41:33 -0700] "GET /nextcloud/apps/news/items?id=28&limit=40&oldestFirst=true&search=&showAll=false&type=1 HTTP/1.1" 200 26319
2601:647:4b01:55e0:149:badb:40e5:73aa - - [13/May/2021:12:41:33 -0700] "GET /nextcloud/apps/news/img/loading.gif HTTP/1.1" 200 3208
2601:647:4b01:55e0:149:badb:40e5:73aa - - [13/May/2021:12:41:33 -0700] "GET /nextcloud/apps/news/%7B%7B%20::Content.getFeed(item.feedId).faviconLink%20%7D%7D HTTP/1.1" 302 -
2601:647:4b01:55e0:149:badb:40e5:73aa - - [13/May/2021:12:41:51 -0700] "GET /nextcloud/apps/news/items?id=233&limit=40&oldestFirst=true&search=&showAll=false&type=0 HTTP/1.1" 200 31247
2601:647:4b01:55e0:149:badb:40e5:73aa - - [13/May/2021:12:41:51 -0700] "GET /nextcloud/apps/news/%7B%7B%20::Content.getFeed(item.feedId).faviconLink%20%7D%7D HTTP/1.1" 302 -

Test item:

MariaDB [owncloud]> select id,title,unread from oc_news_items where id=19025;
+-------+--------------------------------------------------------------------------------+--------+
| id    | title                                                                          | unread |
+-------+--------------------------------------------------------------------------------+--------+
| 19025 | Patrick Marleau presented with amazing gift before his possible final NHL game |      1 |
+-------+--------------------------------------------------------------------------------+--------+
1 row in set (0.000 sec)

Clicking on it:

2601:647:4b01:55e0:149:badb:40e5:73aa - - [13/May/2021:12:45:36 -0700] "POST /nextcloud/apps/news/items/19025/read HTTP/1.1" 200 22

UPDATE `oc_news_items` SET `unread` = 0, `last_modified` = '1620935136595145' WHERE `id` = 19025

MariaDB [owncloud]> select id,title,unread from oc_news_items where id=19025;
+-------+--------------------------------------------------------------------------------+--------+
| id    | title                                                                          | unread |
+-------+--------------------------------------------------------------------------------+--------+
| 19025 | Patrick Marleau presented with amazing gift before his possible final NHL game |      0 |
+-------+--------------------------------------------------------------------------------+--------+
1 row in set (0.000 sec)

Reload feed:

		 22552 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE feeds.user_id = 'xarph' ORDER BY `items`.`last_modified` DESC, `items`.`id` DESC LIMIT 1
		 22552 Query	SELECT `feeds`.*, COUNT(`items`.`id`) AS `unreadCount` FROM `oc_news_feeds` `feeds` LEFT JOIN `oc_news_items` `items` ON items.feed_id = feeds.id AND items.unread = 1 WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) GROUP BY `feeds`.`id`
		 22552 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) AND (starred = '1')
		 22552 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE (feeds.deleted_at = 0) AND (feeds.user_id = 'xarph') AND (items.feed_id = '233') AND (items.unread = '1') ORDER BY `items`.`id` ASC LIMIT 40
		 22556 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE feeds.user_id = 'xarph' ORDER BY `items`.`last_modified` DESC, `items`.`id` DESC LIMIT 1
		 22556 Query	SELECT `feeds`.*, COUNT(`items`.`id`) AS `unreadCount` FROM `oc_news_feeds` `feeds` LEFT JOIN `oc_news_items` `items` ON items.feed_id = feeds.id AND items.unread = 1 WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) GROUP BY `feeds`.`id`
		 22556 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) AND (starred = '1')
		 22556 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE (feeds.deleted_at = 0) AND (feeds.user_id = 'xarph') AND (items.feed_id = '233') AND (items.unread = '1') ORDER BY `items`.`id` ASC LIMIT 40
		 
2601:647:4b01:55e0:149:badb:40e5:73aa - - [13/May/2021:12:46:08 -0700] "GET /nextcloud/apps/news/items?id=233&limit=40&oldestFirst=true&search=&showAll=false&type=0 HTTP/1.1" 200 29763

Now let's mark all as read:

2601:647:4b01:55e0:149:badb:40e5:73aa - - [13/May/2021:12:46:41 -0700] "POST /nextcloud/apps/news/feeds/233/read HTTP/1.1" 200 58
		 22565 Query	UPDATE `oc_news_items` SET `unread` = 0 WHERE (id IN (11652, 11653, 11654, 11655, 11656, 11657, 11658, 11659, 11660, 11661, 11714, 11732, 12548, 12556, 12560, 12604, 12688, 12716, 12744, 12888, 12909, 12928, 12967, 12990, 13003, 13072, 13075, 13080, 13092, 13093, 13109, 13135, 13146, 13148, 13222, 13247, 13275, 13296, 13353, 13439, 13445, 13547, 13548, 13613, 13661, 13673, 13681, 13722, 13734, 13745, 13808, 13905, 13936, 13966, 14038, 14132, 14164, 14167, 14218, 14339, 14375, 14417, 14431, 14434, 14549, 14579, 14603, 14640, 14744, 14756, 14782, 14785, 14867, 14926, 14957, 14976, 14986, 15005, 15115, 15125, 15158, 15203, 15356, 15383, 15409, 15457, 15462, 15548, 15572, 15586, 15603, 15629, 15639, 15653, 15729, 15807, 15873, 15918, 15953, 15983, 16120, 16168, 16201, 16215, 16231, 16270, 16289, 16337, 16389, 16390, 16404, 16440, 16512, 16560, 16607, 16642, 16704, 16800, 16819, 16878, 16903, 16951, 16961, 16999, 17032, 17040, 17041, 17045, 17056, 17179, 17236, 17272, 17286, 17432, 17433, 17491, 17503, 17507, 17567, 17576, 17644, 17689, 17690, 17718, 17760, 17846, 17882, 17942, 18050, 18083, 18088, 18125, 18142, 18177, 18258, 18287, 18341, 18460, 18477, 18495, 18533, 18551, 18612, 18639, 18642, 18675, 18700, 18736, 18811, 18856, 18909, 18944, 18990, 19014, 19018, 19025)) AND (unread != 0)

This worked in the web UI.

Now look at the fiery feeds app:

		 22586 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) ORDER BY `items`.`id` DESC
		 22587 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) AND (items.starred = '1') ORDER BY `items`.`id` DESC
		 22588 Query	SELECT `feeds`.*, COUNT(`items`.`id`) AS `unreadCount` FROM `oc_news_feeds` `feeds` LEFT JOIN `oc_news_items` `items` ON items.feed_id = feeds.id AND items.unread = 1 WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) GROUP BY `feeds`.`id`
		 22588 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE (feeds.user_id = 'xarph') AND (feeds.deleted_at = 0) AND (starred = '1')
		 22588 Query	SELECT `items`.* FROM `oc_news_items` `items` INNER JOIN `oc_news_feeds` `feeds` ON items.feed_id = feeds.id WHERE feeds.user_id = 'xarph' ORDER BY `items`.`last_modified` DESC, `items`.`id` DESC LIMIT 1
		 
172.16.1.2 - xarph [13/May/2021:12:47:33 -0700] "GET /nextcloud/index.php/apps/news/api/v1-2/feeds HTTP/1.1" 200 2710
172.16.1.2 - xarph [13/May/2021:12:47:34 -0700] "GET /nextcloud/index.php/apps/news/api/v1-2/folders HTTP/1.1" 200 115
172.16.1.2 - xarph [13/May/2021:12:47:34 -0700] "GET /nextcloud/index.php/apps/news/api/v1-2/items?batchSize=-1&getRead=false&type=3 HTTP/1.1" 200 4923784
172.16.1.2 - xarph [13/May/2021:12:47:35 -0700] "GET /nextcloud/index.php/apps/news/api/v1-2/items?batchSize=-1&getRead=true&type=2 HTTP/1.1" 200 32

All items are returned.

SMillerDev · 2021-05-14T19:58:14Z

172.16.1.2 - xarph [13/May/2021:12:47:34 -0700] "GET /nextcloud/index.php/apps/news/api/v1-2/items?batchSize=-1&getRead=false&type=3 HTTP/1.1" 200 4923784

I think this is where it goes wrong, since there is a dedicated unread type I don't take getRead into account for /items calls. But I could just make an internal redirect for this.

tweitzel · 2021-05-15T18:47:30Z

Fiery feeds just released an update that works around this issue. So at least the gordian knot is cut.

HorlogeSkynet · 2021-05-22T16:12:45Z

Fix confirmed in v15.4.4 :

curl \
	-H "Accept: application/json" \
	-H "Authorization: Basic XXXXXXXXXXXXXX" \
	"https://nextcloud.domain.name/apps/news/api/v1-2/items?getRead=false" \
		| json_pp
{
   "items" : []
}

Thank you, all 🙏

HorlogeSkynet added the bug label May 2, 2021

SMillerDev mentioned this issue May 14, 2021

controller: getRead + all_items is now unread type #1356

Merged

HorlogeSkynet closed this as completed May 22, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[API] `getRead` attribute behavior change in 15.4.0 #1328

[API] `getRead` attribute behavior change in 15.4.0 #1328

HorlogeSkynet commented May 2, 2021

SMillerDev commented May 3, 2021

HorlogeSkynet commented May 3, 2021

HorlogeSkynet commented May 3, 2021 •

edited

Loading

captainfodder commented May 4, 2021 •

edited

Loading

SMillerDev commented May 4, 2021

HorlogeSkynet commented May 4, 2021

HorlogeSkynet commented May 7, 2021

SMillerDev commented May 10, 2021

HorlogeSkynet commented May 10, 2021

tweitzel commented May 10, 2021

tweitzel commented May 13, 2021

SMillerDev commented May 14, 2021

tweitzel commented May 15, 2021

HorlogeSkynet commented May 22, 2021

[API] getRead attribute behavior change in 15.4.0 #1328

[API] getRead attribute behavior change in 15.4.0 #1328

Comments

HorlogeSkynet commented May 2, 2021

IMPORTANT

Explain the Problem

Steps to Reproduce

System Information

SMillerDev commented May 3, 2021

HorlogeSkynet commented May 3, 2021

HorlogeSkynet commented May 3, 2021 • edited Loading

captainfodder commented May 4, 2021 • edited Loading

SMillerDev commented May 4, 2021

HorlogeSkynet commented May 4, 2021

HorlogeSkynet commented May 7, 2021

SMillerDev commented May 10, 2021

HorlogeSkynet commented May 10, 2021

tweitzel commented May 10, 2021

tweitzel commented May 13, 2021

SMillerDev commented May 14, 2021

tweitzel commented May 15, 2021

HorlogeSkynet commented May 22, 2021

[API] `getRead` attribute behavior change in 15.4.0 #1328

[API] `getRead` attribute behavior change in 15.4.0 #1328

HorlogeSkynet commented May 3, 2021 •

edited

Loading

captainfodder commented May 4, 2021 •

edited

Loading