Retain message position when refreshing search buffer, and regularly poll for new messages (#560)

[witten]

This change prevents the message cursor from jumping to the top when refreshing the search buffer (unless the message disappears during the refresh), and adds periodic refreshing of search buffers so that new threads show up.

Not sure if you want to take this approach, but I figure this pull request will at least get the discussion started.

[witten]

What platforms does alot officially support? I can add a hard inotify dependency, but I don't wan to do that if, say, FreeBSD is a target platform.

[pazz]

I agree with the first patch.
Regarding the second, I think i'd be nicer to have some more "generic" solution for regularly scheduled command lines / python code instead of a hardcoded refresh for search buffers. The next thing that'll happen is that s/o wants automagic refreshes for ThreadBuffers as well.
Maybe we could keep something like a buffer-local crontab, a set of pairs (timeout, commandline)
per buffer/buffer type that is only active when we have this buffer open.
This way, we circumvent the problem of triggering events for buffers not even in focus.
Also, this would allow people to periodically trigger their mail-fetching-notmuch-new command..
It might be tricky to reflect this in the user settings..

[witten]

Do you think this sort of regularly scheduled command feature would even be necessary if laarmen's idea of inotify was implemented? With inotify, the various buffers could find out as soon as they need to be refreshed without relying on periodically scheduled commands.

[pazz]

hmm. inotify is linux specific isn't it? I know at least a handful of people happily using alot on OSX...

[laarmen]

Yes, that's why I added the "On Linux" bit. On BSDs I think the kqueue mechanism is somewhat similar, and a quick search for Mac comes up with https://pypi.python.org/pypi/MacFSEvents/0.2.1
I'll see if I can put together a small module to abstract all that into a single API.

[witten]

This Python module may be relevant: https://pypi.python.org/pypi/watchdog

[pazz]

lets focus on the first issue of this PR first. If someone comes up with a good solution for the second point he/she should open up a new PR or issue to discuss this there.

So, I just gave your first patch a try. it works beautifully for small result sets but there is an issue with large ones.
This is due to your use of consume_pipe in https://github.com/witten/alot/commit/747139cb77cd37ee7ad9e7a157da43f2fa16f342#L0R305 :
this blocks the UI and potentially takes ages.
a quick and dirty fix would be to add a parameter refocus or some threshold on the size of the result set (or both)
that is a precondition on your refocusing feature. One could also refocus in a callback that is only triggered after the pipe is fully read.

[witten]

Note that anything that calls consume_pipe() has the same problem. For instance, enter ":move last" in a search buffer with a large result set, and it'll similarly take forever to respond. I'll see what I can do.

[witten]

Not entirely happy with this, but I've updated the pull request to time out if the refocus is taking too long and just revert to the original behavior of focusing the first thread. This isn't great from a UX perspective because the user won't know what to expect when hitting refresh ("Will the focus jump to the top? Will it stay on the same thread?"). But in practice it's good enough for the common case of refreshing one's inbox. And it's preferable to trying to background the refocusing, and then suddenly jumping the cursor after 10 seconds of asynchronous processing.

I think the underlying issue is that notmuch API doesn't currently support slicing. If you're on the last page of a 20,000-thread result set and you hit refresh, you shouldn't have to consume 20,000 thread objects just to get to the ones you care about. And stuffing those thread IDs through an IPC pipe and unconditionally creating widgets from each one isn't helping either. But the real problem is that it's a "sequential scan", in relational database parlance. This same complaint applies to the ":move last" case as well.

[pazz]

I'm retiring this PR for the above discussed shortcomings of the lib. Feel free to re-open if you can think of a possible workaround that is stable enough for the main branch.