[WIP] Records order preservation ActiveRecord POC

[tyler-boyd]

Fixes #463

As you know, currently the records method is not ordered by Elasticsearch's hit ordering. Taking heavy inspiration from order_as_specified gem, this preserves the order of the search results.

It should be noted that this is quite slow to do, and should only be done if you really need an ActiveRecord::Relation. Otherwise, it's possibly better to just use the existing in-memory representation.

Todo

• Ensure compatibility with all ActiveRecord DBs (MySQL, PostgreSQL, SQLite, etc)
• Ensure that the existing results_query preserves order for Mongoid/NoBrainer

Let me know if you have any thoughts.

also, cc @abevoelker @milushov

[jimmybaker]

I've also made a pull request to solve this problem using a different and more performant way. #672

Of course milage may vary but I was using a method similar to one you are proposing here and the query time to bring by 25 records was ~100ms which is slow as you mentioned in the description. With what I'm proposing in my PR I was able to get the same 25 records in 0.7ms.

Check it out and let me know if you have any thoughts on it.

[ankane]

Hey @Mavvie, thanks for the PR. Here's how Elasticsearch Rails handles it.

• ActiveRecord
• Mongoid

It should be a more performant approach.

[jimmybaker]

@ankane - Would the way ElasticSearch Rails handles it work for us when we paginate search results?

[jimmybaker]

This is an interesting problem. I'm working up a PR now and when writing the test I noticed that calling normal enumerable methods like map, each, or collect on .records doesn't call to_a. Should we also override these methods to ensure that things are done in the proper order?

[jimmybaker]

Since we're returning an active record relation with .records I still think the best way to handle this is to ensure the ordering is done on the db side based on the ordering of hits from es.

Anyone else following me here?

[tyler-boyd]

I think something similar is possible, but I'm not sure exactly how it would be implemented. I don't really know how pagination libraries work.

It could make sense to do the ordering in Ruby-land if we're only loading per_page records from the db. However, the db might still do it better in that case as well. Not sure really.

[khiav223577]

Maybe you could try the order method le0pard shared here: khiav223577/find_with_order#5
(no need to define a db function)

[tyler-boyd]

@ankane maybe we could add that gem as a dependency? Or should we copy what we need from it?

[ankane]

Hey, the plan is to remove the records method in Searchkick 3.0, so going to close this PR. Feel free to create another gem with this functionality that can tie into Searchkick.