Add a .search(automaton: fst::Automaton) to the TermDictionary

[fulmicoton]

This is required for #219 (sublime text like search) and #272 (FuzzyTermQuery).

Note that while similar, they are quite different in that

• in #219, the user probably want the list of terms that matched, possibly sorted by a function of the doc frequencies and some function of the query and the term`. (See the ticket comments for some links)
• in #272, this is effectively a query. The user wants the list of ranked documents.

The method already exists is already implemented in the fst crate.
Unfortunately it does not give access to the automaton state in the resulting Streamer.

I opened a ticket in the fst crate to discuss whether this feature should be added upstream.
Otherwise, the raw module of the fst crate exposes enough public API to reimplement the intersection in tantivy if required.

[drusellers]

Just to be clear, what struct should have this search method added? I'm guessing the Searcher.

Is the automaton in this case an implementation of https://github.com/BurntSushi/fst/blob/master/src/automaton/mod.rs#L24 ?

[fulmicoton]

I edited the title to make it clearer. The automaton I am talking about is indeed fst::Automaton.

The idea would be to have a method like this in the TermDictonary.

fn search<A: Automaton>(&'a self, automaton: A) ->TermStreamerBuilder<'a, A>

The TermStreamerBuilder would take an extra generics argument, and rely on the StreamBuilder<'a, A> interface.

If you feel uncomfortable, you can just start to write a simple unit test of what you feel should be expected

[drusellers]

Ah, ok. Thank you. :) let me pivot and start to look at it from the direction. this makes more sense to me now. :)