Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Owned value relying on Vec instead of BTreeMap #2364

Merged
merged 4 commits into from
Apr 22, 2024
Merged

Conversation

fulmicoton
Copy link
Collaborator

No description provided.

@fulmicoton fulmicoton requested a review from PSeitz April 19, 2024 07:22
@@ -363,7 +364,8 @@ impl From<PreTokenizedString> for OwnedValue {

impl From<BTreeMap<String, OwnedValue>> for OwnedValue {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this change to From<Vec<(String, OwnedValue)>> as well or do we actually need the conversion (instead of just wrapping as before)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is used due to the serde_json conversion.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And you do not want to apply

diff --git a/src/schema/document/default_document.rs b/src/schema/document/default_document.rs
index fcf374dfe..077554f13 100644
--- a/src/schema/document/default_document.rs
+++ b/src/schema/document/default_document.rs
@@ -1,4 +1,4 @@
-use std::collections::{BTreeMap, HashMap, HashSet};
+use std::collections::{HashMap, HashSet};
 use std::net::Ipv6Addr;
 
 use common::DateTime;
@@ -150,7 +150,7 @@ impl TantivyDocument {
     }
 
     /// Add a dynamic object field
-    pub fn add_object(&mut self, field: Field, object: BTreeMap<String, OwnedValue>) {
+    pub fn add_object(&mut self, field: Field, object: Vec<(String, OwnedValue)>) {
         self.add_field_value(field, object);
     }
 
diff --git a/src/schema/document/owned_value.rs b/src/schema/document/owned_value.rs
index 738194908..fd1b232a6 100644
--- a/src/schema/document/owned_value.rs
+++ b/src/schema/document/owned_value.rs
@@ -1,4 +1,3 @@
-use std::collections::BTreeMap;
 use std::fmt;
 use std::net::Ipv6Addr;
 
@@ -359,10 +358,9 @@ impl From<PreTokenizedString> for OwnedValue {
     }
 }
 
-impl From<BTreeMap<String, OwnedValue>> for OwnedValue {
-    fn from(object: BTreeMap<String, OwnedValue>) -> OwnedValue {
-        let key_values = object.into_iter().collect();
-        OwnedValue::Object(key_values)
+impl From<Vec<(String, OwnedValue)>> for OwnedValue {
+    fn from(object: Vec<(String, OwnedValue)>) -> OwnedValue {
+        OwnedValue::Object(object)
     }
 }

because it is technically breaking and you do need this ASAP as a fix for the performance bug it implies?

Personally, I think the effect so large and the add_object API so new that this might be worth a little bit of semver breakage. (Worst case would be yanking 0.20.0 after 0.20.1 is out.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this won't be included in a bugfix release

@fulmicoton
Copy link
Collaborator Author

fulmicoton commented Apr 19, 2024

Impact on resident memory on quickwit.
Before on the left, After on the right

image

@PSeitz PSeitz force-pushed the owned_value_on_vec branch from 796568d to edbb115 Compare April 22, 2024 02:52
@PSeitz PSeitz merged commit 8861366 into main Apr 22, 2024
4 checks passed
@PSeitz PSeitz deleted the owned_value_on_vec branch April 22, 2024 07:38
philippemnoel pushed a commit to paradedb/tantivy that referenced this pull request Aug 31, 2024
* Owned value relying on Vec instead of BTreeMap

* fmt

* fix build

* fix serialization

---------

Co-authored-by: Pascal Seitz <pascal.seitz@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants