Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Average query with subType field creates unnecessary typeIndexJoins under parallelNode #640

Closed
shahzadlone opened this issue Jul 15, 2022 · 1 comment · Fixed by #774
Closed
Assignees
Labels
area/query Related to the query component bug Something isn't working perf Performance issue or suggestion

Comments

@shahzadlone
Copy link
Member

Consider the following explain query:

query @explain {
  author {
    _avg(books: {field: pages})
  }
}

This results in a plan graph with three typeIndexJoin children under parallelNode:

"explain":
  "selectTopNode":
    "averageNode":
      "countNode":
        "sumNode":
          "selectNode":
            "parallelNode": []
              {
                "typeIndexJoin":
                  "selectTopNode":
                    "selectNode":
                      "scanNode":
              }
              {
                "typeIndexJoin":
                  "selectTopNode":
                    "selectNode":
                      "scanNode":
              }
              {
                "typeIndexJoin":
                  "selectTopNode":
                    "selectNode":
                      "scanNode":
              }

Hunches

Bug is probably in mapper that tries to find existing children (tip from Andy).

Full Explain graph of the above query in question.

{
  "explain": dataMap{
    "selectTopNode": dataMap{
      "averageNode": dataMap{
        "sumField":   "_sum",
        "countField": "_count",
        "countNode": dataMap{
          "sources": []dataMap{
            {
              "fieldName": "books",
              "filter": dataMap{
                "pages": dataMap{
                  "_ne": nil,
                },
              },
            },
          },
          "sumNode": dataMap{
            "sources": []dataMap{
              {
                "childFieldName": "pages",
                "fieldName":      "books",
                "filter": dataMap{
                  "pages": dataMap{
                    "_ne": nil,
                  },
                },
              },
            },
            "selectNode": dataMap{
              "filter": nil,
              "parallelNode": []dataMap{
                {
                  "typeIndexJoin": dataMap{
                    "joinType": "typeJoinMany",
                    "rootName": "author",
                    "root": dataMap{
                      "scanNode": dataMap{
                        "collectionID":   "3",
                        "collectionName": "author",
                        "filter":         nil,
                        "spans": []dataMap{
                          {
                            "start": "/3",
                            "end":   "/4",
                          },
                        },
                      },
                    },
                    "subTypeName": "books",
                    "subType": dataMap{
                      "selectTopNode": dataMap{
                        "selectNode": dataMap{
                          "filter": nil,
                          "scanNode": dataMap{
                            "collectionID":   "2",
                            "collectionName": "book",
                            "filter": dataMap{
                              "pages": dataMap{
                                "_ne": nil,
                              },
                            },
                            "spans": []dataMap{
                              {
                                "start": "/2",
                                "end":   "/3",
                              },
                            },
                          },
                        },
                      },
                    },
                  },
                },
                {
                  "typeIndexJoin": dataMap{
                    "joinType": "typeJoinMany",
                    "rootName": "author",
                    "root": dataMap{
                      "scanNode": dataMap{
                        "collectionID":   "3",
                        "collectionName": "author",
                        "filter":         nil,
                        "spans": []dataMap{
                          {
                            "start": "/3",
                            "end":   "/4",
                          },
                        },
                      },
                    },
                    "subTypeName": "books",
                    "subType": dataMap{
                      "selectTopNode": dataMap{
                        "selectNode": dataMap{
                          "filter": nil,
                          "scanNode": dataMap{
                            "collectionID":   "2",
                            "collectionName": "book",
                            "filter": dataMap{
                              "pages": dataMap{
                                "_ne": nil,
                              },
                            },
                            "spans": []dataMap{
                              {
                                "start": "/2",
                                "end":   "/3",
                              },
                            },
                          },
                        },
                      },
                    },
                  },
                },
                {
                  "typeIndexJoin": dataMap{
                    "joinType": "typeJoinMany",
                    "rootName": "author",
                    "root": dataMap{
                      "scanNode": dataMap{
                        "collectionID":   "3",
                        "collectionName": "author",
                        "filter":         nil,
                        "spans": []dataMap{
                          {
                            "start": "/3",
                            "end":   "/4",
                          },
                        },
                      },
                    },
                    "subTypeName": "books",
                    "subType": dataMap{
                      "selectTopNode": dataMap{
                        "selectNode": dataMap{
                          "filter": nil,
                          "scanNode": dataMap{
                            "collectionID":   "2",
                            "collectionName": "book",
                            "filter": dataMap{
                              "pages": dataMap{
                                "_ne": nil,
                              },
                            },
                            "spans": []dataMap{
                              {
                                "start": "/2",
                                "end":   "/3",
                              },
                            },
                          },
                        },
                      },
                    },
                  },
                },
              },
            },
          },
        },
      },
    },
  },
}

@shahzadlone
Copy link
Member Author

Found the bug, leaving notes here:

The host was not being found and kept getting added incorrectly, this is because reflect.DeepEqual Fails when you have a map where the key is pointer.

	a1 := "str"
	a2 := "str"
	a3 := map[*string]bool{&a1: true}
	a4 := map[*string]bool{&a2: true}
	fmt.Println(reflect.DeepEqual(a3, a4)) // False

In this case it was slightly more hidden because the key was an interface with an underlying pointer. Here is a more accurate example on what was happening (the last case represents what was happening):

type Key interface{ Get() any }
type GoodKey struct{ Name any }

func (g GoodKey) Get() any { return g.Name }

func main() {
	a := GoodKey{Name: "key"}            // {key}
	b := GoodKey{Name: "key"}            // {key}
	fmt.Println(reflect.DeepEqual(a, b)) // true

	ma := map[Key]any{a: "non-pointer"}    // map[{key}:non-pointer]
	mb := map[Key]any{b: "non-pointer"}    // map[{key}:non-pointer]
	fmt.Println(reflect.DeepEqual(ma, mb)) // true

	pa := &a                               // &{key}
	pb := &b                               // &{key}
	fmt.Println(reflect.DeepEqual(pa, pb)) // true

	mpa := map[Key]any{pa: "pointer"}        // map[0xc000014250:pointer]
	mpb := map[Key]any{pb: "pointer"}        // map[0xc000014260:pointer]
	fmt.Println(reflect.DeepEqual(mpa, mpb)) // false
}

Can run the above here: https://go.dev/play/p/bBvUubM8u_N

shahzadlone added a commit that referenced this issue Sep 15, 2022
- Resolves #640

- Description:
* Add limit check to `tryGetTarget` function.
* Fix the equality check with the filter to find the host.
* Update the previously documented tests that had incorrect behavior.
shahzadlone added a commit to shahzadlone/defradb that referenced this issue Feb 23, 2024
…etwork#774)

- Resolves sourcenetwork#640

- Description:
* Add limit check to `tryGetTarget` function.
* Fix the equality check with the filter to find the host.
* Update the previously documented tests that had incorrect behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/query Related to the query component bug Something isn't working perf Performance issue or suggestion
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants