🎁 octavia-cli: add telemetry #11896
🎁 octavia-cli: add telemetry #11896
Conversation
alafanechere
force-pushed
the
augustin/octavia-cli/track
branch
from
c629d04 to
0e7a5ae
Compare
alafanechere
requested review from
cgardens,
lmossman,
marcosmarxm and
ChristopheDuong
| user_id = ctx.obj.get("WORKSPACE_ID") | ||
| anonymous_id = None if user_id else str(uuid.uuid1()) | ||
|
|
||
| segment_context = {"app": {"name": "octavia-cli", "version": ctx.obj.get("OCTAVIA_VERSION")}} |
There was a problem hiding this comment.
tracking the AIRBYTE_VERSION of the api that is being interacted with could also help, but we'd need an API to know what version the server is running?
There was a problem hiding this comment.
It looks like there's no endpoint in the API that gives the airbyte version information 🤦
| * Success or failure of the command run and the error type. | ||
| * The workspace id. It is unique to each Airbyte instance, but we can't match it to a username or email address. | ||
|
|
||
| You can disable telemetry by setting the `OCTAVIA_ENABLE_TELEMETRY` environment to `false` or using the `--disable-telemetry` flag. |
There was a problem hiding this comment.
Maybe the "default" telemetry behavior can also follow the boolean for tracking or not on the airbyte instance too?
There was a problem hiding this comment.
The boolean for tracking or not is the TRACKING_STRATEGY env var. Unfortunately, the API only returns the anonymousDataCollection field which does not mean the user has turned off analytics on their instance, it just mean they want to avoid mapping their workplace id to their email. Our default is the "anonymous data collection" as we don't handle emails.
| public void testUpdateConnectionWhenWorkflowUnreachable() throws Exception { | ||
| // This test only covers the specific behavior of updating a connection that does not have an underlying temporal workflow. | ||
| // This case only occurs with the new scheduler, so the entire test is inside the feature flag conditional. | ||
| // Also, this test doesn't verify correctness of the schedule update applied, as adding the ability to query a workflow for its current | ||
| // schedule is out of scope for the issue (https://github.com/airbytehq/airbyte/issues/11215). This test just ensures that the underlying workflow | ||
| // This test only covers the specific behavior of updating a connection that does not have an | ||
| // underlying temporal workflow. | ||
| // This case only occurs with the new scheduler, so the entire test is inside the feature flag | ||
| // conditional. | ||
| // Also, this test doesn't verify correctness of the schedule update applied, as adding the ability | ||
| // to query a workflow for its current | ||
| // schedule is out of scope for the issue (https://github.com/airbytehq/airbyte/issues/11215). This | ||
| // test just ensures that the underlying workflow |
There was a problem hiding this comment.
This was not formatted on master and broke the build.
| This CLI has some telemetry tooling to send Airbyte some data about the usage of this tool. | ||
| We use this data to measure the tool's adoption and detect common errors users encounter to improve it. | ||
| The telemetry sends data about: | ||
| * Which command was run (not the arguments or options used). |
There was a problem hiding this comment.
(not the arguments or options used)
Just curious, why are we not tracking the arguments/options used in a request? Is this because we expect the arguments to be different for each user, so it will not be helpful in the aggregate user tracking we want to get out of this?
There was a problem hiding this comment.
I wanted to add the least amount of user data to this telemetry. As arguments or options are free user inputs I did not want to risk exposing sensitive information in the telemetry. I'm also thinking that to grasp which resources are managed with Octavia, we can rely on the existing telemetry from the core platform. This is why I added a dedicated user agent for octavia. We can find Octavia related traffic if the user agent is available in the API telemetry. We can also join the workspace id from octavia telemetry with the workpace id from the API telemetry to get a bit more insights.
I wanted to focus Octavia telemetry on getting a sense of which commands are used, which commands cause errors, that's all.
What
We want to add telemetry to octavia-cli in order to track the adoption and the tool and detect common errors.
How
OctaviaCommandsubclass of click'sCommand. I patched the parent'smake_contextandinvokemethods to collect and send telemetry in case of success or error.Recommended reading order
Other changes are made to make the commands use the custom
OctaviaCommand.Feel free to read the unit tests too!
🚨 User Impact 🚨
This should be transparent for users