Skip to content

Commit 16e3dfb

Browse files
mingyexiaAlice Berard
authored and
Alice Berard
committed
[AIRFLOW-3125] Monitor Task Instances creation rates (apache#3966)
Montor Task Instances creation rates by Operator type. These stats can provide some visibility on how much workload Airflow is getting. They can be used for resource allocation in the long run (i.e. to determine when we should scale up workers) and debugging in scenarios like the creation rate of certain type of Task Instances spikes.
1 parent 19f59bf commit 16e3dfb

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

airflow/models.py

+3
Original file line numberDiff line numberDiff line change
@@ -5281,6 +5281,9 @@ def verify_integrity(self, session=None):
52815281
continue
52825282

52835283
if task.task_id not in task_ids:
5284+
Stats.incr(
5285+
"task_instance_created-{}".format(task.__class__.__name__),
5286+
1, 1)
52845287
ti = TaskInstance(task, self.execution_date)
52855288
session.add(ti)
52865289

0 commit comments

Comments
 (0)