Skip to content

Commit

Permalink
[OSX] use host_statistics64 to get memory metrics (#2502)
Browse files Browse the repository at this point in the history
We have lots of sporadic failures on OSX related to free virtual_memory(), whose value does not exactly match vm_stat CLI utility. With this PR we use the exact same approach of vm_stat CLI tool, whose source code is here:
https://github.com/apple-open-source/macos/blob/master/system_cmds/vm_stat/vm_stat.c
Hopefully this will reduce such sporadic failures.
  • Loading branch information
giampaolo authored Jan 31, 2025
1 parent 08d7d43 commit 9c114a5
Show file tree
Hide file tree
Showing 4 changed files with 13 additions and 7 deletions.
3 changes: 3 additions & 0 deletions HISTORY.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ XXXX-XX-XX

- 2496_, [Linux]: Avoid segfault (a cPython bug) on ``Process.memory_maps()``
for processes that use hundreds of GBs of memory.
- 2502_, [macOS]: `virtual_memory()`_ now relies on ``host_statistics64``
instead of ``host_statistics``. This is the same approach used by ``vm_stat``
CLI tool, and should grant more accurate results.

**Compatibility notes**

Expand Down
3 changes: 1 addition & 2 deletions psutil/_psosx.py
Original file line number Diff line number Diff line change
Expand Up @@ -112,8 +112,7 @@ def virtual_memory():
"""System virtual memory as a namedtuple."""
total, active, inactive, wired, free, speculative = cext.virtual_mem()
# This is how Zabbix calculate avail and used mem:
# https://github.com/zabbix/zabbix/blob/trunk/src/libs/zbxsysinfo/
# osx/memory.c
# https://github.com/zabbix/zabbix/blob/master/src/libs/zbxsysinfo/osx/memory.c
# Also see: https://github.com/giampaolo/psutil/issues/1277
avail = inactive + free
used = active + wired
Expand Down
1 change: 1 addition & 0 deletions psutil/_psutil_osx.c
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
*/

#include <Python.h>
#include <sys/time.h> // needed for old macOS versions
#include <sys/proc.h>
#include <netinet/tcp_fsm.h>

Expand Down
13 changes: 8 additions & 5 deletions psutil/arch/osx/mem.c
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@
// from psutil/_psutil_osx.c in 2023. This is the GIT blame before the move:
// https://github.com/giampaolo/psutil/blame/efd7ed3/psutil/_psutil_osx.c

// See:
// https://github.com/apple-open-source/macos/blob/master/system_cmds/vm_stat/vm_stat.c

#include <Python.h>
#include <mach/host_info.h>
#include <sys/sysctl.h>
Expand All @@ -17,12 +20,12 @@


static int
psutil_sys_vminfo(vm_statistics_data_t *vmstat) {
psutil_sys_vminfo(vm_statistics64_t vmstat) {
kern_return_t ret;
mach_msg_type_number_t count = sizeof(*vmstat) / sizeof(integer_t);
unsigned int count = HOST_VM_INFO64_COUNT;
mach_port_t mport = mach_host_self();

ret = host_statistics(mport, HOST_VM_INFO, (host_info_t)vmstat, &count);
ret = host_statistics64(mport, HOST_VM_INFO64, (host_info64_t)vmstat, &count);
if (ret != KERN_SUCCESS) {
PyErr_Format(
PyExc_RuntimeError,
Expand All @@ -46,7 +49,7 @@ psutil_virtual_mem(PyObject *self, PyObject *args) {
int mib[2];
uint64_t total;
size_t len = sizeof(total);
vm_statistics_data_t vm;
vm_statistics64_data_t vm;
long pagesize = psutil_getpagesize();
// physical mem
mib[0] = CTL_HW;
Expand Down Expand Up @@ -86,7 +89,7 @@ psutil_swap_mem(PyObject *self, PyObject *args) {
int mib[2];
size_t size;
struct xsw_usage totals;
vm_statistics_data_t vmstat;
vm_statistics64_data_t vmstat;
long pagesize = psutil_getpagesize();

mib[0] = CTL_VM;
Expand Down

0 comments on commit 9c114a5

Please sign in to comment.