zdb segfaults when ran with a pool name argument (as a non-root user, without appropriate permissions)

[vedranmiletic]

System information

Type	Version/Name
Distribution Name	Fedora
Distribution Version	41
Kernel Version	6.13.8-200.fc41.x86_64
Architecture	x86-64
OpenZFS Version	zfs-2.3.1-1 / zfs-kmod-2.3.1-1

Describe the problem you're observing

If I run zdb on a pool in my system as a regular user, without sudo, it crashes with a segfault.

Describe how to reproduce the problem

Run zdb as a regular user with the pool name as the only argument.

Include any warning/errors/backtraces from the system logs

$ zdb pool05
zdb: can't open 'pool05': Permission denied

ZFS_DBGMSG(zdb) START:
metaslab.c:1689:spa_set_allocator(): spa allocator: dynamic
spa.c:5840:spa_open_common(): spa_open_common: opening pool05
spa_misc.c:429:spa_load_note(): spa_load(pool05, config trusted): LOADING
spa_misc.c:429:spa_load_note(): spa_load(pool05, config untrusted): vdev tree has 1 missing top-level vdevs.
spa_misc.c:429:spa_load_note(): spa_load(pool05, config untrusted): current settings allow for maximum 2 missing top-level vdevs at this stage.
spa_misc.c:415:spa_load_failed(): spa_load(pool05, config untrusted): FAILED: unable to open vdev tree [error=13]
vdev.c:235:vdev_dbgmsg_print_tree():   vdev 0: root, guid: 18294341717373886597, path: N/A, can't open
vdev.c:235:vdev_dbgmsg_print_tree():     vdev 0: mirror, guid: 12788986560798994756, path: N/A, can't open
vdev.c:235:vdev_dbgmsg_print_tree():       vdev 0: disk, guid: 3378021235836726124, path: /dev/disk/by-id/ata-ST8000VN004-2M2101_WKD022L1-part1, can't open
vdev.c:235:vdev_dbgmsg_print_tree():       vdev 1: disk, guid: 15224328164650351331, path: /dev/disk/by-id/ata-ST8000DM004-2CX188_WCT07V9L-part1, can't open
spa_misc.c:429:spa_load_note(): spa_load(pool05, config untrusted): UNLOADING
ZFS_DBGMSG(zdb) END
ASSERT at module/zfs/spa_misc.c:964:spa_close()
zfs_refcount_count(&spa->spa_refcount) > spa->spa_minref || MUTEX_HELD(&spa_namespace_lock) || spa->spa_load_thread == curthread || spa->spa_export_thread == curthread
  PID: 12798     COMM: zdb
  TID: 12798     NAME: zdb
Registers:
  RAX: 0x00007fffef55d7a0  RDX: 0x00007f9ebbc7d45b  RCX: 0x0000000000000000
  RBX: 0x0000000000000002  RSI: 0x00007fffef55d5b0  RDI: 0x00007fffef55cf60
  RBP: 0x00007fffef55d760  RSP: 0x00007fffef55ceb0   R8: 0x0000000000000001
   R9: 0x00000000ffffffff  R10: 0x0000000000000080  R11: 0x00007fffef55cfd0
  R12: 0x0000000000000000  R13: 0x00007fffef55d330  R14: 0x00007fffef55cf60
  R15: 0x00007fffef55d7a0  RIP: 0x00007f9ebbc7cee7
Call trace:
  [0x00007f9ebbc7cee7] libspl_backtrace+0x47 (in /usr/lib64/libzpool.so.6.0.0 +0x27cee7)
  [0x00007f9ebbc7d48b] libspl_assertf+0x15b (in /usr/lib64/libzpool.so.6.0.0 +0x27d48b)
  [0x00007f9ebba20131] ??? (in /usr/lib64/libzpool.so.6.0.0 +0x20131)
  [0x00007f9ebbb31140] spa_close+0x80 (in /usr/lib64/libzpool.so.6.0.0 +0x131140)
  [0x000055cc49ab4e06] ??? (in /usr/sbin/zdb +0xee06)
  [0x000055cc49ab54ea] ??? (in /usr/sbin/zdb +0xf4ea)
  [0x000055cc49aab1ba] ??? (in /usr/sbin/zdb +0x51ba)
  [0x00007f9ebb210248] __libc_start_call_main+0x78 (in /usr/lib64/libc.so.6 +0x3248)
  [0x00007f9ebb21030b] __libc_start_main+0x8b (in /usr/lib64/libc.so.6 +0x330b)
  [0x000055cc49aab625] ??? (in /usr/sbin/zdb +0x5625)
Registers:
  RAX: 0x0000000000000000  RDX: 0x00007fffef55d080  RCX: 0x00007f9ebb27fb54
  RBX: 0x0000000000000002  RSI: 0x00007fffef55d1b0  RDI: 0x00007fffef55c7b0
  RBP: 0x00007fffef55cfb0  RSP: 0x00007fffef55c700   R8: 0xfffffffffffffff8
   R9: 0x00007fffef55cac0  R10: 0x0000000000000000  R11: 0x0000000000000000
  R12: 0x0000000000000000  R13: 0x00007fffef55cb80  R14: 0x00007fffef55c7b0
  R15: 0x00007fffef55d7a0  RIP: 0x00007f9ebbc7cee7
Call trace:
  [0x00007f9ebbc7cee7] libspl_backtrace+0x47 (in /usr/lib64/libzpool.so.6.0.0 +0x27cee7)
  [0x000055cc49aaff02] ??? (in /usr/sbin/zdb +0x9f02)
  [0x00007f9ebb227050] ??? (in /usr/lib64/libc.so.6 +0x1a050)
  [0x00007f9ebb27fb54] __pthread_kill_implementation+0x114 (in /usr/lib64/libc.so.6 +0x72b54)
  [0x00007f9ebb226f9e] gsignal+0x1e (in /usr/lib64/libc.so.6 +0x19f9e)
  [0x00007f9ebb20e942] abort+0xdf (in /usr/lib64/libc.so.6 +0x1942)
  [0x00007f9ebba1655b] ??? (in /usr/lib64/libzpool.so.6.0.0 +0x1655b)
  [0x00007f9ebba20131] ??? (in /usr/lib64/libzpool.so.6.0.0 +0x20131)
  [0x00007f9ebbb31140] spa_close+0x80 (in /usr/lib64/libzpool.so.6.0.0 +0x131140)
  [0x000055cc49ab4e06] ??? (in /usr/sbin/zdb +0xee06)
  [0x000055cc49ab54ea] ??? (in /usr/sbin/zdb +0xf4ea)
  [0x000055cc49aab1ba] ??? (in /usr/sbin/zdb +0x51ba)
  [0x00007f9ebb210248] __libc_start_call_main+0x78 (in /usr/lib64/libc.so.6 +0x3248)
  [0x00007f9ebb21030b] __libc_start_main+0x8b (in /usr/lib64/libc.so.6 +0x330b)
  [0x000055cc49aab625] ??? (in /usr/sbin/zdb +0x5625)

ZFS_DBGMSG(zdb) START:
metaslab.c:1689:spa_set_allocator(): spa allocator: dynamic
spa.c:5840:spa_open_common(): spa_open_common: opening pool05
spa_misc.c:429:spa_load_note(): spa_load(pool05, config trusted): LOADING
spa_misc.c:429:spa_load_note(): spa_load(pool05, config untrusted): vdev tree has 1 missing top-level vdevs.
spa_misc.c:429:spa_load_note(): spa_load(pool05, config untrusted): current settings allow for maximum 2 missing top-level vdevs at this stage.
spa_misc.c:415:spa_load_failed(): spa_load(pool05, config untrusted): FAILED: unable to open vdev tree [error=13]
vdev.c:235:vdev_dbgmsg_print_tree():   vdev 0: root, guid: 18294341717373886597, path: N/A, can't open
vdev.c:235:vdev_dbgmsg_print_tree():     vdev 0: mirror, guid: 12788986560798994756, path: N/A, can't open
vdev.c:235:vdev_dbgmsg_print_tree():       vdev 0: disk, guid: 3378021235836726124, path: /dev/disk/by-id/ata-ST8000VN004-2M2101_WKD022L1-part1, can't open
vdev.c:235:vdev_dbgmsg_print_tree():       vdev 1: disk, guid: 15224328164650351331, path: /dev/disk/by-id/ata-ST8000DM004-2CX188_WCT07V9L-part1, can't open
spa_misc.c:429:spa_load_note(): spa_load(pool05, config untrusted): UNLOADING
ZFS_DBGMSG(zdb) END
Aborted (core dumped)

[snajpa]

(line numbers valid for 2.3.x, done on top of current WIP 2.3.2 PR)

(gdb) bt
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
#1  0x00007fea54e9debf in __pthread_kill_internal (threadid=<optimized out>, signo=6) at ./nptl/pthread_kill.c:78
#2  0x00007fea54e49c82 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007fea54e324f0 in __GI_abort () at ./stdlib/abort.c:79
#4  0x00007fea55659f30 in libspl_assertf (file=file@entry=0x7fea55914ff4 "module/zfs/spa_misc.c", func=func@entry=0x7fea559532f8 <__FUNCTION__.30> "spa_close", line=line@entry=965, format=format@entry=0x7fea5591511f "%s") at lib/libspl/assert.c:115
#5  0x00007fea5577eeed in libspl_assert (line=965, func=0x7fea559532f8 <__FUNCTION__.30> "spa_close", file=0x7fea55914ff4 "module/zfs/spa_misc.c", 
    buf=0x7fea55935da0 "zfs_refcount_count(&spa->spa_refcount) > spa->spa_minref || MUTEX_HELD(&spa_namespace_lock) || spa->spa_load_thread == curthread || spa->spa_export_thread == curthread") at ./lib/libspl/include/assert.h:60
#6  spa_close (spa=0x55604f7ba8d0, tag=tag@entry=0x556022bf8398 <__func__.110>) at module/zfs/spa_misc.c:965
#7  0x0000556022bdccb7 in zdb_exit (reason=reason@entry=1) at cmd/zdb/zdb.c:3476
#8  0x0000556022bdd3b1 in fatal (fmt=fmt@entry=0x556022bf220a "can't open '%s': %s") at cmd/zdb/zdb.c:883
#9  0x0000556022bd78ac in main (argc=<optimized out>, argv=0x7ffc2bfb6850) at cmd/zdb/zdb.c:9814

my first impression is that spa_close shouldn't expect an increased reference when spa_open was used instead of spa_open_ref

nah that won't be it, it's probably just that if the pool couldn't be opened it obviously wouldn't increase ref even if done as a separate step after spa_open... so probably it's just about the assert which was added to spa_close in #16093

[snajpa]

ping @grwilson please if you could take a look :)

[vedranmiletic]

Thanks for the reply. FWIW, I can still reproduce this in OpenZFS 2.3.2 on Linux 6.14, so please let me know if you need any more details.