Added fixtures for generating bindable TCP/UDP ports

[agronholm]

Changes

Adds fixtures for generating bindable TCP/UDP ports.
This is the same as pytest-asyncio's unused_*_port and unused_*_port_factory fixtures.

Checklist

If this is a user-facing code change, like a bugfix or a new feature, please ensure that
you've fulfilled the following conditions (where applicable):

• You've added tests (in tests/) added which would fail without your patch
• You've updated the documentation (in docs/, in case of behavior changes or new
  features)
• You've added a new changelog entry (in docs/versionhistory.rst).

If this is a trivial change, like a typo fix or a code reformatting, then you can ignore
these instructions.

Updating the changelog

If there are no entries after the last release, use **UNRELEASED** as the version.
If, say, your patch fixes issue #123, the entry should look like this:

- Fix big bad boo-boo in task groups
  (`#123 <https://github.com/agronholm/anyio/issues/123>`_; PR by @yourgithubaccount)

If there's no issue linked, just link to your pull request instead by updating the
changelog after you've created the PR.

[graingert]

I think the fixtures should have an anyio prefix

Also I wonder if they're that much more useful than ephemeral-port-reserve?

[gschaffner]

These fixtures look leaky currently: generated.remove is never called. Should they call generated.remove during teardown?

[agronholm]

I think the assumption is that the test suite won't need (or want to) reuse already allocated ports, limiting the number of stored ports to 64511 anyway. pytest-asyncio at least does not bother removing them, IIRC.

[gschaffner]

Since these can cause people to see flaky test failures, should they have a minor note to document that? Something like "Warning: The ephemeral port returned may be taken by someone else before you start using it. If you encounter/want to prevent flaky test failures due to this, you can catch EADDRINUSE errors that happen when you try to bind to the port and retry with a different ephemeral port."

I guess it is possible to know that you should retry in the specific case where bind gives EADDRINUSE and you already made sure it was free a moment ago. (python-trio/trio#283 (comment))

[agronholm]

I've added a warning now.

[gschaffner]

This can return a port that is already taken on IPv6, so it can fail when the code being tested tries to open an IPv6 socket on that port (either a dualstack IPv6 socket, one socket for each IP version, or an IPv6-only socket).

E.g. await anyio.create_tcp_listener(local_port=free_tcp_port) tries to open one socket for each IP version.

I think that (on platforms with dualstack support) it should do dualstack_sock.bind(("", 0)) to get a v4-free + v6-free port from the kernel.

(The annoying part is dealing with platforms without dualstack support, though.

On older platforms with IPv4 and IPv6 support but not dualstack support, there are non-ideal options like trying v4_sock.bind(("localhost", 0)); v6_sock.bind(("", v4_sock_port)) in a loop. The main problem is how to detect when to break the loop and raise EADDRINUSE (see bind(2)).

Looping like this would actually be quite similar to the while True: port = ...; if port not in generated: return port loop. That loop also has the same problem of not detecting when to break and raise EADDRINUSE; it can get stuck forever.

See also python-trio/trio#283.)

(Related, but somewhat tangential: I think it would be nice to provide this feature outside of the pytest plugin too. For instance, it would be nice if multilistener = await create_tcp_listener(local_port=0) used the same port for both of the listeners. This would be roughly something like

def get_free_ipv4_ipv6_port(kind, bind_ipv4_addr, bind_ipv6_addr):
    if platform_has_dualstack and bind_ipv4_addr == bind_ipv6_addr == "":
        # We can use the dualstack_sock.bind(("", 0)) trick
        ...
    else:
        ...

Currently, folks who need this have to implement it downstream.)

[agronholm]

This can return a port that is already taken on IPv6, so it can fail when the code being tested tries to open an IPv6 socket on that port (either a dualstack IPv6 socket, one socket for each IP version, or an IPv6-only socket).

E.g. await anyio.create_tcp_listener(local_port=free_tcp_port) tries to open one socket for each IP version.

I think that (on platforms with dualstack support) it should do dualstack_sock.bind(("", 0)) to get a v4-free + v6-free port from the kernel.

(The annoying part is dealing with platforms without dualstack support, though.

I just did the same thing pytest-asyncio was doing. I also thought about allowing a fixture request parameter for the address family.

On older platforms with IPv4 and IPv6 support but not dualstack support, there are non-ideal options like trying v4_sock.bind(("localhost", 0)); v6_sock.bind(("", v4_sock_port)) in a loop. The main problem is how to detect when to break the loop and raise EADDRINUSE (see bind(2)).

Looping like this would actually be quite similar to the while True: port = ...; if port not in generated: return port loop. That loop also has the same problem of not detecting when to break and raise EADDRINUSE; it can get stuck forever.

See also python-trio/trio#283.)

(Related, but somewhat tangential: I think it would be nice to provide this feature outside of the pytest plugin too. For instance, it would be nice if multilistener = await create_tcp_listener(local_port=0) used the same port for both of the listeners. This would be roughly something like

def get_free_ipv4_ipv6_port(kind, bind_ipv4_addr, bind_ipv6_addr):
    if platform_has_dualstack and bind_ipv4_addr == bind_ipv6_addr == "":
        # We can use the dualstack_sock.bind(("", 0)) trick
        ...
    else:
        ...

Currently, folks who need this have to implement it downstream.)

Ow, that's definitely something I would like to fix! Strange that nobody has reported that as an issue. I definitely meant for the listener to bind to the same ephemeral port if the local port was omitted.

[gschaffner]

I just did the same thing pytest-asyncio was doing. I also thought about allowing a fixture request parameter for the address family.

I also thought about that. I think it would be a good feature, but not enough to fix this problem: what family do I pass to indicate that I need a port that's both free on v4 and free on v6?

It could have a parameter families: Sequence[AddressFamily]. families could default to [AF_INET, AF_INET6] on platforms that have both, in order to ensure that the fixture-without-parameters example create_tcp_listener(local_port=free_tcp_port) in your docs works on all platforms.

Ow, that's definitely something I would like to fix! Strange that nobody has reported that as an issue. I definitely meant for the listener to bind to the same ephemeral port if the local port was omitted.

I opened #857 :)

[agronholm]

It could have a parameter families: Sequence[AddressFamily]. families could default to [AF_INET, AF_INET6] on platforms that have both, in order to ensure that the fixture-without-parameters example create_tcp_listener(local_port=free_tcp_port) in your docs works on all platforms.

On dual-stack platforms, using AF_INET6 should work for both families – you'd have to explicitly opt out of IPv4 with sock.setsockopt(IPPROTO_IPV6, socket.IPV6_V6ONLY, 1). The only reason this might be an issue in create_tcp_listener() is because we iterate over the results of getaddrinfo() where the IPv4 and IPv6 results are distinct from each other. So a naive implementation is not going to work well outside of the test suite.

[gschaffner]

On dual-stack platforms, using AF_INET6 should work for both families – you'd have to explicitly opt out of IPv4 with sock.setsockopt(IPPROTO_IPV6, socket.IPV6_V6ONLY, 1).

Unfortunately IPV6_V6ONLY can default to 1 (https://man.archlinux.org/man/core/man-pages/ipv6.7.en#IPV6_V6ONLY). I think it is not very common in the wild but it can apparently happen. A quick search for bindv6only finds some tutorials mentioning it, e.g. https://www.digitalocean.com/community/tutorials/how-to-configure-tools-to-use-ipv6-on-a-linux-vps.

Also, AFAICT platforms can support IPv4 and IPv6 but not dualstack sockets (https://stackoverflow.com/a/2804499). Possibly that SO answer is no longer relevant? I do also see an answer from 2024 stating that OpenBSD does not support dualstack sockets (https://stackoverflow.com/a/78403590).

The only reason this might be an issue in create_tcp_listener() is because we iterate over the results of getaddrinfo() where the IPv4 and IPv6 results are distinct from each other. So a naive implementation is not going to work well outside of the test suite.

Yeah. That is why I am hoping we can add a non-naïve implementation, as AFAICT it is necessary in order to make create_tcp_listener(local_port=0) use the same port for both IP versions. (#857)

[agronholm]

@gschaffner I've simplified the code to not rely on dual stack anymore. Would you take another look?

[agronholm]

I think the fixtures should have an anyio prefix

Also I wonder if they're that much more useful than ephemeral-port-reserve?

The library you're referring to does not even try to work with IPv6, so it's a non-starter if we want to do things properly. What I'm unsure about is how well its technique works on Windows. I recall SO_REUSEADDR working a bit differently there. Would we need to add SO_EXCLUSIVEADDRUSE to the socket options?

[Zac-HD]

This looks good to me, but if there was a socket bug of some kind I wouldn't expect to spot it 😅

[agronholm]

This looks good to me, but if there was a socket bug of some kind I wouldn't expect to spot it 😅

Ah. well, thanks for trying 😄