Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

configs: Enable MPTCP support #5487

Closed
wants to merge 1 commit into from

Conversation

matttbe
Copy link
Contributor

@matttbe matttbe commented Jun 1, 2023

First, thank you for all the work around the RPi, it is really a useful project, especially from an educational point of view!

About this PR and MPTCP, here is the official description says:

Multipath TCP (MPTCP) connections send and receive data over multiple subflows in order to utilize multiple network paths. Each subflow uses the TCP protocol, and TCP options carry header information for MPTCP.

In other words, we can easily use multiple Internet links at the same time and for the same connection. For example, we can plug a 4G/5G USB dongle in a RPi and use both the WiFi and the cellular networks to get more speed, more redundancy, etc.

An example of use with a RPi4 and Ubuntu 22.04: https://www.tessares.net/mptcp-proxy-setup-at-home-easier-now/

MPTCP support is light and enabled in most Linux distributions today: RHEL, Debian, Ubuntu, Fedora, ArchLinux, etc.

Note that to use MPTCP with IPv6 addresses, it is required to have IPV6 not built as a module, see commit 0ed37ac ("mptcp: depends on IPV6 but not as a module") for more details. When talking with netdev maintainers before the introduction of MPTCP in the upstream kernel in v5.6, it looks like there is not so much reasons to still have IPv6 as a module as IPv6 is (supposed to be) "today's Internet". This module will likely be quickly loaded on devices with network devices anyway.

If having IPV6 built-in is really an issue, MPTCP can still be enabled but without IPv6 support which is not recommended today.

CONFIG_INET_MPTCP_DIAG is also enabled as a module, similar to CONFIG_INET_DIAG.

Closes #4254

@matttbe
Copy link
Contributor Author

matttbe commented Jun 1, 2023

Note: the error with checkpatch is because the GitHub action doesn't fetch all the Git history and it doesn't have info about the mentioned commit:

WARNING: Unknown commit id '0ed37ac586c0', maybe rebased or not pulled?

The official description says:

  Multipath TCP (MPTCP) connections send and receive data over multiple
  subflows in order to utilize multiple network paths. Each subflow uses
  the TCP protocol, and TCP options carry header information for MPTCP.

In other words, we can easily use multiple Internet links at the same
time and for the same connection. For example, we can plug a 4G/5G USB
dongle in a RPi and use both the WiFi and the cellular networks to get
more speed, more redundancy, etc.

An example of use with a RPi4 and Ubuntu 22.04:

  https://www.tessares.net/mptcp-proxy-setup-at-home-easier-now/

MPTCP support is light and enabled in most Linux distributions today:
RHEL, Debian, Ubuntu, Fedora, ArchLinux, etc.

Note that to use MPTCP with IPv6 addresses, it is required to have IPV6
not built as a module, see commit ("mptcp: depends on IPV6 but not as a
module") for more details. When talking with netdev maintainers before
the introduction of MPTCP in the upstream kernel in v5.6, it looks like
there is not so much reasons to still have IPv6 as a module as IPv6 is
(supposed to be) "today's Internet". This module will likely quickly be
loaded on devices with network devices anyway.

If having IPV6 built-in is really an issue, MPTCP can still be enabled
but without IPv6 support which is not recommended today.

CONFIG_INET_MPTCP_DIAG is also enabled as a module, similar to
CONFIG_INET_DIAG.

Closes: raspberrypi#4254
Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
@matttbe
Copy link
Contributor Author

matttbe commented Jun 12, 2023

I just removed the mention of 0ed37ac commit in the commit message just to please the CI, the code is still the same.

@pelwell
Copy link
Contributor

pelwell commented Jul 14, 2023

MPTCP itself is moderately large, but it's building in IPV6 that really makes the difference. These builds are for a Pi 1, which is going to be most memory-constrained:

no MPTCP:
-rwxrwxr-x 1 phil phil 14216160 Jul 14 14:08 arch/arm/boot/Image
-rwxrwxr-x 1 phil phil  6463096 Jul 14 14:08 arch/arm/boot/zImage

with MPTCP:
-rwxrwxr-x 1 phil phil 14784492 Jul 14 14:11 arch/arm/boot/Image
-rwxrwxr-x 1 phil phil  6763384 Jul 14 14:11 arch/arm/boot/zImage

with MPTCP but no IPV6:
-rwxrwxr-x 1 phil phil 14334852 Jul 14 14:48 arch/arm/boot/Image
-rwxrwxr-x 1 phil phil  6523616 Jul 14 14:48 arch/arm/boot/zImage

@pelwell
Copy link
Contributor

pelwell commented Jul 17, 2023

You mention that Ubuntu includes MPTCP support, but from what I've seen it doesn't include MPTCP over IPV6 (I'm looking at 6.2.0 kernel from Apr 3 2023). As we have done for a few advanced features recently I will consider adding MPTCP (without IPV6) to the aarch64 kernel.

Looking at the aarch64 build instead:

without:
-rw-rw-r-- 1 phil phil 22475264 Jul 17 09:54 arch/arm64/boot/Image
-rw-rw-r-- 1 phil phil  8258343 Jul 17 09:54 kernel8.img

with MPTCP and IPV6:
-rw-rw-r-- 1 phil phil 23237120 Jul 17 10:02 arch/arm64/boot/Image
-rw-rw-r-- 1 phil phil  8564388 Jul 17 10:02 kernel8.img

with MPTCP but no IPV6:
-rw-rw-r-- 1 phil phil 22632960 Jul 17 09:52 arch/arm64/boot/Image
-rw-rw-r-- 1 phil phil  8322396 Jul 17 09:52 kernel8.img

That's a 160kB tax for MPTCP and 760kB (that's a 3.3% increase) for MPTCP and IPV6.

If @popcornmix is in agreement I'll just add the MPTCP=y setting, and only to the ARCH=arm64 bcm2711_defconfig.

@popcornmix
Copy link
Collaborator

OK

pelwell added a commit that referenced this pull request Jul 17, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
@pelwell
Copy link
Contributor

pelwell commented Jul 17, 2023

See 216be50.

@pelwell pelwell closed this Jul 17, 2023
@matttbe
Copy link
Contributor Author

matttbe commented Jul 17, 2023

@pelwell thank you for having checked this!

What is the size we need to minimise here? Is it the size of what is loaded in RAM or the size of the .img?

I suspect IPv6 will be loaded in most setup by default, no? So maybe there is no need to have it in a separated module and it could be inline to save a bit of space and CPU cycles?

These builds are for a Pi 1, which is going to be most memory-constrained:

I understand MPTCP might not be enabled in this environment.

You mention that Ubuntu includes MPTCP support, but from what I've seen it doesn't include MPTCP over IPV6 (I'm looking at 6.2.0 kernel from Apr 3 2023).

Oh, I missed that. Thank you for having reported this. I will ask Ubuntu devs if it is normal.

If @popcornmix is in agreement I'll just add the MPTCP=y setting, and only to the ARCH=arm64 bcm2711_defconfig.

Thank you :)

Is this architecture that limited in memory too? (especially if IPv6 is enabled and used by default for local addresses, etc.?)

popcornmix added a commit to raspberrypi/firmware that referenced this pull request Jul 17, 2023
See: raspberrypi/linux#4683

kernel: configs: Add MPTCP for 64-bit platforms
See: raspberrypi/linux#5487
popcornmix added a commit to raspberrypi/rpi-firmware that referenced this pull request Jul 17, 2023
See: raspberrypi/linux#4683

kernel: configs: Add MPTCP for 64-bit platforms
See: raspberrypi/linux#5487
@pelwell
Copy link
Contributor

pelwell commented Jul 17, 2023

RAM size is the most important metric, because it isn't something that can be expanded. The early Pis had as little as 128MB, which would make a loss of >500kB painful.

Every optional feature has some kind of overhead, either in RAM size or execution speed (or both), otherwise it wouldn't be optional. When evaluating the inclusion of features we think of those overheads as being a tax on users who don't use the feature. With MPTCP I feel the non-users will vastly outweigh the users, and the same goes for IPV6. The fact that both features are quite large makes the combination unattractive.

@matttbe
Copy link
Contributor Author

matttbe commented Jul 17, 2023

@pelwell thank you for the explanation!

With MPTCP I feel the non-users will vastly outweigh the users, and the same goes for IPV6. The fact that both features are quite large makes the combination unattractive.

I do understand for MPTCP.

For IPv6, I don't have a good view on how the RPi are mostly used but I guess that if the devices are connected to a network via WiFi or Ethernet, IPv6 will be loaded, no? Very likely at least a link-local IPv6 address will be created, IPv6 packets will be broadcasted, etc.

I looked a bit at what they were doing on Ubuntu side:

  • The last LTS version has IPV6=y + MPTCP_IPV6=y (armhf - arm64)
  • The version for Lunar with a 6.2.0 doesn't (arm64) (the versions with 5.15 and 5.19 have IPV6=y)
  • But if you look at the source code (here and here), IPV6 (and MPTCP_IPV6) should be enabled:
CONFIG_IPV6                                     policy<{'amd64': 'y', 'arm64': 'y', 'armhf': 'y', 'ppc64el': 'y', 's390x': 'y'}>
CONFIG_IPV6                                     mark<ENFORCED> note<if this is a module we get a module load for every ipv6 packet>
  • The note is very interesting: it looks like it is a bug if we don't have IPV6=y (I will follow up on that on Ubuntu side) and they seem to recommend to do that.

In other words, I would say that if one day you switch to CONFIG_IPV6=y as apparently recommended by Ubuntu devs (and others), don't hesitate to enable CONFIG_MPTCP_IPV6 too (probably not a lot of diff if IPV6 is already =y) :)

@matttbe
Copy link
Contributor Author

matttbe commented Jul 17, 2023

(FYI, here is the bug report regarding CONFIG_IPV6=m on Ubuntu side: https://bugs.launchpad.net/ubuntu/+source/linux-raspi/+bug/2027983)

pelwell added a commit that referenced this pull request Jul 18, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
pelwell added a commit that referenced this pull request Jul 18, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Jul 20, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Jul 24, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Jul 24, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Jul 27, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Jul 31, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Aug 3, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Aug 8, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Aug 9, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Aug 14, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Aug 17, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Aug 21, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
@matttbe
Copy link
Contributor Author

matttbe commented Aug 27, 2023

(FYI, here is the bug report regarding CONFIG_IPV6=m on Ubuntu side: https://bugs.launchpad.net/ubuntu/+source/linux-raspi/+bug/2027983)

FYI, the modification has been done on Ubuntu to have IPV6 built-in:

the ipv6 module is automatically loaded so probably not worth modularizing it (and losing functionality because of that).

So thanks to that, we can have CONFIG_MPTCP_IPV6=y.

popcornmix pushed a commit that referenced this pull request Aug 29, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Sep 5, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Sep 5, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Sep 8, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Sep 12, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Sep 13, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Sep 13, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Sep 19, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Sep 25, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Oct 6, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Oct 11, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Oct 24, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Oct 30, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Nov 16, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Nov 21, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
popcornmix pushed a commit that referenced this pull request Nov 29, 2023
Enable MPTCP for 64-bit kernels, but not IPV6 support because that
requires IPV6=y and bloats the kernel by 500kB.

See: #5487

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Suggestion: Include MPTCP
3 participants