Bump ubuntu 22.04 as default SKU for Azure Batch

[pditommaso]

This PR bumps Ubuntu 22.04 as default SKU for Azure Batch node pools. This is required because Ubuntu 20.04 is going to be retired by April 2025 (See below):

Support for Ubuntu 20.04 LTS for Azure Batch pools will be retired on 23 April 2025
You're receiving this notice because you're currently using either an Ubuntu 20.04 LTS Marketplace or derived image with Azure Batch pools subject to support end of life.

Azure Batch typically follows standard end of life timelines set by publishers for supported images from the Azure Marketplace. Ubuntu 20.04 LTS is reaching the end of standard support life. Batch pools with Ubuntu 20.04 LTS VM images and the Batch node agent SKU batch.node.ubuntu 20.04 will no longer be supported in Batch after 23 April 2025.

[netlify]

✅ Deploy Preview for nextflow-docs-staging canceled.

Name	Link
🔨 Latest commit	fd26a07
🔍 Latest deploy log	https://app.netlify.com/sites/nextflow-docs-staging/deploys/67b86636eba93400088368d3

[bentsherman]

Docs need to be updated here

[pditommaso]

Well spotted

…

On Thu, Feb 20, 2025 at 2:25 PM Ben Sherman ***@***.***> wrote: Docs need to be updated here <https://nextflow.io/docs/latest/azure.html#pool-nodes> — Reply to this email directly, view it on GitHub <#5804 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGHOSG6SQMIMWPW2MRRRP32QXJVHAVCNFSM6AAAAABXQXQ5MKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNZRGQ4TMOBQHE> . You are receiving this because you authored the thread.Message ID: ***@***.***> [image: bentsherman]*bentsherman* left a comment (nextflow-io/nextflow#5804) <#5804 (comment)> Docs need to be updated here <https://nextflow.io/docs/latest/azure.html#pool-nodes> — Reply to this email directly, view it on GitHub <#5804 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGHOSG6SQMIMWPW2MRRRP32QXJVHAVCNFSM6AAAAABXQXQ5MKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNZRGQ4TMOBQHE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[pditommaso]

@adamrtalbot happy with this?

[adamrtalbot]

Just needed to correct the docs: ad7634e

[adamrtalbot]

Wait hold on, I get an error:

> nextflow-dev run hello
N E X T F L O W  ~  version 25.01.0-edge
Launching `https://github.com/nextflow-io/hello` [admiring_nobel] DSL2 - revision: afff16a9b4 [master]
ERROR ~ Error executing process > 'sayHello (3)'

Caused by:
  Cannot find a matching VM image with publisher=microsoft-azure-batch; offer=ubuntu-server-container; OS type=linux; verification type=verified

[pditommaso]

umm, how tests are passing?!

[adamrtalbot]

There is no Azure Batch ubuntu-2204 image 🤦 I think we need to use the DSVM replacement.

publisher = "microsoft-dsvm"
offer = "ubuntu-hpc"
sku = "22.04"
agent = "batch.node.ubuntu 22.04"

Microsoft just do a "latest supported" tag :shakefist:

[adamrtalbot]

A more recent alternative would be this:

publisher = "canonical"
offer = "ubuntu-24_04-lts
sku = "server"
agent = "batch.node.ubuntu 24.04"

Will give it a try.

[adamrtalbot]

A more recent alternative would be this:

publisher = "canonical"
offer = "ubuntu-24_04-lts
sku = "server"
agent = "batch.node.ubuntu 24.04"

Will give it a try.

Did not work - DSVM it is (which comes with GPU drivers and stuff so might be better anyway).

[pditommaso]

Deprecation notes mention

Required action

Stop using Ubuntu 20.04 LTS based VM images for Batch pools before 23 April 2025. Please migrate your Batch pools to a VM image based on Ubuntu 22.04 or later, or the microsoft-dsvm ubuntu-hpc 2204 (or later) image for container and/or workloads requiring Infiniband/GPU support. You can also migrate to any other Batch-supported VM image, if amenable for your workload. Existing Batch pools cannot be updated with a new VM image reference and creating a new Batch pool is required for migration.

[adamrtalbot]

feat(Azure-Batch): Switch to using Microsoft DSVM machine 22.04 inste…

Ok so 2e69a40 is correct

[pditommaso]

Can you revert and revert and test it?

[adamrtalbot]

You mean rollback to 192699b? There isn't a 22.04 image so this doesn't work.

Or do you want me to test the CI?

[pditommaso]

I've tried:

• sku: batch.node.ubuntu 22.04
• offer: ubuntu-hpc
• publisher: microsoft-dsvm

It tries to create the pool, but then it fails with:

{"Code":"BadRequest","Message":"The selected VM size 'Standard_D4_v3' cannot boot Hypervisor Generation '2'. If this was a Create operation please check that the Hypervisor Generation of the Image matches the Hypervisor Generation of the selected VM Size. If this was an Update operation please select a Hypervisor Generation '2' VM Size. For more information, see https://aka.ms/azuregen2vm"} (Code: Provider Error Json)

Not sure what to do

[adamrtalbot]

That's a very old VM size which isn't a generation 2 VM, so it doesn't support the virtual machine image. It's been deprecated now so you should switch to something like a Dsv6 machine.

Note: you'll need to make sure you have quota in the CI batch account.

[pditommaso]

Can you suggest one in concrete?

[adamrtalbot]

Let's go with a Standard_D4_v5, cheap and should be widely available.

[pditommaso]

Hanging with this error message when creating the pool The specified account has reached VM series core quota for standardDv5Family (Code: AccountVMSeriesCoreQuotaReached)

I see two problems, 1) it should not hang on pool creation errors, 2) still unable to find a valid default instance type for ubuntu.

If you could give a try to address at least the latter it would be appreciated.

[adamrtalbot]

Hanging with this error message when creating the pool The specified account has reached VM series core quota for standardDv5Family (Code: AccountVMSeriesCoreQuotaReached)

We need to make sure there is sufficient quota in the Batch account. Which Batch account is it running in?

We can either increase the quota or use a machine with quota available.

Note spot machines have their own generic quota for all machine types, so this might be an option.

p.s. there is one alternative which is much, much more complicated but consistent with AWS/Google.

[adamrtalbot]

Regarding this:

1. it should not hang on pool creation errors

Yeah it sucks, Azure node pools are detached from running so I don't know whether we can catch it in Nextflow. Seqera Platform should be able to check the pool is in active state though.