Explore the impact of new GraalVM defaults on Quarkus

[zakkak]

Description

oracle/graal#10178 changes some defaults around reachability metadata, namely:

1. All reflectively-accessed types now have all their fields registered for runtime access by default
2. All reflectively-accessible classes are now registered for unsafe allocation
3. All serializable types can now be serialized with all possible constructors

The goal of this issue is to track the impact of this changes on the Quarkus core integration tests to see if/when Quarkus should opt-out of these new defaults.

Implementation ideas

The new changes are being tested in https://github.com/graalvm/mandrel/actions/runs/12114945249 (with collector API tag graal-pr-10178-reachability-test) and will be compared to the results of https://github.com/graalvm/mandrel/actions/runs/12114969951 (with collector API tag graal-pr-10178-reachability-test-before)

[quarkus-bot]

/cc @Karm (mandrel), @galderz (mandrel)

[zakkak]

The results indicate that in most cases the binary size difference is negligible and in the worst case we observe ~1% increase in the image size.

Although ~1% doesn't seem excessive, we probably need to opt-out of this new default for registrations managed by Quarkus, to reduce the impact as much as possible since oracle/graal#9679 is expected to bring more increases like this.

The raw results are:

Image Name	Total Bytes Difference (%)
quarkus-elytron-security-jdbc-integration-test-3.17.2	0.01
quarkus-elytron-security-ldap-integration-test-3.17.2	0.20
amazon-lambda-3.17.2	0.14
amazon-lambda-http-3.17.2	0.09
amazon-lambda-rest-funqy-3.17.2	0.01
amazon-lambda-rest-reactive-routes-3.17.2	0.01
amazon-lambda-rest-resteasy-reactive-3.17.2	0.01
amazon-lambda-rest-servlet-3.17.2	0.01
amqp-3.17.2	0.10
awt-3.17.2	0.01
awt-packaging-3.17.2	0.01
bouncycastle-3.17.2	0.60
bouncycastle-fips-3.17.2	0.01
bouncycastle-jsse-3.17.2	0.08
cache-3.17.2	0.00
devtools-registry-client-3.17.2	0.01
elasticsearch-java-client-3.17.2	0.01
elasticsearch-rest-client-3.17.2	0.01
elytron-resteasy-3.17.2	0.01
elytron-resteasy-reactive-3.17.2	0.83
elytron-security-3.17.2	0.01
elytron-security-oauth2-3.17.2	0.00
elytron-undertow-3.17.2	0.01
flyway-3.17.2	0.01
grpc-interceptors-3.17.2	0.00
grpc-mutual-auth-3.17.2	0.00
grpc-plain-text-gzip-3.17.2	0.00
grpc-plain-text-mutiny-3.17.2	0.09
grpc-proto-v2-3.17.2	0.00
grpc-test-random-port-3.17.2	0.01
grpc-tls-3.17.2	0.00
grpc-tls-p12-3.17.2	0.00
hibernate-orm-envers-3.17.2	0.07
hibernate-orm-panache-3.17.2	0.43
hibernate-orm-panache-kotlin-3.17.2	0.00
hibernate-orm-rest-data-panache-3.17.2	0.00
hibernate-orm-tenancy-connection-resolver-3.17.2	0.01
hibernate-orm-tenancy-connection-resolver-legacy-qualifiers-3.17.2	0.07
hibernate-orm-tenancy-datasource-3.17.2	0.01
hibernate-orm-tenancy-schema-3.17.2	0.07
hibernate-reactive-db2-3.17.2	0.08
hibernate-reactive-mariadb-3.17.2	0.00
hibernate-reactive-mysql-3.17.2	0.01
hibernate-reactive-mysql-agroal-flyway-3.17.2	0.00
hibernate-reactive-panache-3.17.2	0.00
hibernate-reactive-panache-kotlin-3.17.2	0.47
hibernate-reactive-postgresql-3.17.2	0.00
hibernate-search-orm-elasticsearch-3.17.2	0.00
hibernate-search-orm-elasticsearch-outbox-polling-3.17.2	0.00
hibernate-search-orm-elasticsearch-tenancy-3.17.2	0.00
hibernate-search-orm-opensearch-3.17.2	0.00
hibernate-search-standalone-elasticsearch-3.17.2	0.10
hibernate-search-standalone-opensearch-3.17.2	0.10
hibernate-validator-3.17.2	0.00
infinispan-cache-3.17.2	0.00
infinispan-cache-jpa-3.17.2	0.01
infinispan-client-3.17.2	0.10
jackson-3.17.2	0.01
jaxb-3.17.2	0.01
jaxp-3.17.2	0.01
jpa-3.17.2	0.07
jpa-db2-3.17.2	0.07
jpa-derby-3.17.2	0.08
jpa-h2-3.17.2	0.00
jpa-h2-embedded-3.17.2	0.01
jpa-mapping-xml-legacy-app-3.17.2	0.07
jpa-mapping-xml-modern-app-3.17.2	0.50
jpa-mariadb-3.17.2	0.01
jpa-mssql-3.17.2	0.07
jpa-mysql-3.17.2	0.00
jpa-oracle-3.17.2	0.00
jpa-postgresql-3.17.2	0.01
jpa-postgresql-withxml-3.17.2	0.07
jpa-without-entity-3.17.2	0.86
jsonb-3.17.2	0.12
kafka-3.17.2	0.08
kafka-avro-apicurio2-3.17.2	0.07
kafka-json-schema-apicurio2-3.17.2	0.00
kafka-oauth-keycloak-3.17.2	0.01
kafka-sasl-3.17.2	0.09
kafka-snappy-3.17.2	0.69
kafka-ssl-3.17.2	0.01
kafka-streams-3.17.2	0.01
keycloak-authorization-3.17.2	0.00
kotlin-serialization-3.17.2	0.01
kubernetes-client-3.17.2	0.35
kubernetes-service-binding-jdbc-3.17.2	0.58
liquibase-3.17.2	0.01
liquibase-mongodb-3.17.2	0.52
locales-all-3.17.2	0.63
locales-default-3.17.2	0.01
locales-some-3.17.2	0.01
logging-gelf-3.17.2	0.01
logging-json-3.17.2	0.94
logging-min-level-set-3.17.2	0.01
logging-min-level-unset-3.17.2	1.05
mailer-3.17.2	0.00
main-3.17.2	0.04
management-interface-3.17.2	0.22
management-interface-auth-3.17.2	0.01
micrometer-mp-metrics-3.17.2	0.11
micrometer-prometheus-3.17.2	0.00
mongodb-client-3.17.2	0.01
mongodb-devservices-3.17.2	0.11
mongodb-panache-3.17.2	0.01
mongodb-rest-data-panache-3.17.2	0.10
mutiny-native-jctools-3.17.2	0.01
narayana-jta-3.17.2	0.09
native-config-profile-3.17.2	0.01
no-awt-3.17.2	0.01
observability-lgtm-3.17.2	0.00
oidc-3.17.2	0.00
oidc-client-3.17.2	0.00
oidc-client-reactive-3.17.2	0.00
oidc-client-wiremock-3.17.2	0.00
oidc-code-flow-3.17.2	0.00
oidc-tenancy-3.17.2	0.00
oidc-wiremock-3.17.2	0.00
oidc-wiremock-providers-3.17.2	0.00
openshift-client-3.17.2	0.00
opentelemetry-3.17.2	0.00
opentelemetry-jdbc-instrumentation-3.17.2	0.04
opentelemetry-mongodb-client-instrumentation-3.17.2	0.00
opentelemetry-redis-instrumentation-3.17.2	0.00
picocli-native-3.17.2	0.00
quartz-3.17.2	0.01
qute-3.17.2	0.00
reactive-db2-client-3.17.2	0.78
reactive-messaging-kafka-3.17.2	0.00
reactive-messaging-pulsar-3.17.2	0.00
reactive-messaging-rabbitmq-3.17.2	0.78
reactive-messaging-rabbitmq-dyn-3.17.2	0.87
reactive-mysql-client-3.17.2	0.77
reactive-oracle-client-3.17.2	0.01
reactive-pg-client-3.17.2	0.78
redis-cache-3.17.2	0.00
redis-client-3.17.2	0.00
rest-client-3.17.2	0.01
rest-client-reactive-3.17.2	0.00
rest-client-reactive-kotlin-serialization-3.17.2	0.00
rest-client-reactive-multipart-3.17.2	0.80
rest-client-reactive-stork-3.17.2	0.00
resteasy-client-oidc-token-propagation-3.17.2	0.00
resteasy-jackson-3.17.2	0.01
resteasy-reactive-kotlin-standard-3.17.2	0.00
security-webauthn-3.17.2	0.07
smallrye-config-3.17.2	0.08
smallrye-graphql-3.17.2	0.00
smallrye-graphql-client-3.17.2	0.00
smallrye-graphql-client-keycloak-3.17.2	0.59
smallrye-jwt-token-propagation-3.17.2	0.64
smallrye-metrics-3.17.2	0.10
spring-boot-properties-3.17.2	0.01
spring-cloud-config-client-3.17.2	0.00
spring-data-jpa-3.17.2	0.00
spring-data-rest-3.17.2	0.00
spring-di-3.17.2	0.01
spring-web-3.17.2	0.01
test-extension-tests-3.17.2	0.01
vertx-3.17.2	0.10
vertx-graphql-3.17.2	0.01
vertx-http-3.17.2	0.11
vertx-http-compressors-all-3.17.2	0.13
vertx-http-compressors-some-3.17.2	0.01
vertx-web-3.17.2	0.89
vertx-web-jackson-3.17.2	1.05
web-dependency-locator-3.17.2	0.11
websockets-3.17.2	0.13
websockets-next-3.17.2	1.00
test	0.01

The table was generated by parsing
before.json
after.json

with:

import json

with open('before.json', 'r') as file:
    before_data = json.load(file)

with open('after.json', 'r') as file:
    after_data = json.load(file)

# filter out the images that start with acme and foo
before_data = [item for item in before_data if item['img_name'].startswith('quarkus-integration-test')]
# index after_data by image name
after_data = {item['img_name']: item for item in after_data if item['img_name'].startswith('quarkus-integration-test')}

print("| Image Name | Total Bytes Difference (%) |")
print("|------------|----------------------------|")

# Collect data for histogram
differences = []

for item in before_data:
    img_name = item['img_name']
    before = item['image_size_stats']['total_bytes']
    after = after_data[img_name]['image_size_stats']['total_bytes']
    diff = ( after - before) / after * 100
    img_name = img_name.removeprefix('quarkus-integration-test-').removesuffix('-runner')
    differences.append(diff)

    print(f"| {img_name:70s} | {diff:.2f} |")

import matplotlib.pyplot as plt

# Plot histogram
plt.hist(differences, bins=20, edgecolor='black')
plt.title('Histogram of Total Bytes Difference (%)')
plt.xlabel('Total Bytes Difference (%)')
plt.ylabel('Frequency')
plt.grid(True)
plt.savefig('histogram.png')
plt.close()

[zakkak]

oracle/graal#10178 has now been split in:

1. [GR-60234] Register superclass serialization constructors by default oracle/graal#10213
2. [GR-60235] Register reflectively-accessed types as unsafe allocated oracle/graal#10214

The "All reflectively-accessed types now have all their fields registered for runtime access by default" change is postponed for now.

The split changes are being evaluated in https://github.com/graalvm/mandrel/actions/runs/12140368060 (with collector API tag graal-pr-10213-reachability-test) and https://github.com/graalvm/mandrel/actions/runs/12140386106 (with collector API tag graal-pr-10213-and-10214-reachability-test).

The results indicate that oracle/graal#10213 has zero impact (except for websockets where we observe and images size increase of 0.48%) while with oracle/graal#10214 in most cases the binary size difference is negligible and in the worst case we observe ~1% increase in the image size.

The oracle/graal#10214 results are suspiciously similar to the ones from
oracle/graal#10178 indicating that the overhead is actually coming from registering types as unsafe allocated and not from registering all fields. As a result, I have requested having the option to opt-out from this new default and we will be able to do so by explicitly setting "unsafeAllocated": false in the configuration files we generate.

Comparing the results of oracle/graal#10214 with those of oracle/graal#10178 I see that there are only 3 tests that seem to be affected more than 0.1% by the fields registration.

Image Name	Total Bytes Difference (%)
amazon-lambda-3.17.2	0.14
jaxp-3.17.2	-0.80
logging-min-level-unset-3.17.2	0.93
rest-client-reactive-multipart-3.17.2	0.82
vertx-http-compressors-some-3.17.2	-0.12
vertx-web-jackson-3.17.2	0.12

The json files used are pr10214.json, pr10213.json, and before-pr10213.json
in addition to the ones from #44864 (comment)

I scheduled a new run (see https://github.com/graalvm/mandrel/actions/runs/12157739699) to get the results right before oracle/graal#10213 and oracle/graal#10214 to make sure what I am seeing is not an artifact of another commit on master, (the collector tag is graal-pr-10213-and-10214-reachability-test-before)

I will update once more when I have the results. Updated