(Chore) Improve production bundle size (#1890)

* (Chore) Improve production bundle size

* Chunks all dependencies

* Sets minimum bundle size; splits into specific bundles

* Revert babel.config change

* Adds ADR

* Combined before and after stats

* Converts table to HTML

* - Removes unused dependencies
- Updates Babel config
- Adds `canvas` package for Canvas support is JSDOM
- Removes unused `tsconfig.test.json`

* Set ADR status to accepted

Author: Jan Jaap van Deursen

babel.config.js

@@ -1,26 +1,12 @@
 // SPDX-License-Identifier: MPL-2.0
 // Copyright (C) 2019 - 2021 Gemeente Amsterdam
 module.exports = (api) => {
-  const presetEnv =
-    api.env() === 'lint'
-      ? ''
-      : [
-          '@babel/preset-env',
-          {
-            modules: false,
-            useBuiltIns: 'usage',
-            targets: {
-              esmodules: true,
-              chrome: 42,
-              firefox: 68,
-            },
-            corejs: 3,
-          },
-        ]
+  api.cache(false)
 
   return {
     plugins: [
       'styled-components',
+      '@babel/plugin-transform-modules-commonjs',
       '@babel/plugin-proposal-class-properties',
       [
         'inline-react-svg',
@@ -29,33 +15,39 @@ module.exports = (api) => {
         },
       ],
       '@babel/plugin-syntax-dynamic-import',
+      '@babel/plugin-transform-runtime',
     ],
     presets: [
-      presetEnv,
       [
         '@babel/preset-react',
         {
           runtime: 'automatic',
         },
       ],
       '@babel/preset-typescript',
-    ].filter(Boolean),
+      [
+        '@babel/preset-env',
+        {
+          useBuiltIns: 'usage',
+          targets: {
+            esmodules: true,
+            chrome: 42,
+            firefox: 68,
+          },
+          corejs: 3,
+        },
+      ],
+    ],
     env: {
       production: {
         only: ['src'],
-        plugins: [
-          'transform-react-remove-prop-types',
-          '@babel/plugin-transform-react-inline-elements',
-          '@babel/plugin-transform-react-constant-elements',
-        ],
+        plugins: ['transform-react-remove-prop-types'],
       },
       test: {
         plugins: [
-          '@babel/plugin-transform-modules-commonjs',
-          'dynamic-import-node',
           [
             'babel-plugin-styled-components',
-            { ssr: false, displayName: false, namespace: 'sc' },
+            { ssr: false, displayName: false, namespace: 'sc', pure: true },
           ],
         ],
       },

docs/adr/0012-bundle-splitting-strategy.md

@@ -0,0 +1,113 @@
+# Bundle splitting strategy
+
+Date: 2021-11-19
+
+## Status
+
+Accepted
+
+## Context
+
+So far, no significant effort has been put into optimizing the way bundles and chunks are generated for the application, other than lazy loading major parts of the front and back-office modules by means of implementing [React code splitting](https://reactjs.org/docs/code-splitting.html).
+
+Since the application has only one entry point, `app.js`, Webpack doesn't know how to split the code and its dependencies into usable chunks that are not too small and not too big. Too small would be less than a couple of kilobytes and too big would be a chunk of a couple of hundreds of kilobytes. It takes some tinkering to find the sweet spot.
+
+Without custom settings in Webpack, one 'main' chunk is generated that contains most, if not all, code and dependencies from the entry point. At the time of writing this is a blob of close to 5 MB of code. If only a very small part of that blob changes, let's say a 20 Kb dependency or a 4 Kb component, the browser is forced to download the whole 5 MB instead of just the part that changed. Not a very big impact on a fast office or home connection, but detrimental to the experience on a mobile device with a 4G or 3G connection that has a narrow bandwidth and suffers from latency issues.
+
+To improve loading times and rendering speed, the big main bundle needs to be split up into smaller logical parts. We not only help our website's visitors, we will also reduce the amount of data that has to be transferred over the network and this reducing the load on the web server.
+
+## Goal
+
+Split the code in such a way that the least amount of chunks has to be regenerated on each consecutive build.
+
+## Approach
+
+First, we need to see what the current situation is. Below is a screenshot from the overview that the bundle analyzer plugin generated. To the right of the screenshot we can see the chunks that come from the code splitting with React. On the left, we can also clearly see that the main chunk is really large and contains application code as well as dependencies. Even if the dependencies don't receive any updates, the whole main chunk will be updated, which will be the case for each release that we push to production.
+
+<img src="./img/chunk_optimization_before.png" />
+
+### Aggressive splitting
+
+We can split the code in such a way that each dependency has its own chunk. This way only the updated dependencies need to be downloaded after a new build/release. This will, however, generate such a large amount of chunks that it becomes inefficient; too many requests with too low a payload size.
+
+### Fully customized splitting
+
+Looking at which dependencies belong together (for instance `Leaflet` and `proj4`), which dependencies are likely to be updated often (or not, like `react-reactive-form`), we can come up with a configuration that is fully tailored to the project's list of dependencies. Efficient in terms of load times, but not in terms of maintenance; whenever a dependency is removed, renamed or gets a breaking update, chances are we need to revisit the Webpack configuration to reflect the change.
+
+### Middle ground
+
+A solution that covers both aforementioned scenarios would:
+
+- compile a separate chunk for each dependency
+- have a minimum size for generated chunks (combining chunks if they're too small)
+- have a maximum size for generated chunks (splitting chunks if they're too big)
+- combine dependencies that are unlikely to be updated often
+- have as less custom configuration as possible
+
+## Outcome
+
+Applying all the middle ground requirements, the result is as follows:
+
+<img src="./img/chunk_optimization_after.png" />
+
+### Comparison
+
+__Chunk sizes__
+
+Before: 3.17 MB
+
+After: 2.69 MB
+
+__Average load times (throttled, simulated network traffic)__
+
+<table>
+  <thead>
+    <tr>
+      <th rowspan="2"></th>
+      <th colspan="2">WiFi</th>
+      <th colspan="2">4G</th>
+      <th colspan="2">Good 3G</th>
+    </tr>
+    <tr>
+      <th>Before</th>
+      <th>After</th>
+      <th>Before</th>
+      <th>After</th>
+      <th>Before</th>
+      <th>After</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <th>DOMContentLoaded (ms)</th>
+      <td>180</td>
+      <td>230</td>
+      <td>4200</td>
+      <td>3000</td>
+      <td>13000</td>
+      <td>6250</td>
+    </tr>
+    <tr>
+      <th>load (s)</th>
+      <td>950</td>
+      <td>940</td>
+      <td>6200</td>
+      <td>5200</td>
+      <td>15000</td>
+      <td>13000</td>
+    </tr>
+    <tr>
+      <th>completed (s)</th>
+      <td>650</td>
+      <td>712</td>
+      <td>5500</td>
+      <td>4800</td>
+      <td>13500</td>
+      <td>12000</td>
+    </tr>
+  </tbody>
+</table>
+
+## Conclusion
+
+Load times for WiFi connection are more or less the same, but the most significant change is for the 4G and 3G connections. Totally worth the effort.

docs/adr/README.md

@@ -12,3 +12,4 @@
 9.  [Add MSW for mocking fetch calls](./0009-stop-mocking-fetch-in-tests.md)
 10. [Naming conventions](./0010-naming-conventions.md)
 11. [Embedded application](./0011-embedded-application.md)
+12. [Bundle splitting strategy](./0012-bundle-splitting-strategy.md)

docs/adr/img/chunk_optimization_after.png

@@ -1,5 +1,9 @@
 // SPDX-License-Identifier: MPL-2.0
 // Copyright (C) 2021 Gemeente Amsterdam
-const content = 'svg'
-export const ReactComponent = content
-export default content
+import { createElement } from 'react'
+import type { FunctionComponent, SVGProps } from 'react'
+
+const SvgMock: FunctionComponent<SVGProps<SVGSVGElement>> = (props) =>
+  createElement('svg', props)
+
+export default SvgMock

docs/adr/img/chunk_optimization_before.png

@@ -1,5 +1,6 @@
 // SPDX-License-Identifier: MPL-2.0
 // Copyright (C) 2018 - 2021 Gemeente Amsterdam
+// @ts-check
 const path = require('path')
 const webpack = require('webpack')
 const pkgDir = require('pkg-dir')
@@ -8,50 +9,43 @@ const MiniCssExtractPlugin = require('mini-css-extract-plugin')
 const BundleAnalyzerPlugin =
   require('webpack-bundle-analyzer').BundleAnalyzerPlugin
 const CopyPlugin = require('copy-webpack-plugin')
+const mergeWith = require('lodash/mergeWith')
+const isArray = require('lodash/isArray')
 
 const devMode = process.env.NODE_ENV !== 'production'
 const __rootdir = pkgDir.sync()
 
 const esModules = [
   path.resolve(__rootdir, 'node_modules/@amsterdam/asc-assets'),
   path.resolve(__rootdir, 'node_modules/@amsterdam/asc-ui'),
+  path.resolve(__rootdir, 'node_modules/@amsterdam/arm-core'),
+  path.resolve(__rootdir, 'node_modules/@datapunt/matomo-tracker-js'),
+  path.resolve(__rootdir, 'node_modules/@datapunt/matomo-tracker-react'),
+  path.resolve(__rootdir, 'node_modules/@amsterdam/react-maps'),
 ]
 
-module.exports = ({
-  babelQuery,
-  devtool,
-  entry,
-  mode,
-  optimization,
-  output,
-  performance = {},
-  plugins,
-  tsLoaders,
-}) => ({
-  mode,
-  entry,
+const mergeCustomizer = (objValue, srcValue) => {
+  if (isArray(objValue)) {
+    return objValue.concat(srcValue)
+  }
+}
+
+const baseConfig = /** @type { import('webpack').Configuration } */ {
   output: {
     path: path.resolve(__rootdir, 'build'),
     publicPath: '/',
-    ...output,
-  }, // Merge with env dependent settings
-  optimization,
+  },
+
   module: {
     rules: [
       {
-        test: /\.jsx?$/, // Transform all .js and .jsx files required somewhere with Babel
+        test: /\.(t|j)sx?$/, // Transform all .js and .jsx files required somewhere with Babel
         exclude: /node_modules/,
         include: [path.resolve(__rootdir, 'src'), ...esModules],
         use: {
           loader: 'babel-loader',
-          options: babelQuery,
         },
       },
-      {
-        test: /\.ts(x?)$/,
-        exclude: /node_modules/,
-        use: tsLoaders,
-      },
       {
         test: /\.css$/,
         use: [
@@ -152,6 +146,7 @@ module.exports = ({
       },
     ],
   },
+
   plugins: [
     // Always expose NODE_ENV to webpack, in order to use `process.env.NODE_ENV`
     // inside your code for any environment checks; Terser will automatically
@@ -191,9 +186,8 @@ module.exports = ({
         },
       },
     }),
-  ]
-    .concat(plugins)
-    .filter(Boolean),
+  ].filter(Boolean),
+
   resolve: {
     modules: [path.resolve(__rootdir, 'src'), 'node_modules'],
     extensions: ['.js', '.jsx', '.react.js', '.ts', '.tsx'],
@@ -202,7 +196,13 @@ module.exports = ({
       types: path.resolve(__rootdir, 'src/types/'),
     },
   },
-  devtool,
+
   target: 'web', // Make web variables accessible to webpack, e.g. window
-  performance,
-})
+
+  stats: 'normal',
+}
+
+module.exports = {
+  baseConfig,
+  merge: (objValue, srcValue) => mergeWith(objValue, srcValue, mergeCustomizer),
+}