Here at Rover, our team of front-end engineers and engineers contributing to the web client projects is growing rapidly along with the company. In order to facilitate that growth, we recently spent some time looking at ways to improve the developer experience.
For the uninitiated, Webpack is an incredibly powerful and flexible build tool used to compile source code to deliverable web assets. In addition to being an industry-leading build tool, it is also a powerful development tool. For example, Webpack can build sourcemaps and watch the filesystem for changes. Any quest to improve the development experience for our team was likely to start with Webpack and its configuration.
Hot Module Replacement
The project started with a narrow and relatively modest goal: can we introduce Hot Module Replacement, or HMR, (re-loading of JavaScript modules in a live application with no need to refresh the browser) into the project? This is a feature of Webpack that can be taken advantage of with relatively little configuration, so it seemed like a good place to start.
Gif courtesy Chris Wheatley
We were already using WebpackDevServer
for development, which watches for file changes and recompiles the output JavaScript “bundles” as a result. In theory, the only thing to do was add the HotModuleReplacement
plugin as part of our development Webpack configuration.
plugins: [
new webpack.HotModuleReplacementPlugin(),
],
webpack.config.js
Additionally, it is generally a good idea to add the NamedModulesPlugin
so you can easily identify which modules are being reloaded.
plugins: [
new webpack.NamedModulesPlugin(),
new webpack.HotModuleReplacementPlugin(),
],
webpack.config.js
And that’s it! On to the next one…or not.
As it turns out, we hadn’t revisited our Webpack configuration in some time and in the ensuing months, the build had grown in size and–more importantly–time. Our baseline recompile time in WebpackDevServer
was a whopping 27 seconds and the HMR client code would time out before the build could complete.
There was clearly more work to do.
Loaders, loaders everywhere
Like any modern front-end project of sufficient complexity, we were using many Webpack “loaders” to take source code of various forms and convert it to browser-compatible assets. For example, in our project we have sass-loader
for styles, babel-loader
for modern JavaScript, and vue-loader
for extracting code from single-file Vue components.
Recognizing that all of these loaders were necessary to create our production assets, we wondered if some could be bypassed in development to decrease the amount of work Webpack had to do to recompile.
The first candidate for cutting out was the postcss-loader
. While the postcss-loader
is powerful and has many uses, we only use it for vendor prefixing certain CSS rules. For most development purposes, this is superfluous. However, bypassing it only led to very modest gains.
That is not to say there weren’t gains to be made here. We use another loader/plugin combination to take styles from a JavaScript bundle and extract them into separate CSS assets to avoid flashes of un-styled content. While this is a necessity in production, the ExtractTextPlugin
is not only unnecessary in development, it also makes hot reloading of styles an impossibility.
Removing the ExtractTextPlugin
in development and allowing styles to reside in the JavaScript bundles took a noticeable chunk out of the build time, but it was still not quite enough.
The low hanging fruit
Webpack configurations are big, complex and as varied as the unique front-end framework combinations that they compile. In all of that complexity, it is easy to miss performance knobs that can be turned with very little effort.
For example, the babel-loader
can be configured with a cacheDirectory
so that future builds do not need to recompile files that haven’t been changed.
module: {
rules: [
{
test: /\.js$/,
exclude: /(node_modules|bower_components|vendor)/,
loader: 'babel-loader?cacheDirectory',
},
...
],
},
webpack.config.js
Another easy configuration tweak that made a big dent in development build performance is the noParse
option. For some well-bundled vendor libraries, it is possible to let Webpack not parse them at all. If some of these libraries are large this could have a big impact on build time.
module: {
noParse: /node_modules\/underscore|node_modules\/moment|node_modules\/jquery/,
...
}
webpack.config.js
Mapping the source
One of the well documented performance tweaks that can be made in a Webpack configuration is the source-mapping devtool
choice. In our build we were using the source-map
tool, which is the most expensive of the lot.
The number of options for devtools can be a bit intimidating, but at the end of the day it comes down to a tradeoff between speed, size, and how much it resembles the actual source code. Initially, we determined the best devtool for our development build was the cheap-module-eval-source-map
.
This devtool creates separate source map files, but does so in a way that is faster than the source-map
tool and relies on JavaScript’s eval
(which disqualifies it from being used in production). Essentially, this is the fastest devtool
that maintains a complete 1:1 mapping to the source code.
However, we realized that our vendor
bundle probably did not need source maps. Digging in, we found that the devtools are just abstractions of another Webpack plugin called the SourceMapDevToolPlugin
. If you are willing to use this plugin directly, it affords more flexibility including the ability to exclude certain bundles from being mapped.
To do this, we disabled the devtool
option and added the SourceMapDevToolPlugin
to our list, configuring it to exclude the vendor bundle:
devtool: false,
plugins: [
new webpack.SourceMapDevToolPlugin({
filename: '[file].map',
exclude: ['vendor'],
columns: false,
module: true,
}),
...
]
webpack.config.js
Here the module
option tells the plugin to preserve a mapping to the original source modules. Setting the columns
option to false
makes this source mapping configuration “cheap” by not mapping exact column numbers.
Plight of the commons
Our application is not organized as a traditional Single Page Application that has a single entry point and loads additional code as needed. Instead, it is made up of many smaller entry points that each operate on different pages.
One drawback to this approach is that every page within the application will have some redundant code. For instance, vendor libraries like Vue will be loaded on nearly every page and thus will be included in every bundle. Even more relevant to the current discussion, if a module appears in many different bundles modifying that module will result in a long recompile time.
Webpack provides a mechanism for reducing this redundancy via the CommonsChunkPlugin
. This plugin identifies modules–either automatically or via configuration–that are used in multiple bundles and combines them into a “commons” bundle that can be loaded separately from entry point bundles.
We chose to identify these common modules manually using a combination of our knowledge about the application and the visual guide offered by the extremely helpful BundleAnalyzerPlugin
. To do so, we first defined an entrypoint
that contained some of the worst offenders:
entry: {
common: [
'vue',
'vuex',
...
]
}
webpack.config.js
We then add the CommonsChunkPlugin
to our list of plugins with the minChunks
option set to Infinity
. This tells the plugin to do none of its own analysis on which modules are reused in multiple bundles.
plugins: [
new webpack.optimize.CommonsChunkPlugin({
name: ['common'],
minChunks: Infinity,
}),
...
]
webpack.config.js
This change was the single most impactful that we made. In fact, we decided to make a number of commons chunks for various page types and continued to see performance gains.
The bottom line
Our main goal in the project was to decrease the time taken to rebuild the project and enable Hot Module Replacement. In that, we were extremely successful. With just the changes listed above a hot reload went from 27 seconds to a screamin’ (relatively) 1.9 seconds!
Not only did we achieve the goal here, but we also had a number of happy side-effects. In our analysis of redundant modules across entry points, we found some places where modules were accidentally being included twice on the same page. Understanding our build better and removing those redundancies reduced asset size across the site and in some cases, dramatically.
This was a huge stride, but there is clearly still room to grow. Have any interesting stories or insights about optimizing Webpack? We’d love to hear them. Or better yet…we’re hiring!