Ember.js Module Unification Update & Contribution Tips

Wow! This takes me back! Please check the date this post was authored, as it may no longer be relevant in a modern context.

Interested in contributing to this effort? Give this post a read and join us in #st-module-unification on the Ember.js Community Slack.

Last fall the Ember.js Module Unification RFC was merged after a final comment period. This RFC describes a change to how Ember applications are laid out on disk. The change is a big one, and has several motivations.

  • Simplify the rules for file naming. Ember maps a factory name to a filename using a part of the system called the resolver. For example route:cars is mapped to app/routes/cars.js in your application because of the Ember-CLI resolver. There are a few resolvers in the Ember ecosystem, however none of them have an internally consistent design. Instead, each is a fairly ad-hoc set of hard-coded solutions for whatever use cases came up when it was written. This make the filesystem of an Ember application more difficult to teach and learn than it needs to be.
  • Allow for new features. Existing resolvers were designed without considering two important use-cases:
    • They did not take into account the idea of a component providing context for routed templates (aka “routable components”).
    • Components and helpers in Ember currently share a single application-wide namespace. The rules of existing resolvers limited how we could change that rule and permit components and helpers scoped to only part of an application (aka “local lookup”).
  • Improve how addons are exposed to applications.
    • Ember addons currently expose themselves to a consuming application by merging their app/ folder onto the application’s app/ folder. This approach doesn’t allow for very robust namespacing of addon API away from app and other addon API.
    • Addons today cannot easily add their own “types” to the Ember filesystem. For example, ember-cli-mirage stores files in mirage/ and adds that directory to the build output. A goal of the new design is that addons may define new types that Ember and other addons can know about.
  • Cleaner and better performing code. In existing resolvers the large amount of scenario-solving design makes it difficult to create an implementation that minimizes string allocation and branching logic. Additionally, the rules are so complex and ill-defined doing some of this work during a compilation step is not possible. A simpler, unified design friendly to a compilation step would be preferred.

In this year’s EmberConf keynote Tom and Yehuda made passing mention of module unification, but most of the focus was rightly on the Glimmer application project. Glimmer has been a great opportunity to iterate on parts of the module unification story, and where we take module unification next is coupled to how we can share solutions between Ember and Glimmer.

Let’s break down where the project is today and where you can help.

How Glimmer Implements Module Unification

Glimmer and Ember represent solutions for two ends of the complexity spectrum. The goal is that applications starting with Glimmer can npm install their way to Ember, and that Ember applications can likewise have an offramp if they need to be refactored to simpler use cases.

For that npm install path to work, we need the foundational APIs and libraries of the two systems to be the same. The shared Glimmer VM is the best example of this. Module unification’s design is another example.

Take a look at the layout on disk of a Glimmer application:

src
└── ui
    ├── components
    │   ├── conference-speakers
    │   │   ├── component.ts
    │   │   └── template.hbs
    │   └── hello-glimmer
    │       ├── component.ts
    │       └── template.hbs
    ├── styles
    │   └── app.css
    └── index.html

If you’ve looked at the module unification RFC, this will look pretty familiar. Glimmer applications are using the same rules defined in module unification, but with a slightly different configuration of those rules.

Glimmer applications don’t use any part of the Ember codebase proper. Instead, they rely on extractions of the best ideas in Ember. For example the glimmerjs/glimmer-di repo contains a complete rewrite of Ember’s container (the part of Ember managing dependency injection) in TypeScript. This library, like Ember’s container, expects to be provided a resolver in order to map type:name factory names to module identifiers.

The Glimmer application pipeline contains a module unification configuration which is passed to an instance of the glimmerjs/glimmer-resolver. The resolver is then passed to an instance of glimmerjs/glimmer-di to create a complete container system. The Glimmer application pipeline also experiments with pre-compilation of the module map via the glimmerjs/resolution-map-builder compilation step.

Combined, these libraries represent a nearly full implementation of the module unification RFC for Glimmer apps.

However adopting this work into Ember requires jumping a few hurdles:

  • The resolver API implemented by glimmerjs/glimmer-resolver isn’t the same as Ember’s own resolver API.
  • The Glimmer application story doesn’t yet include addons.
  • In addition to changing where JavaScript and template files are on disk, module unification for Ember will also change the location of CSS and HTML files. The Glimmer applications pipeline was written from scratch and has yet to be re-unified with Ember’s own build pipeline. For now, this means changes to the Ember-CLI EmberApp build pipeline used by Ember applications are required.

Using Module Unification in Ember

Despite module unification being incomplete you can try several parts of the story today. Version 4.1.0 of ember-cli/ember-resolver ships with a build-time feature flag which, when enabled, exposes a wrapper around glimmerjs/glimmer-resolver. Through this wrapper and an Ember module unification configuration, you can use module unification in Ember apps.

Please do not confuse these examples with production-ready solutions! Work in the Ember project happens entirely in the open. You are encouraged to play with these solutions and start learning how you can contribute to making them ready for the real world. If you’re concerned about what this very in-the-weeds blog post means for your production app, please read Safety of the Herd. Only when a mature migration path exists for applications will deprecations be added and defaults in the Ember stack be changed.

But for the brave, let’s dive in!

Starting a New Ember Application

This one is easy. Install Ember-CLI 2.14-beta (or newer):

npm install -g ember-cli@2.14.0-beta.2

Then generate an application using the ember-module-unification-blueprint package:

ember new my-app -b ember-module-unification-blueprint

This command will create a fresh application using the following configuration:

Eventually the default Ember-CLI application blueprint will be updated to be modeled on this one. At this time, there is an open task to define an addon blueprint for module unification, and to port other existing blueprints to module unification. The exact timing of when updated blueprints are released is an open question.

Migrating an Ember Application

Existing Ember applications, for the purposes of module unification, come in four flavors based on their resolver:

  • Globals resolver: Applications using the Ember.DefaultResolver. These applications don’t use modules at all. They are quite rare in the Ember ecosystem at this point. The best example of this style of application is emberjs.jsbin.com.
  • Classic resolver: Applications using the ember-cli/ember-resolver. This is the majority of Ember applications.
  • Pods resover: Applications using the ember-cli/ember-resolver with pods. The implementation of pods is mixed into the classic resolver code.
  • Custom resolver: Applications with a custom Ember resolver based on the Ember.Resolver API. Creating these resolvers is a use of a public Ember API, and we need to support these custom resolvers until 3.0.

The migration strategy for existing applications will follow these steps:

  1. Migrate files on disk from an old layout to the new layout. For applications using the “classic” or pods resolver this can be largely automated. Work is underway in the rwjblue/ember-module-migrator repo. This automation works well enough that you can find examples of its usage in the module unification RFC. There remain many open issues to get it production-ready. For example module paths that change must have their import paths in files referencing them updated.
  2. Opt-in to the new resolver. This step will probably remain manual and result in a resolver file that looks like the one in the emberjs/ember-module-unification-blueprint blueprint. Some applications may also want to use a “fallback resolver” that starts by checking src/ for files then falls back to app/. Some naive versions of this have been implemented and there is an issue tracking adding a fallback resolver to ship with ember-resolver.

Aligning the Glimmer & Ember DI systems

The GlimmerWrapper resolver shipped by ember-cli/ember-resolver can be good enough for production and certainly for development apps. Making the module unification story work well on that foundation is an important step forward.

But it isn’t perfect. The GlimmerWrapper is exactly what it purports to be: A class that implements the current Ember.Resolver API but uses the glimmerjs/glimmer-resolver implementation internally. The ideal end state for the Glimmer/Ember ecosystems are that they share implementation where the design is common. Basic primitives of a web app framework like the rendering engine and dependency injection implementation are exactly where this alignment should occur.

Just as Ember’s resolver API and existing implementations are considered under-designed, the dependency injection system in Ember (commonly the “container”) is considered less than perfect. In glimmerjs/glimmer-di, the Ember project has already invested in a new DI implementation written with TypeScript and in a new resolver API between that library and glimmerjs/glimmer-resolver.

In parallel to the effort to get module unification working via the Ember.Resolver API, and in parallel to the work on the automated migration path, we’ve also started work toward Ember adopting glimmerjs/glimmer-di to replace its own container implementation. When this work lands, this will also mean a phasing out the current Ember resolver API.

This work will eventually fully align the Glimmer and Ember DI/resolver systems. The effort shouldn’t be considered a blocker for using module unification layouts, but it is a prerequisite for Ember to deliver on its goals of simplicity, Glimmer/Ember alignment, and performance.

Some of the tactical steps we need to handle to get Glimmer-DI into Ember:

  • Ember’s current test suite is largely coupled to the Ember.DefaultResolver “globals” resolver. In #15058 there is an effort to refactor tests to use a resolver stub instead of the classic resolver.
  • Once the test suite can be run without requiring the classic resolver, a feature flag can be added to Ember swapping the current container with Glimmer-DI. There has already been some work showing this is viable, however landing this feature flag is blocked on the testing refactor.
  • The Glimmer-DI project has not considered parts of Ember like the Ember Inspector. As the library is integrated in Ember, those design issues will need to be handled. At the same time, we should ensure the performance of Glimmer-DI does not regress.
  • After the Glimmer-DI refactoring lands behind a feature flag, a deprecation and upgrade path still needs to be paved for changing the Ember resolver API.
  • Supporting the current Ember.Resolver API after adopting Glimmer-DI as Ember’s container system may be untenable. Without learning more, my current suggestion is that both systems ship in Ember and we rely on Ember’s svelte project to strip out the un-used and deprecated container code supporting the Ember.Resolver API when it is unused.

Open Questions & How to Help

We’re tracking work in these two quest issues:

And additionally you can find people actively working on module unification in the #st-module-unification chatroom on the Ember.js Community Slack.

To get a bit more concrete: A lot of the work on a first phase of module unification support via the GlimmerWrapper is already complete. You can help get it across the finish line!

There are some trickier open issues being explored as well.

  • The RFC makes suggestions for how component namespaces work in module unification applications. Implementing that part of the RFC has not been tackled yet, and unknown issues may remain.
  • Additionally, with addons, it would be ideal for module unification apps to still consume legacy addons. Module unification apps do not have an app/ folder for those addons to merge into, so the exact path forward is unknown.

To land phase two of the work, the replacement of Ember’s DI system with Glimmer-DI and the deprecation of the current Ember resolver API, a few additional hurdles need to be vaulted:

  • [EPIC] Decouple Ember tests from the globals resolver (emberjs/ember.js)
  • “Project Svelte” (stripping of deprecated code when it is un-used) probably needs to land before we can ship the Glimmer-DI container replacement to release. Without svelte Ember would ship two DI systems in production builds which is unacceptable. Some of this work is tracked in emberjs/ember.js#15062.

Module Unification FAQs

Last Friday I asked if there where any questions about module unification on Twitter. I got a number of replies and not all of the questions were answered in this post. So here we go!

There are a lot of directions this question could head. The answer I would give is “no”, an addon written in the module unification style will not Just Work with an older version of Ember-CLI. There will be some minimum version of Ember-CLI required for an application to consume module unification addons.

However since Ember-CLI addons can control the build of an application, it seems possible that they could ship with their own compatibility layer that exposes an app/ tree in the traditional addon style.

This part of the story just isn’t fleshed out enough to give a very good answer. It is in the best interest of the Ember project to have as much backwards compatibility between apps and addons during the community transition to module unification.

Code-splitting in Ember is currently available via Ember Engines, however I think @too_mitch is talking about something different here. When you visit a given route for an Ember app, in theory the server could ship a customized build of assets just for that URL. For example if you visit /videos you might automatically disregard the templates for /categories.

This kind of automatic code-splitting relies more on static analysis of Ember templates and ES Modules than on module unification, however the consistency of module unification rules might make the implementation a bit easier. I’m not familiar with any specific initiatives to make automatic code-splitting happen at this time, only discussions.

The rwjblue/ember-module-migrator tool already supports migration from pods to module unification layout.

For what its worth, I teach “classic” filesystem layout to new Ember developers myself and will continue to teach it until the module unification story ships in release. I know classic apps will have a smooth migration path.

A common thread on the whole Twitter discussion was “how can I help”. Thank you! I hope a few opportunities have been highlighted above and please feel free to join us in #st-module-unification on the Ember.js Community Slack.

One of the many ways to support the module unification effort is by sponsoring OSS development through 201 Created. We believe in creating stable and progressive platforms to keep your business innovating. Email us at hello@201-created.com.