Interested in contributing to this effort? Give this post a read and join us in #st-module-unification on the Ember.js Community Slack.
Last fall the Ember.js Module Unification RFC was merged after a final comment period. This RFC describes a change to how Ember applications are laid out on disk. The change is a big one, and has several motivations.
- Simplify the rules for file naming. Ember maps a factory name to a filename using a part of the system called the
resolver. For example
route:carsis mapped to
app/routes/cars.jsin your application because of the Ember-CLI resolver. There are a few resolvers in the Ember ecosystem, however none of them have an internally consistent design. Instead, each is a fairly ad-hoc set of hard-coded solutions for whatever use cases came up when it was written. This make the filesystem of an Ember application more difficult to teach and learn than it needs to be.
- Allow for new features. Existing resolvers were designed without considering two important use-cases:
- They did not take into account the idea of a component providing context for routed templates (aka “routable components”).
- Components and helpers in Ember currently share a single application-wide namespace. The rules of existing resolvers limited how we could change that rule and permit components and helpers scoped to only part of an application (aka “local lookup”).
- Improve how addons are exposed to applications.
- Ember addons currently expose themselves to a consuming application by
app/folder onto the application’s
app/folder. This approach doesn’t allow for very robust namespacing of addon API away from app and other addon API.
- Addons today cannot easily add their own “types” to the Ember filesystem.
For example, ember-cli-mirage stores
mirage/and adds that directory to the build output. A goal of the new design is that addons may define new types that Ember and other addons can know about.
- Ember addons currently expose themselves to a consuming application by merging their
- Cleaner and better performing code. In existing resolvers the large amount of scenario-solving design makes it difficult to create an implementation that minimizes string allocation and branching logic. Additionally, the rules are so complex and ill-defined doing some of this work during a compilation step is not possible. A simpler, unified design friendly to a compilation step would be preferred.
In this year’s EmberConf keynote Tom and Yehuda made passing mention of module unification, but most of the focus was rightly on the Glimmer application project. Glimmer has been a great opportunity to iterate on parts of the module unification story, and where we take module unification next is coupled to how we can share solutions between Ember and Glimmer.
Let’s break down where the project is today and where you can help.
How Glimmer Implements Module Unification
Glimmer and Ember represent solutions for two ends of the complexity spectrum.
The goal is that applications starting with Glimmer can
npm install their way
to Ember, and that Ember applications can likewise have an offramp if they
need to be refactored to simpler use cases.
npm install path to work, we need the foundational APIs and libraries
of the two systems to be the same. The shared Glimmer
VM is the best example of this.
Module unification’s design is another example.
Take a look at the layout on disk of a Glimmer application:
src └── ui ├── components │ ├── conference-speakers │ │ ├── component.ts │ │ └── template.hbs │ └── hello-glimmer │ ├── component.ts │ └── template.hbs ├── styles │ └── app.css └── index.html
If you’ve looked at the module unification RFC, this will look pretty familiar. Glimmer applications are using the same rules defined in module unification, but with a slightly different configuration of those rules.
Glimmer applications don’t use any part of the Ember codebase proper. Instead,
they rely on extractions of the best ideas in Ember. For example
the glimmerjs/glimmer-di repo
a complete rewrite of Ember’s container (the part of Ember managing
dependency injection) in TypeScript. This library, like Ember’s
container, expects to be provided a resolver in order to map
factory names to module identifiers.
The Glimmer application pipeline contains a module unification configuration which is passed to an instance of the glimmerjs/glimmer-resolver. The resolver is then passed to an instance of glimmerjs/glimmer-di to create a complete container system. The Glimmer application pipeline also experiments with pre-compilation of the module map via the glimmerjs/resolution-map-builder compilation step.
Combined, these libraries represent a nearly full implementation of the module unification RFC for Glimmer apps.
However adopting this work into Ember requires jumping a few hurdles:
- The resolver API implemented by glimmerjs/glimmer-resolver isn’t the same as Ember’s own resolver API.
- The Glimmer application story doesn’t yet include addons.
module unification for Ember will also change the location of CSS and HTML
files. The Glimmer applications pipeline was written from scratch and has
yet to be re-unified with Ember’s own build pipeline. For now, this means
changes to the Ember-CLI
EmberAppbuild pipeline used by Ember applications are required.
Using Module Unification in Ember
Despite module unification being incomplete you can try several parts of the story today. Version 4.1.0 of ember-cli/ember-resolver ships with a build-time feature flag which, when enabled, exposes a wrapper around glimmerjs/glimmer-resolver. Through this wrapper and an Ember module unification configuration, you can use module unification in Ember apps.
Please do not confuse these examples with production-ready solutions! Work in the Ember project happens entirely in the open. You are encouraged to play with these solutions and start learning how you can contribute to making them ready for the real world. If you’re concerned about what this very in-the-weeds blog post means for your production app, please read Safety of the Herd. Only when a mature migration path exists for applications will deprecations be added and defaults in the Ember stack be changed.
But for the brave, let’s dive in!
Starting a New Ember Application
This one is easy. Install Ember-CLI 2.14-beta (or newer):
npm install -g email@example.com
Then generate an application using the
ember new my-app -b ember-module-unification-blueprint
This command will create a fresh application using the following configuration:
- The emberjs/ember-module-unification-blueprint
package will define the default layout on disk. This application has a
src/directory instead of an
app/directory, and also has the appropriate dependencies to boot an application using module unification.
- The created application is dependent upon ember-cli/ember-resolver
4.1.0+. This version of the Ember-CLI default resolver exposes a feature flag labelled
EMBER_RESOLVER_MODULE_UNIFICATION. The blueprint enables that feature flag and imports the
GlimmerWrapperresolver for a resolver base class instead of the classic Ember-CLI resolver.
- The application will be dependent on a branch of
Ember-CLI that includes
pending compatibility fixes for applications using
Eventually the default Ember-CLI application blueprint will be updated to be modeled on this one. At this time, there is an open task to define an addon blueprint for module unification, and to port other existing blueprints to module unification. The exact timing of when updated blueprints are released is an open question.
Migrating an Ember Application
Existing Ember applications, for the purposes of module unification, come in four flavors based on their resolver:
- Globals resolver: Applications using the
Ember.DefaultResolver. These applications don’t use modules at all. They are quite rare in the Ember ecosystem at this point. The best example of this style of application is emberjs.jsbin.com.
- Classic resolver: Applications using the ember-cli/ember-resolver. This is the majority of Ember applications.
- Pods resover: Applications using the ember-cli/ember-resolver with pods. The implementation of pods is mixed into the classic resolver code.
- Custom resolver: Applications with a custom Ember resolver based on the
Ember.ResolverAPI. Creating these resolvers is a use of a public Ember API, and we need to support these custom resolvers until 3.0.
The migration strategy for existing applications will follow these steps:
- Migrate files on disk from an old layout to the new layout. For applications
using the “classic” or pods resolver this can be largely automated. Work is
underway in the
repo. This automation works well enough that you can find examples of its
usage in the module unification RFC. There remain many
open issues to get it production-ready. For example module paths that change
must have their
importpaths in files referencing them updated.
- Opt-in to the new resolver. This step will probably remain manual and result
in a resolver file that looks like the one in the
Some applications may also want to use a “fallback resolver” that starts
src/for files then falls back to
app/. Some naive versions of this have been implemented and there is an issue tracking adding a fallback resolver to ship with ember-resolver.
Aligning the Glimmer & Ember DI systems
GlimmerWrapper resolver shipped by ember-cli/ember-resolver can be good enough
for production and certainly for development apps. Making the module unification
story work well on that foundation is an important step forward.
But it isn’t perfect.
GlimmerWrapper is exactly what it purports to be: A class that
implements the current
Ember.Resolver API but uses the
glimmerjs/glimmer-resolver implementation internally.
The ideal end state for the Glimmer/Ember ecosystems
are that they share implementation where the design is common. Basic primitives
of a web app framework like the rendering engine and dependency injection
implementation are exactly where this alignment should occur.
Just as Ember’s resolver API and existing implementations are considered under-designed, the dependency injection system in Ember (commonly the “container”) is considered less than perfect. In glimmerjs/glimmer-di, the Ember project has already invested in a new DI implementation written with TypeScript and in a new resolver API between that library and glimmerjs/glimmer-resolver.
In parallel to the effort to get module unification working via the
API, and in parallel to the work on the automated migration path, we’ve
also started work toward Ember adopting
glimmerjs/glimmer-di to replace its own container implementation. When this
work lands, this will also mean a phasing out the current Ember resolver API.
This work will eventually fully align the Glimmer and Ember DI/resolver systems. The effort shouldn’t be considered a blocker for using module unification layouts, but it is a prerequisite for Ember to deliver on its goals of simplicity, Glimmer/Ember alignment, and performance.
Some of the tactical steps we need to handle to get Glimmer-DI into Ember:
- Ember’s current test suite is largely coupled to the
Ember.DefaultResolver“globals” resolver. In #15058 there is an effort to refactor tests to use a resolver stub instead of the classic resolver.
- Once the test suite can be run without requiring the classic resolver, a feature flag can be added to Ember swapping the current container with Glimmer-DI. There has already been some work showing this is viable, however landing this feature flag is blocked on the testing refactor.
- The Glimmer-DI project has not considered parts of Ember like the Ember Inspector. As the library is integrated in Ember, those design issues will need to be handled. At the same time, we should ensure the performance of Glimmer-DI does not regress.
- After the Glimmer-DI refactoring lands behind a feature flag, a deprecation and upgrade path still needs to be paved for changing the Ember resolver API.
- Supporting the current
Ember.ResolverAPI after adopting Glimmer-DI as Ember’s container system may be untenable. Without learning more, my current suggestion is that both systems ship in Ember and we rely on Ember’s svelte project to strip out the un-used and deprecated container code supporting the
Ember.ResolverAPI when it is unused.
Open Questions & How to Help
We’re tracking work in these two quest issues:
- Module Unification Epic (ember-cli/ember-cli)
- Quest: Module Unification (emberjs/ember.js)
And additionally you can find people actively working on module unification in the #st-module-unification chatroom on the Ember.js Community Slack.
To get a bit more concrete: A lot of the work on a first phase of module unification support via the
GlimmerWrapper is already complete. You can help get it across the finish
- Pick up an issue from the rwjblue/ember-module-migrator issues list. This tool will help move existing apps from their classic or pods layout to the module unification layout. Use the migrator on your own codebase, identify issues, and open PRs with fixes.
- Try the
GlimmerWrapperin a new app or in a ported app, and help identify issues and squash bugs on the ember-cli/ember-resolver issues. Known issues include resolving module without a default export, implementing a fallback resolver, and using babel-plugin-minify-dead-code-elimination to ensure builds of ember-cli/ember-resolver that don’t use the feature flag don’t include glimmerjs/glimmer-di in their build output.
There are some trickier open issues being explored as well.
- The RFC makes suggestions for how component namespaces work in module unification applications. Implementing that part of the RFC has not been tackled yet, and unknown issues may remain.
- Additionally, with addons, it would be ideal for module unification apps to
still consume legacy addons. Module unification apps do not have an
app/folder for those addons to merge into, so the exact path forward is unknown.
To land phase two of the work, the replacement of Ember’s DI system with Glimmer-DI and the deprecation of the current Ember resolver API, a few additional hurdles need to be vaulted:
- [EPIC] Decouple Ember tests from the globals resolver (emberjs/ember.js)
- “Project Svelte” (stripping of deprecated code when it is un-used) probably needs to land before we can ship the Glimmer-DI container replacement to release. Without svelte Ember would ship two DI systems in production builds which is unacceptable. Some of this work is tracked in emberjs/ember.js#15062.
Module Unification FAQs
Last Friday I asked if there where any questions about module unification on Twitter. I got a number of replies and not all of the questions were answered in this post. So here we go!
I’m going to write a blog post on the status of Ember’s module unification project today. What do you want to know?— mixonic (@mixonic) May 26, 2017
@mixonic Can I write an addon with the new layout and expect it to work with older Ember CLI versions?— Marten Schilstra (@Martndemus) May 27, 2017
There are a lot of directions this question could head. The answer I would give is “no”, an addon written in the module unification style will not Just Work with an older version of Ember-CLI. There will be some minimum version of Ember-CLI required for an application to consume module unification addons.
However since Ember-CLI addons can control the build of an application, it seems
possible that they could ship with their own compatibility layer that exposes an
app/ tree in the traditional addon style.
This part of the story just isn’t fleshed out enough to give a very good answer. It is in the best interest of the Ember project to have as much backwards compatibility between apps and addons during the community transition to module unification.
@mixonic what is the path to code splitting by route/feature?— Mitch Lloyd (@too_mitch) May 27, 2017
Code-splitting in Ember is currently available via Ember
Engines, however I think @too_mitch is talking about
something different here. When you visit a given route for an Ember app, in
theory the server could ship a customized build of assets just for that URL.
For example if you visit
/videos you might automatically disregard the
This kind of automatic code-splitting relies more on static analysis of Ember templates and ES Modules than on module unification, however the consistency of module unification rules might make the implementation a bit easier. I’m not familiar with any specific initiatives to make automatic code-splitting happen at this time, only discussions.
@mixonic what’s the story on migrating from the existing structure to the new one? How will it work for people using a mix of pods/not pods?— Alex LaFroscia (@alexlafroscia) May 26, 2017
The rwjblue/ember-module-migrator tool already supports migration from pods to module unification layout.
For what its worth, I teach “classic” filesystem layout to new Ember developers myself and will continue to teach it until the module unification story ships in release. I know classic apps will have a smooth migration path.
@mixonic Other than upgrading our apps are there any clear ways the community can help rolling this out? Are there open issues on tooling,guides?etc— Ryan LaBouve (@RyanLaBouve) May 26, 2017
A common thread on the whole Twitter discussion was “how can I help”. Thank you! I hope a few opportunities have been highlighted above and please feel free to join us in #st-module-unification on the Ember.js Community Slack.
One of the many ways to support the module unification effort is by sponsoring OSS development through 201 Created. We believe in creating stable and progressive platforms to keep your business innovating. Email us at firstname.lastname@example.org.