Painless Monorepo Dependency Management with Bit

It is no secret that with the right tooling a monorepo has many advantages over a one project per repository setup. JavaScript tools have come a long way to provide good DX for handling multiple packages in a single repository. I have personally added support for it to pnpm around the same time Yarn has. npm also added support for it last year. All these three JavaScript package managers handle dependencies about the same way in a monorepo. And they come with the same challenges. In this article, I want to highlight these challenges and show how Bit can improve your workflow in a monorepo (aka workspace).

NOTE: In this article, when you see the term "workspace", think about a "set of packages".

In case you prefer to watch rather than read, you can see me talking about Bit and monoreps here:

One of the hardest parts of managing a monorepo is dependency management. In a monorepo, you may have hundreds of packages each with its own package.json file and a set of its prod and dev dependencies.

When you use npm or Yarn Classic to manage your workspace, you will get all the dependencies of all the workspace packages in the root of the monorepo. So, if you have one component using lodash and another one using ramda, you will have this directory structure:

workspace
├── node_modules
│   ├── lodash
│   └── ramda
├── button (dependencies: {lodash: '4.17.21'})
└── card (dependencies: {ramda: '0.28.0'})

Copied

Even though button has only lodash in its dependencies, button has access to both ramda and lodash. Hence, if button imports ramda, the test will pass. But once button is published and installed as a dependency, its code will fail.

There are two ways to prevent packages from using undeclared dependencies.

You may use the import/no-extraneous-dependencies rule of ESLint's import plugin. With this plugin, if button will import any package that is not in its dependencies, you will get an error during linting.

So, if button imports ramda, the linter will fail. You will have to manually install ramda as a dependency of button to fix the issue.

Another solution to this issue is changing how dependencies are installed. Neither pnpm nor Yarn v2 have the phantom dependencies issue (with default configuration).

pnpm uses an isolated directory structure to install dependencies in a workspace:

workspace
├── node_modules
│   └── .pnpm
│       ├── lodash@4.17.21/node_modules/lodash
│       └── ramda@0.28.0/node_modules/ramda
├── button
│   └── node_modules
│       └── lodash --> ../../node_modules/.pnpm/lodash@4.17.0/node_modules/lodash
└── card
    └── node_modules
        └── ramda --> ../../node_modules/.pnpm/ramda@0.28.0/node_modules/ramda

Copied

As you can see, in this case, button has no access to ramda. If button tries to import ramda, the code will fail during local development.

In case of Yarn, it uses Plug'n'Play, which overrides Node's resolution algorithm preventing it from resolving phantom dependencies.

Another issue, which is not specific to monorepos, is using dev dependencies from runtime code. This issue may happen when a dependency was already used in tests and then runtime code also started using it. The project will work perfectly locally but will fail when installed as a dependency. In a non-bit workspace, this can only be solved by linting.

In a workspace, many packages use the same dependency. In this case, the same dependency and its version are duplicated to multiple package.json files throughout the workspace. Guarantying version consistency is hard and usually requires additional tooling. Like syncpack for instance.

Even though Bit uses pnpm or Yarn for installation, it uses them without any package.json files. When you work with a Bit workspace, you don't care which component uses which dependencies or whether a dependency is a dev dependency or a runtime one. All you need to remember is to run bit status and install any missing dependencies as dependencies of the workspace, not as dependencies of individual packages in the workspace.

Let's see it on an example. Create a new directory and run:

bit init
bit create node lib

Copied

Edit the file at my-scope/lib/lib.ts and add an import statement to it importing ramda:

import R from 'ramda';

export function lib() {
  return 'Hello world!';
}

Copied

If you run bit status, you'll get a report about any missing dependencies:

new components
(use "bit tag --all [version]" to lock a version with all your changes)

     > lib ...  issues found
       missing dists (run "bit compile")

       missing packages or links from node_modules to the source (run "bit install" to fix both issues. if it's an external package, make sure it's added as a package dependency):
          lib.composition.tsx -> react
          lib.ts -> ramda

learn more at https://bit.dev/docs/components/adding-components

Copied

You see that react and ramda are missing. React is used in the composition and a reference to ramda you have just added. To install the missing dependencies, you just need to run:

bit install react ramda

Copied

Unlike with npm, pnpm, or Yarn, you don't need to tell Bit which component should the dependency be installed to. You also don't need to tell Bit what the type of the installed dependency is. Is it a prod dependency or is it a dev dependency? Bit knows how to get this information without you telling it.

Now if you run bit show lib, you'll see that ramda is in the dependencies of lib:

Now edit the lib.ts again and remove the ramda import statement:

export function lib() {
  return 'Hello world!';
}

Copied

Your component doesn't use ramda anymore. With npm/pnpm/Yarn you would need to manually remove ramda from the dependencies of lib. Or you would need to run a CLI command to do it. There's also a probability that you would assume ramda is still used in another file, and it would remain in the dependencies of your component. With Bit, you don't need to take any extra steps. Run bit show lib again, and you'll see that ramda is not in the dependencies of your component anymore!

ramda is still in the dependencies of your workspace, and you may remove it if you don't need it anymore. But none of your components will have ramda in their dependencies.

Now edit the lib.spec.ts file and import ramda in that file:

import ramda from 'ramda';
import { lib } from './lib';

it('should return the correct value', () => {
  expect(lib()).toBe('Hello world!');
});

Copied

Your component now depends on ramda again but only in the tests. With npm/pnpm/Yarn you would need to install ramda as a dev dependency in this package. However, with Bit you should do nothing as ramda is already a dependency of the workspace. Run bit show lib, and you'll see that ramda is now listed as a dev dependency of the component.

So, with Bit:

1. you will never forget to change the type of your dependency
2. you will never forget to add a package that you use to the dependencies of your project
3. you will not publish your package with redundant dependencies
4. you will get the same dependency versions in all your components (by default)
5. you will do a lot less micromanagement of dependencies in your workspace

Bit is not another npm alternative. It is a lot more than that. In fact, Bit doesn't implement its own installation algorithm. Instead, it uses either pnpm or Yarn under the hood. However, as you already know, a Bit workspace has no package.json files. So, how do pnpm and Yarn work? Bit dynamically generates the package.json files and passes them directly to the package manager, using its programmatical API. This is handled by the dependency resolver aspect.

By default, pnpm is used as the package manager. If you want to switch to Yarn, change the package manager to Yarn in the workspace.jsonc:

/**
   * main configuration for component dependency resolution.
   **/
  "teambit.dependencies/dependency-resolver": {
    /**
     * choose the package manager for Bit to use. you can choose between 'yarn', 'pnpm'
     */
    "packageManager": "teambit.dependencies/yarn",
    "policy": {
      "dependencies": {},
      "peerDependencies": {}
    }
  },

Copied

Both Yarn and pnpm support multiple node linkers. pnpm uses an "isolated" node linker by default. While Yarn uses a classic "hoisted" linker. In most cases, the default configuration should work fine. But if you want to change the node linker, just specify your preference in the workspace.jsonc:

"teambit.dependencies/dependency-resolver": {
    "packageManager": "teambit.dependencies/pnpm",
    "nodeLinker": "hoisted"
  },

Copied

Most pnpm and Yarn features are supported and may be configured either via settings in workspace.json, .npmrc, or .yarnrc.yml. For instance, here is how to use the dependency override feature:

"teambit.dependencies/dependency-resolver": {
    "packageManager": "teambit.dependencies/pnpm",
    "overrides": {
      "react": "16.0.0"
    }
  },

Copied

This will override any React dependency to v16.0.0. It works with both pnpm and Yarn.

Unlike in a traditional workspace, in a Bit workspace you list all your dependencies in a single place, in the workspace.jsonc file. After you run bit install ramda, this is how the dependencies section in your workspace.jsonc will look like:

"teambit.dependencies/dependency-resolver": {
    "packageManager": "teambit.dependencies/pnpm",
    "policy": {
      "dependencies": {
        "ramda": "0.28.0"
      }
    }
  },

Copied

Sometimes, you might need to have a different version of ramda in some of the packages. In this case, in a traditional workspace you would use other versions of ramda in some of the package.json files. In case of Bit, you may use variants to configure groups of components. For instance, if you need to use ramda@0.27 for any package that is inside the ui directory, use this configuration:

"teambit.dependencies/dependency-resolver": {
    "packageManager": "teambit.dependencies/pnpm",
    "policy": {
      "dependencies": {
        "ramda": "0.28.0"
      }
    }
  },
  "teambit.workspace/variants": {
    "{ui/**}": {
      "teambit.dependencies/dependency-resolver": {
        "policy": {
          "dependencies": {
            "ramda": "0.27.0"
          }
        }
      }
    }
  },

Copied

A Bit workspace automates a lot of the things that you must do manually in a "traditional" JavaScript workspace. Once you and your team get used to Bit, going back to your old setup will feel like going back to the stone age.

This blog post only scratches the surface of what Bit can do to improve your workflow. In our next blog post, we plan to write about how Bit allows creating reusable environments for your components. With environments, you free yourself from the burden of repetitive tooling configuration and peer dependencies management.

Zoltan is the master of dependencies and packages at Bit and the maker of pnpm. He likes Tacos🌮.