Practical Git Workflows #7: Monorepos and Git — sparse-checkout, Submodules, and LFS

The six parts so far applied regardless of repository size. Past a certain scale, however, a new class of problems appears. In a monorepo that gathers several projects into one repository, a clone starts taking tens of minutes, and large binaries such as design source files or model files keep inflating the repository. These problems are not solved by branching strategies or commit cleanup — they need the tools Git provides specifically for them.

It runs in seven parts.

• #1 Branching strategies — GitHub Flow and trunk-based
• #2 rebase vs merge — decision criteria and the golden rule
• #3 Interactive rebase — cleaning up commits with squash and fixup
• #4 Resolving conflicts — structure, mergetool, and rerere
• #5 Running Pull Requests — review size, commit messages, and draft PRs
• #6 stash, cherry-pick, bisect — everyday tools
• #7 Monorepos and Git — sparse-checkout, submodules, and LFS ← this post

This post is not a deep tutorial on each tool. The goal is a map of which tools exist and when to reach for them. We will start from the symptom and match a tool to each one.

What slows down as a repository grows #

As we confirmed in the Git Basics series, a clone does not fetch just the latest files — it fetches the entire history from the first commit to now. While the repository is small, you only see the upsides of this model: everything works without a network, and every commit can be inspected locally. As the repository grows, the same model becomes a cost.

• Clone time and disk — you download ten years of history, tens of thousands of files, and every version of every binary ever committed.
• Everyday command speed — git status has to scan the files in the working directory, so it slows down in proportion to the number of checked-out files. In a monorepo with 500,000 files, a single status starts taking seconds.

In short, the problem has two directions: how much you download (the full history and blobs) and how much you spread out (the number of files in the working directory). The tool that reduces the former is partial clone; the tool that reduces the latter is sparse-checkout.

Partial clone — all of the history, content on demand #

A partial clone fetches the entire commit graph but defers file contents (blobs) until they are actually needed.

git clone --filter=blob:none https://github.com/example/big-monorepo.git

--filter=blob:none means: fetch all commits and directory structures, but skip the blobs. At checkout time only the latest versions of the blobs are downloaded, and commands that need past versions (git log -p, git diff, and so on) fetch them from the server as they run. History inspection occasionally touches the network, and in exchange the first clone becomes dramatically lighter.

A similar-looking option is the shallow clone.

git clone --depth 1 https://github.com/example/big-monorepo.git

--depth 1 fetches only the single latest commit and cuts off the history entirely. It fits a CI environment that builds once and throws the checkout away, but with no history, log inspection and bisect are limited and later fetch behavior picks up wrinkles. Split the use cases: partial clone for development machines, shallow clone for one-shot CI.

sparse-checkout — spread out only what you work on #

If the only part of the monorepo you touch is services/web, there is no reason to keep hundreds of thousands of other files spread out in your working directory. sparse-checkout narrows the checkout to the directories you specify.

git sparse-checkout init --cone
git sparse-checkout set services/web shared/ui

The working directory now contains only the top-level files plus services/web and shared/ui. The history is still all there, so log inspection and branch work are unrestricted, and git status scans fewer files, making everyday commands fast again. When your scope of work changes, run git sparse-checkout set again to adjust.

Submodules — referencing another repository at a pinned commit #

Now for a different direction: not one repository growing, but tying several repositories together. A submodule references another Git repository at a subpath of your repository, pinned to a specific commit.

git submodule add https://github.com/example/vendor-lib.git vendor/lib
git commit -m "Add vendor-lib submodule"

What gets recorded in your repository is not the files of vendor-lib but a URL and a single commit hash. The reference is pinned to an exact version, so submodules fit situations where an external dependency must be locked to a specific version in source form. An external SDK referenced by firmware, or a third-party library you maintain as a fork, are the typical examples.

That said, submodules carry a reputation, and the reasons are mostly these three.

• If you clone without git clone --recurse-submodules, the submodule directory is left empty. Someone on the team forgets this option again and again.
• Updating a submodule to a new version is a separate procedure: check out the desired commit inside the submodule, then commit the changed reference in the outer repository.
• The inside of a submodule sits in a detached HEAD state by default, so a casual commit made inside creates a commit that belongs to no branch.

The practical rule of thumb is simple. If the dependency can be handled by a package manager (npm, pip, Go modules), that comes first; if several projects always change together, a monorepo is a better fit. Submodules remain the option for the case where pinning source at an exact version is a hard requirement.

Git LFS — moving large binaries out #

The last problem is file size. Git is optimized for text source code, so committing a several-hundred-MB design source file (PSD) or a trained model file makes the repository heavy fast. A binary is effectively stored whole for every version, and every version ever committed stays in history, permanently adding to clone time.

Git LFS (Large File Storage) commits a small pointer file to the repository instead of the large file itself, and uploads the actual content to separate storage.

git lfs install                # install the LFS hooks in the repo (once)
git lfs track "*.psd"          # mark PSD files as LFS-managed
git add .gitattributes         # commit the tracking rule itself
git add design/main.psd
git commit -m "Add main design file"

The .gitattributes file that git lfs track creates is the actual tracking rule, so always commit it together. From then on, pointers are swapped for the real files at checkout, so day-to-day use looks almost the same as regular files. The caveat is hosting cost: GitHub’s free allowance for LFS storage and bandwidth is small, so a team with many large assets should check plans or external storage integration in advance.

Choosing a tool by symptom #

Folding this post’s map into one table.

The series in review #

One line for each of the seven parts.

• #1 A branching strategy is the team’s decision between GitHub Flow and trunk-based, matched to team size and deployment cadence.
• #2 rebase and merge are a matter of use, not superiority — and rebasing a shared branch is the one taboo.
• #3 Interactive rebase shapes commits into review-friendly units with squash and fixup.
• #4 Understand conflicts through the 3-way structure, and cut their cost with mergetool and rerere.
• #5 Keep PRs small, and put the intent of a change into commit messages and descriptions — that is what decides review quality.
• #6 stash, cherry-pick, and bisect are the dedicated tools for context switching, selective commit transfer, and regression hunting.
• #7 When the repository grows, manage how much you download and spread out with partial clone, sparse-checkout, submodules, and LFS.

Wrapping up #

The Git Basics series built the foundations — the snapshot model and the three areas, branching and remotes, undoing changes — and this series extended them to the real situations of team collaboration: branching strategies, history cleanup, conflicts, PR operations, everyday tools, and large-repository handling. With these two series, you can judge for yourself most of the Git situations that day-to-day work brings.

What remains is what happens after an accident: restoring deleted commits with reflog, and rewriting history when a secret key or a huge file has been committed. Those recovery topics are coming as standalone, search-friendly posts. This concludes the Practical Git Workflows series.