IT Literacy for Non-Developers #4: Bugs, Hotfixes, and Rollbacks — How Developers Respond to Incidents
In the last post, we said that putting a developer’s code onto a server to make it available to users is called deployment. At the end, I left you with one question: what happens if the code you deployed has a problem? The moment a bad change goes out to production, users feel it right away. Payments get blocked, screens break, or the app won’t even open.
This kind of situation is commonly called an incident. And when an incident hits, familiar words start showing up in the dev team’s chat: “We’ve got an incident,” “a hotfix is going out,” “I rolled it back for now.” In this post, we’ll look at how a dev team responds to problems through three words: bug, hotfix, and rollback.
This series, IT Literacy for Non-Developers, consists of five parts.
- #1 What Is a Website Made Of? — Frontend, Backend, and Database
- #2 What Is an API? — The Promise That Lets Services Talk to Each Other
- #3 Servers, the Cloud, and Deployment
- #4 Bugs, Hotfixes, and Rollbacks — How Developers Respond to Incidents ← this post
- #5 Git and Version Control — How Many People Edit One Codebase
A Bug Is When Something Behaves Differently Than Intended #
A bug is when a program behaves differently than the person who made it intended. You pay 1,000 won but 1,100 won is withdrawn, you clearly press a button but it doesn’t respond, or a list comes out in the wrong order - these are all bugs. A small mistake in a single line of code can look like a big problem on the user’s screen.
How the word “bug” came to mean an insect is a fun story in its own right. If you’re curious, I recommend reading Where the Word “Bug” Came From — The Story of the First Computer Bug separately.
The important thing is that not all bugs are the same kind of bug. There are minor bugs, like text that’s a shade off color, and there are serious bugs, like payments failing entirely or personal data being exposed by mistake. So when a dev team finds a bug, the first thing they assess is severity. The response changes completely depending on whether it’s urgent enough to halt the service right now, or something that can be fixed later at the next regular deployment.
A Hotfix Is an Emergency Fix to Put Out the Fire #
A hotfix is, quite literally, an emergency fix that cools down a hot situation. Normally, a new feature is tested and reviewed thoroughly, then deployed on a set schedule. But for a problem that’s hurting users right now, like blocked payments, you can’t wait for that schedule.
In that case, the dev team narrowly fixes only the broken part and deploys it faster than usual. A fix that skips the regular schedule and goes out urgently like this is a hotfix. “A hotfix is going out” means an emergency fix is being deployed to stop an urgent problem right now.
That said, a hotfix isn’t free either. When you fix something in a hurry, it can go out without being tested enough, and that can cause yet another problem. So the principle for a hotfix is to be fast but as narrow as possible, touching only the part that absolutely needs it.
A Rollback Is Reverting to the Previous Working Version #
If a hotfix is a way of moving forward to fix something, a rollback is a way of going backward. If the new version you just deployed caused a problem, you can revert to the last version that was working fine while you slowly track down the cause. This is a rollback.
The advantage of a rollback is speed. Analyzing what went wrong in the new version takes time, but reverting to the version from before the problem appeared is much faster. You stop the harm users are experiencing first, and find the cause calmly afterward. “I rolled it back for now” means the new deployment was pulled and the service was returned to its previous stable state.
Here, a question comes up. We revert to the previous version — but where on earth is that previous version kept? This story leads into Git and version control, which we’ll cover in the next post.
Incident Response Usually Follows This Order #
When an incident hits, a dev team usually moves in this order.
- Detection: A monitoring tool flags an anomaly, or a user report tips you off to the problem.
- Emergency response: Before digging deep into the cause, you stop the harm to users first with a rollback or a hotfix.
- Root cause analysis: Once the fire is out, you calmly find out what went wrong and why.
- Prevention: You reinforce checks or tests so the same problem doesn’t happen again.
In the last step, the dev team often writes a retrospective (postmortem). Rather than assigning blame, it’s a process of laying out what made the problem possible and how to prevent it. The better the team, the more it treats an incident as a chance to learn rather than a chance to blame.
Why Knowing This Makes a Non-Developer’s Job Easier #
Once you understand the flow of incident response, you can move more calmly and accurately the moment a problem hits.
- You can read the situation accurately. When you hear “I rolled it back, and we’re looking into the cause now,” you can understand it to mean the harm to users has stopped for now and analysis is underway.
- You can report bugs well. Along with what went wrong and how, if you also note the severity - whether it’s urgent like a payment issue or minor like a display glitch - the response gets prioritized faster.
- You can understand the answer to “why can’t you fix it right now?” Because a hotfix that goes out wrong can trigger a bigger problem, even under pressure some minimum verification is needed. The goal isn’t speed at all costs — it’s being fast safely.
Wrap-Up #
In this post, we looked at three words for incident situations.
- A bug is when a program behaves differently than intended, and the response varies by severity.
- A hotfix is an emergency fix that skips the regular schedule and narrowly patches a problem to stop something urgent.
- A rollback is an emergency measure that quickly reverts to the previous working version when a new deployment causes a problem.
The fact that a rollback is possible means the previous versions are kept somewhere, neatly stacked up. How many developers edit the same code without losing any versions — in the next post, we’ll look at the final topic, what Git and version control are.