Rust in Practice #3 File IO and Parsing: Read with BufReader, Turn a Log Line into a Struct

Programming Language Rust CLI
5 min read

With the skeleton (Part 1) and the error paths (Part 2) in place, we now build the tool’s heart: reading and interpreting the widely used nginx/Apache combined log format.

access.log 
203.0.113.9 - - [23/Aug/2026:10:12:01 +0900] "GET /api/users HTTP/1.1" 200 512

Reading strategy: load it whole, or stream it #

There are two broad ways to read a file.

src/main.rs 
// Strategy 1: whole file into memory
let text = std::fs::read_to_string(path)?;

// Strategy 2: line-by-line streaming
use std::io::{BufRead, BufReader};
let reader = BufReader::new(File::open(path)?);
for line in reader.lines() {
    let line = line?;
    // process one line
}

read_to_string is simple but uses as much memory as the file is large — a 10 GB log means 10 GB. BufReader streams line by line through a fixed-size buffer, so memory stays constant regardless of file size. loglens defaults to streaming (Part 5’s parallelism will give us a reason to return to strategy 1, and we compare them again there).

The reason to interpose BufReader is system call cost. Reading a File directly issues an OS call per read request, which is fatal in fine-grained patterns like line-by-line processing. BufReader pre-reads several kilobytes into a buffer and serves line requests from it. Wrapping with BufReader::new when reading a file by lines is a fundamental that requires no deliberation.

LogEntry: the destination of a line #

src/parser.rs 
#[derive(Debug, PartialEq)]
pub struct LogEntry {
    pub ip: String,
    pub method: String,
    pub path: String,
    pub status: u16,
    pub bytes: u64,
}

This is the struct the parser produces. #[derive(Debug, PartialEq)] is the standing prescription from basics Part 8, and PartialEq is preparation for assert_eq! in Part 6’s tests. The timestamp field is omitted for now since this series’ analyses don’t use it — when it becomes necessary, the match expressions will point out every place to update, as compile errors.

The parser: from &str to LogEntry #

src/parser.rs 
pub fn parse_line(line: &str) -> Result<LogEntry, ParseError> {
    if line.trim().is_empty() {
        return Err(ParseError::Empty);
    }

    // split on quotes to isolate the "GET /api/users HTTP/1.1" section
    let mut quoted = line.splitn(3, '"');
    let before = quoted.next().unwrap_or("");
    let request = quoted.next().ok_or(ParseError::TooFewFields { found: 0 })?;
    let after = quoted.next().ok_or(ParseError::TooFewFields { found: 0 })?;

    let ip = before
        .split_whitespace()
        .next()
        .ok_or(ParseError::TooFewFields { found: 0 })?;

    let mut req = request.split_whitespace();
    let method = req.next().ok_or(ParseError::TooFewFields { found: 1 })?;
    let path = req.next().ok_or(ParseError::TooFewFields { found: 2 })?;

    let mut tail = after.split_whitespace();
    let status: u16 = tail
        .next()
        .ok_or(ParseError::TooFewFields { found: 3 })?
        .parse()?; // ParseIntError → ParseError::BadStatus (#[from])
    let bytes: u64 = tail.next().and_then(|s| s.parse().ok()).unwrap_or(0);

    Ok(LogEntry {
        ip: ip.to_string(),
        method: method.to_string(),
        path: path.to_string(),
        status,
        bytes,
    })
}

A short function, but dense with callbacks to the basics course: the input is &str (the Part 4 convention), the return is Result<LogEntry, ParseError> (Part 2’s design), ok_or is the bridge from Option to Result (Parts 5–6), and the ? on status.parse() converts ParseIntError into ParseError::BadStatus thanks to #[from] (Part 2). Splitting on quotes first is necessary because the request line ("GET /api ...") contains spaces, so a naive split_whitespace over the whole line cannot cut safely. Lines whose bytes field is - (responses without a body) are accepted leniently as 0. The parser decides where to be strict and where to be lenient, and that decision is documented in the error type — that is the core of this design.

Wiring it up: the errors command, finished #

With a parser in hand, one of Part 1’s todo!()s becomes a real implementation.

src/main.rs 
fn cmd_errors(path: &Path) -> anyhow::Result<()> {
    let reader = BufReader::new(open_log(path)?); // Part 2's open, context included
    for line in reader.lines() {
        let line = line?;
        if let Ok(entry) = parse_line(&line) {
            if entry.status >= 500 {
                println!("{line}");
            }
        }
    }
    Ok(())
}

Lines that fail to parse pass by quietly (if let Ok) — when extracting 5xx lines is the goal, broken lines are out of scope. Next part’s stats, by contrast, must also report “how many lines couldn’t be read,” so it counts failures. Same parser, different treatment of failure per command: that is the flexibility of a parser that returns Result.

Extension: accepting JSON Lines logs #

Many servers now log in JSON Lines (one JSON object per line). With serde, this support is startlingly short.

Install 
cargo add serde --features derive
cargo add serde_json
src/parser.rs 
#[derive(Debug, PartialEq, serde::Deserialize)]
pub struct LogEntry { /* same fields */ }

pub fn parse_json_line(line: &str) -> Result<LogEntry, serde_json::Error> {
    serde_json::from_str(line)
}

Adding Deserialize to the derive list is essentially all of it; serde performs the field name and type validation. The handwritten parser and the derived one produce the same LogEntry, so the aggregation code that follows never needs to know which format the log was in.

Summary #

  • Large files default to BufReader streaming: memory becomes independent of file size, and buffering absorbs the system call cost.
  • The parser takes &str and returns Result<LogEntry, ParseError>. Because the request line contains spaces, splitting on quotes comes first.
  • Where to be strict (status codes) and where lenient (- bytes) is a parser design decision, documented by the error type.
  • The same parser, different failure policies per command: errors skips, stats counts. Returning Result is the source of that flexibility.
  • JSON Lines support is one Deserialize derive. Different formats, same destination (LogEntry) — the downstream code doesn’t change.
  • Next part stacks analysis on top of this parser: completing stats and top with iterator chains, and measuring how much the release build changes performance.
X