Rust in Practice #4 Iterators at Work: Aggregation, Top-N, and the Release Build Difference

Programming Language Rust CLI
4 min read

Part 3 turned one line into a LogEntry; this part turns millions of lines into statistics. It is where the iterator chains and HashMap from basics Part 7 report for duty — and at the end, for the first time in this series, we pull out a stopwatch.

stats: aggregation that counts failures too #

First, the requirements for stats: total lines, parse successes, parse failures, and the status code distribution. Unlike errors (Part 3), failures must be counted rather than discarded, so we build an aggregation struct and feed lines into it.

src/main.rs 
use std::collections::HashMap;

#[derive(Default)]
struct Stats {
    total: u64,
    parsed: u64,
    failed: u64,
    by_status: HashMap<u16, u64>,
    bytes_sum: u64,
}

impl Stats {
    fn feed(&mut self, line: &str) {
        self.total += 1;
        match parse_line(line) {
            Ok(entry) => {
                self.parsed += 1;
                *self.by_status.entry(entry.status).or_insert(0) += 1;
                self.bytes_sum += entry.bytes;
            }
            Err(ParseError::Empty) => {} // empty lines don't count as failures
            Err(_) => self.failed += 1,
        }
    }
}

Part 2’s requirement — “empty lines and malformed lines deserve different treatment” — becomes real here. Because ParseError is an enum, match lets us declare a policy per kind: empty lines land in total but stay out of failed. With anyhow’s opaque errors, this branch would have been possible only through string comparison, the worst of all methods. #[derive(Default)] generates a Stats::default() that initializes every field to zero and an empty map.

The command body is short.

src/main.rs 
fn cmd_stats(path: &Path) -> anyhow::Result<()> {
    let reader = BufReader::new(open_log(path)?);
    let mut stats = Stats::default();
    for line in reader.lines() {
        stats.feed(&line?);
    }

    println!("total lines:  {}", stats.total);
    println!("parsed ok:    {}", stats.parsed);
    println!("parse failed: {}", stats.failed);
    println!("bytes total:  {}", stats.bytes_sum);

    let mut statuses: Vec<_> = stats.by_status.iter().collect();
    statuses.sort();
    for (status, count) in statuses {
        println!("  {status}: {count}");
    }
    Ok(())
}

The two lines before the output loop are a practical idiom. A HashMap has no order, so for human display we collect into a Vec and sort. Output whose order changes between runs erodes trust in a tool — and destabilizes the tests we write in Part 6.

top: top-N by sorting #

src/main.rs 
fn cmd_top(path: &Path, count: usize) -> anyhow::Result<()> {
    let reader = BufReader::new(open_log(path)?);

    let mut by_path: HashMap<String, u64> = HashMap::new();
    for line in reader.lines() {
        if let Ok(entry) = parse_line(&line?) {
            *by_path.entry(entry.path).or_insert(0) += 1;
        }
    }

    let mut ranked: Vec<_> = by_path.into_iter().collect();
    ranked.sort_by(|a, b| b.1.cmp(&a.1)); // descending by hit count
    for (path, hits) in ranked.into_iter().take(count) {
        println!("{hits:>8}  {path}");
    }
    Ok(())
}

The aggregation is the same entry idiom as stats; what’s new is the closing three lines. into_iter() takes ownership of the map (the basics Part 7 distinction) and collects into a Vec, sort_by gets a descending comparison, and take(count) cuts the top N. The --count value clap parsed in Part 1 plugs straight into take — declaration meets logic. Sorting is plenty for a few million paths; beyond that, switching to a heap (BinaryHeap) is the option worth remembering.

The stopwatch: the distance between debug and release #

The tool works, so let’s time it. The standard library is enough for instrumentation.

src/main.rs 
let start = std::time::Instant::now();
// ... run the aggregation ...
eprintln!("elapsed: {:?}", start.elapsed());

Note eprintln! (standard error), not println!. Timing is not part of the analysis results, so it must not leak into the file of a user who redirects with loglens stats access.log > result.txt. Stdout for data, stderr for side information — another long-standing CLI convention.

Compare the two builds on a multi-million-line log and the gap is vivid.

Run 
cargo run -- stats big.log            # debug build
cargo run --release -- stats big.log  # release build

It varies by environment, but release being several to dozens of times faster is normal. The debug build turns optimizations off for compile speed and debugging comfort, and turns on safety checks like the overflow detection from basics Part 2. The claim that iterator chains match handwritten loops (basics Part 7) is also a statement about optimized release builds. So the rule is simple: iterate in debug, measure and ship in release. Drawing the conclusion “Rust is slow” from a debug build is the most common measurement mistake at the beginner stage.

Summary #

  • The structure is a Stats aggregate fed line by line. Because ParseError is an enum, the policy for empty lines vs malformed lines is declared in a match.
  • Before printing a HashMap, collect into a Vec and sort. Output that reorders itself between runs damages trust and test stability.
  • Top-N is into_iter → descending sort_by → take(count). clap’s argument connects directly to the logic.
  • Instrumentation goes to eprintln!. Stdout for data, stderr for side information protects redirecting users.
  • Measure and ship in release, always. The debug gap runs from several times to dozens of times — debug numbers cannot support conclusions.
  • Next part: parallelism. rayon spreads the aggregation across cores, and we watch ownership block a data race at compile time.
X