#Programming Language
189 posts
Python Data Analysis #6 Visualization: matplotlib Fundamentals and Choosing Charts
The minimal matplotlib structure understood through Figure and Axes, fast plotting with DataFrame.plot, picking the right chart for each purpose, fixing broken fonts in CJK environments, and saving with savefig — the visualization fundamentals in one post.
Python Data Analysis #5 Grouping and Joining: groupby, pivot_table, merge
Starting from the classic sales-by-branch-by-month question, we build a mental model for groupby, multi-stat aggregation with agg, pivot_table for Excel users, and merge and concat as the pandas counterparts of SQL JOIN — plus the habit of checking row counts after every join.
Python Data Analysis #4: Transforming Data — New Columns, Dates, and Missing Values
One post covering the data cleanup phase in pandas: vectorized operations for new columns, the str and dt accessors, what NaN really is, how to decide between dropna and fillna, type conversion with astype, and removing duplicates.
Python Data Analysis #3: Selecting and Filtering — loc, iloc, and Boolean Indexing
How to pick out just the rows and columns you want in pandas: single vs. double brackets for column selection, the loc/iloc distinction, boolean indexing where a condition becomes a mask, the query method, and the danger that SettingWithCopyWarning is warning you about.
Python Data Analysis #2: Loading Data — CSV, Excel, and First Exploration
The encoding, sep, and dtype arguments of read_csv and the legacy-codepage trap, sheet selection in read_excel, and the routine of checking your data with head, info, and describe right after loading.
Python Data Analysis #1: Getting Started with pandas — Notebooks and the DataFrame
pandas is the Python library for working with tabular data. We set up a notebook environment with uv, build Series and DataFrame objects by hand, and kick off this seven-part data analysis series.
Python Automation #7: Building Your Own Command — Packaging a CLI with typer and rich
Closing out the series by bundling the automation scripts into a single CLI with typer, polishing the output with rich, and installing it with uv tool install as a command you can call from anywhere.
Python Automation #6: Scheduling — Making Scripts Run While You Sleep
How to make a finished automation script run on its own at a fixed time. We compare cron, launchd, Windows Task Scheduler, and APScheduler, keep run records with logging, get notified on failures, and cover GitHub Actions schedules.
Python Automation #5: Reporting Results — Email, Slack, and Discord Notifications
The last piece of automation is reporting. Send Slack and Discord notifications with a single httpx POST, email Excel attachments with smtplib, and keep tokens out of your code with a .env file.
Python Automation #4: Web Scraping Part 2 — Dynamic Pages with Playwright
Handle JavaScript-rendered pages that return empty HTML to requests by driving a real browser with Playwright. From install and a first script to wait_for_selector, login automation, infinite scroll, and saving to CSV — all in one pass.
Python Automation #3: Web Scraping Part 1 — Static Pages with httpx and BeautifulSoup
Let code read the price, notice, and stock pages you check by hand every day. We fetch HTML with httpx, pick out just the data we need with BeautifulSoup CSS selectors, and save it to CSV — static page scraping from start to finish.
Python Automation #2: Excel Automation — Ending Repetitive Reports with openpyxl
Read Excel files with openpyxl, merge per-branch reports from a folder into a single sheet, and finish with header styling and number formats. We also cover the formula and data_only pitfalls, merged cells, and large-file performance.