Read any Substack publication as a local, full-text-searchable corpus — keyless for free posts, your own session for what you subscribe to.
Created by @maxswinguy (Maxime Delavergne).
Contributors: @lagiosv (Vasileios Lagios), @giuseppebisemi (Giuseppe Bisemi).
Substack Reader archives whole publications into a local SQLite mirror you can search, SQL-query, and read offline. Free posts need no login; paid posts you're entitled to unlock with your own session cookie — never redistributed, always opt-in. Unlike every other Substack tool it builds a corpus that compounds instead of fetching live per call.
Authentication
Free and public posts are keyless — zero setup. Every command works the moment the CLI is installed: no account, no API key, nothing to configure. Authentication only ever matters for paid posts.
There is one optional layer: your own Substack session cookie (substack.sid). Substack decides entitlement server-side from the session you present, so handing the reader your own cookie unlocks exactly the paid posts you already subscribe to — and nothing else. This is your own cookie, never an API key, it is never required for free content, and one cookie covers every subscription on your account (entitlement is account-level, not per-publication).
Provide it any of these three ways (first hit wins):
# 1. Environment variable — the bare substack.sid value...
export SUBSTACK_SESSION="s%3A..."
# ...or the whole cookie fragment, if that is what you copied
export SUBSTACK_SESSION="substack.sid=s%3A..."
# 2. A JSON cookie file at the default config-dir path
# (~/.config/substack-reader-pp-cli/cookie.json on the platform default).
# The directory does not exist on a clean install — create it first.
mkdir -p ~/.config/substack-reader-pp-cli && chmod 700 ~/.config/substack-reader-pp-cli
echo '{"substack.sid":"s%3A..."}' > ~/.config/substack-reader-pp-cli/cookie.json
chmod 600 ~/.config/substack-reader-pp-cli/cookie.json
# 3. The same JSON cookie file, kept anywhere on disk
export SUBSTACK_COOKIE_FILE=/path/to/substack-cookie.json
Precedence is SUBSTACK_SESSION, then SUBSTACK_COOKIE_FILE, then the default config-dir file. SUBSTACK_COOKIE_FILE is authoritative once set: a missing, unreadable, unparseable or substack.sid-less file there is a hard error, never a silent fall-through to a stale default. The default config-dir file is best-effort: a file that exists but cannot be read or parsed degrades to anonymous with a warning, while an absent file — or one that parses but carries no substack.sid — degrades to anonymous silently. The config directory itself is relocatable with SUBSTACK_READER_CONFIG_DIR or SUBSTACK_READER_HOME.
Getting the cookie out of your browser. Open DevTools → Application → Cookies → https://substack.com, and copy the value of substack.sid. Only substack.sid is needed — connect.sid is not. Copy it rather than retyping it: a single capital-I/lowercase-l slip yields a cookie that silently reads as anonymous.
With the MCP server. The MCP server resolves the session exactly as the CLI does, so the cookie file in the config directory needs no env block at all — save cookie.json and the server picks it up. If you would rather configure it explicitly, set either variable under env in your Claude Desktop config (the MCPB bundle prompts for it as an optional field).
Your cookie stays yours. It never leaves your machine except to Substack itself, is masked in every output path so scripted runs cannot leak it, and is never redistributed. Any copy the CLI persists is written 0600 under a 0700 parent, and a group/other-readable cookie file earns a warning.
Failure is honest, never silent. With no session, a paid post returns Substack's public preview and says so: access: preview, authenticated: false, full: false, plus a preview only (N of ~M words) — no session configured line on stderr. A downgrade is always visible; it never passes itself off as a full read.
Quick Start
# confirm reachability and config before anything else
substack-reader-pp-cli doctor --dry-run
# mirror a publication's recent posts into the local corpus
substack-reader-pp-cli archive astralcodexten --limit 50
# full-text search the corpus offline
substack-reader-pp-cli search "open thread"
# read a single post's full text
substack-reader-pp-cli read astralcodexten/open-thread-441
Unique Features
These capabilities aren't available in any other tool for this API.
Local corpus that compounds
-
archive — Archive a whole Substack publication into a local SQLite mirror you can read, search, and query offline — no other Substack tool builds a persistent corpus. --limit 0 walks the whole archive; when a run stops at --limit instead of exhausting the archive, the output says so (JSON carries "exhausted": false) — never treat a limit-stopped run as a complete archive.
Reach for this to turn a live newsletter into a durable, queryable knowledge base instead of re-fetching every time.
substack-reader-pp-cli archive astralcodexten --limit 0
-
sql — Run read-only SQL over your local Substack corpus for arbitrary analytics — post cadence, audience mix, longest posts — from data you've already archived.
Reach for this for ad-hoc analytics over what you've archived, without re-fetching or writing code.
substack-reader-pp-cli sql "SELECT json_extract(data,'$.audience') AS audience, COUNT(*) FROM resources WHERE resource_type='posts' GROUP BY audience"
Entitlement-aware reading
-
read — Read one or more posts' full text in a single call; free posts keyless, and paid posts you subscribe to via your own session cookie — with an honest 'preview only, you're not entitled' signal. With several posts, JSON mode emits an array of envelopes (a failed post becomes a {"post", "error"} entry and the rest still return); a single post keeps the single-object envelope.
Use to pull posts' full text into an agent workflow — several slugs in one call, no shell loop — respecting exactly what the user is entitled to.
substack-reader-pp-cli read astralcodexten/open-thread-441
substack-reader-pp-cli read astralcodexten/slug-one astralcodexten/slug-two --agent --select slug,subtitle
Topic & comparative intelligence
-
digest — A time-windowed digest across every publication in your local corpus — what's new since you last synced, ranked, in one view.
Use as a personal 'what did I miss across my newsletters' briefing.
substack-reader-pp-cli digest --since 7d
-
author-compare — Compare two publications' cadence, topics, and free/paid mix from the local corpus.
Use to size up a newsletter before subscribing, or to study what a successful author publishes.
substack-reader-pp-cli author-compare astralcodexten blog.bytebytego.com
Recipes
Build a searchable corpus
substack-reader-pp-cli archive astralcodexten --limit 0 && substack-reader-pp-cli search "prediction markets"
Mirror a whole publication (--limit 0 = no cap) then search it offline with FTS ranking. Compact/agent search hits carry key fields plus a bounded snippet — never full post bodies; use read (or --select) when you want a hit's full text.
Narrow a large post to fields
substack-reader-pp-cli read astralcodexten/open-thread-441 --agent --select title,post_date,audience,body_html
Pull only the fields an agent needs from a verbose post object.
Read several posts in one call
substack-reader-pp-cli read astralcodexten/slug-one astralcodexten/slug-two astralcodexten/slug-three --agent --select slug,subtitle,audience
Batch-read a list of slugs without a shell loop; JSON mode returns an array of post envelopes, and one bad slug doesn't sink the rest.
Audience mix analytics
substack-reader-pp-cli sql "SELECT json_extract(data,'$.audience') AS audience, COUNT(*) FROM resources WHERE resource_type='posts' GROUP BY audience"
Read-only SQL over the local corpus for arbitrary analytics.
Usage
Run substack-reader-pp-cli --help for the full command reference and flag list.
Paths & environment variables
This CLI separates local files into four path kinds:
| Kind | Contents |
|---|
config | User-editable settings such as config.toml and saved profiles |
data | Durable local data such as data.db |
state | Runtime state such as persisted queries, jobs, and teach.log |
cache | Regenerable HTTP/cache files |
Each kind resolves independently. The ladder is:
- Per-kind env var:
SUBSTACK_READER_CONFIG_DIR, SUBSTACK_READER_DATA_DIR, SUBSTACK_READER_STATE_DIR, or SUBSTACK_READER_CACHE_DIR
--home <dir> for this invocationSUBSTACK_READER_HOME for a flat relocated root- XDG env vars:
XDG_CONFIG_HOME, XDG_DATA_HOME, XDG_STATE_HOME, XDG_CACHE_HOME
- Platform defaults matching existing installs
For containers and agent sandboxes, prefer a single relocated root:
export SUBSTACK_READER_HOME=/srv/substack-reader
substack-reader-pp-cli doctor
Under SUBSTACK_READER_HOME=/srv/substack-reader, the four dirs resolve to /srv/substack-reader/config, /srv/substack-reader/data, /srv/substack-reader/state, and /srv/substack-reader/cache.
MCP servers do not receive CLI flags from the host. Put relocation in the host env block:
{
"mcpServers": {
"substack-reader": {
"command": "substack-reader-pp-mcp",
"env": {
"SUBSTACK_READER_HOME": "/srv/substack-reader"
}
}
}
}
Precedence matters in fleets: an ambient per-kind variable such as SUBSTACK_READER_DATA_DIR overrides an explicit --home for that kind. Use SUBSTACK_READER_HOME or the per-kind variables for durable fleet relocation; treat --home as the weaker per-invocation lever.
Relocation is one-way. Unsetting SUBSTACK_READER_HOME does not move files back to platform defaults, and doctor cannot find files left under a former root. Move the files manually before unsetting relocation variables.
Existing installs keep working because the platform-default rung matches the legacy layout. Run substack-reader-pp-cli doctor --fail-on warn to check path warnings in automation.
Session environment variables
Two further variables carry the optional Tier-1 session cookie (see Authentication). Neither is required, and free or public posts never need them:
| Name | Kind | Required | Description |
|---|
SUBSTACK_SESSION | env | No | Your own Substack session cookie — the bare substack.sid value, or a substack.sid=<value> cookie fragment. Unlocks paid posts you already subscribe to. |
SUBSTACK_COOKIE_FILE | env | No | Path to a flat-JSON cookie file {"substack.sid":"s%3A.."}, anywhere on disk. Authoritative when set; otherwise the default <config>/cookie.json is used. |
Commands
categories
Browse Substack's publication categories
substack-reader-pp-cli categories browse - List publications in a categorysubstack-reader-pp-cli categories list - List all Substack categories
publications
Discover Substack publications
substack-reader-pp-cli publications <query> - Search Substack publications by name (best-effort; may return few results anonymously)
Self-learning loop
This CLI caches per-question discovery so repeat queries skip the walk and structurally similar queries get answered via entity substitution. The loop also self-captures: every invocation is journaled locally, and failed-flag corrections plus fresh teaches surface as candidates on the next recall for confirm/reject judgment. Agents call recall before discovery and fire teach & after answering. See the ## Automatic learning section in SKILL.md for the full protocol.
substack-reader-pp-cli recall <query> - Look up cached resources for a query before running discoverysubstack-reader-pp-cli teach - Record a query -> resource mapping (silent on success, safe to background with &)substack-reader-pp-cli learnings list - Inspect taught rowssubstack-reader-pp-cli learnings forget <query> - Undo a teachsubstack-reader-pp-cli learnings candidates - List auto-captured candidates awaiting confirm/rejectsubstack-reader-pp-cli learnings stats - Local loop metrics: recall hit rate, teach-to-reuse, playbook resolution, candidate countssubstack-reader-pp-cli teach-pattern - Install a query/resource template up frontsubstack-reader-pp-cli teach-lookup - Add an entity mapping (e.g. country code, team alias) for pattern substitution
Pass --no-learn or set SUBSTACK_READER_NO_LEARN=true to disable the loop for deterministic flows.
The local store's schema version stamp is one-way: once this version of substack-reader-pp-cli opens the database, older binaries refuse it with a version error — upgrade the binary rather than downgrading.
Output Formats
# Human-readable table (default in terminal, JSON when piped)
substack-reader-pp-cli categories list
# JSON for scripting and agents
substack-reader-pp-cli categories list --json
# Filter to specific fields
substack-reader-pp-cli categories list --json --select id,name,status
# Dry run — show the request without sending
substack-reader-pp-cli categories list --dry-run
# Agent mode — JSON + compact + no prompts in one flag
substack-reader-pp-cli categories list --agent
Agent Usage
This CLI is designed for AI agent consumption:
- Non-interactive - never prompts, every input is a flag
- Pipeable -
--json output to stdout, errors to stderr
- Filterable -
--select id,name returns only fields you need
- Previewable -
--dry-run shows the request without sending
- Read-only by default - this CLI does not create, update, delete, publish, send, or mutate remote resources
- Offline-friendly - sync/search commands can use the local SQLite store when available
- Agent-safe by default - no colors or formatting unless
--human-friendly is set
Exit codes: 0 success, 2 usage error, 3 not found, 5 API error, 7 rate limited, 10 config error.
Health Check
substack-reader-pp-cli doctor
Verifies configuration and connectivity to the API.
Configuration
Run substack-reader-pp-cli doctor to see the resolved config, data, state, and cache directories. The platform-default config path is ~/.config/substack-reader-pp-cli/config.toml; --home, SUBSTACK_READER_HOME, and per-kind env vars can relocate it.
Static request headers can be configured under headers; per-command header overrides take precedence.
Troubleshooting
Not found errors (exit code 3)
- Check the resource ID is correct
- Run the
list command to see available items
API-specific
- A paid post reads as preview-only despite being subscribed — Check that your substack.sid session cookie is set (SUBSTACK_SESSION or the config cookie file) and current — an expired or mistyped cookie silently falls back to the public preview. The reader unlocks paid posts through Substack's authenticated by-id endpoint, so a valid session is all that is needed; you never target a particular host yourself.
- Empty results / HTTP 429 — You are being rate-limited; slow down (archive is self-throttled) — Substack sets no rate headers, so pace requests.
Sources & Inspiration
This CLI was built by studying these projects and resources:
Generated by CLI Printing Press