Command line
The CLI is for looking at a folio, adding files to it, finding things across folios, and managing snapshots without opening a notebook. Reading data is Python's job.
datafolio --help
datafolio --version
Choosing the folio
Four ways, highest priority first:
datafolio -a exp12 describe # registered alias
datafolio -f analysis/experiment-12 describe # explicit path
export DATAFOLIO_PATH=analysis/experiment-12 # environment variable
cd analysis/experiment-12 && datafolio describe # current directory
-a and -f are separate flags, so a name is never mistaken for a path.
If you pass an alias to -f or --to by mistake, the error suggests -a.
Register aliases with datafolio folios alias or
init --alias. Every folio a command uses successfully is also remembered
in a recent list, which datafolio find searches.
Cloud folios
add, find, folios and init accept cloud folios (gs://…,
s3://…), by path or by alias. describe, validate, and the snapshot
commands work on local folios only. Use Python for those on a cloud
folio, or sync it down first.
Commands exit 0 on success and 1 on failure (folio not found, invalid
bundle, unknown snapshot, validation failure).
datafolio init
Create a new folio.
datafolio init # in the current directory
datafolio init experiments/new-exp -d "March review rerun"
datafolio init gs://my-bucket/experiment -d "Cloud experiment"
| Option | |
|---|---|
-d, --description TEXT |
bundle description, stored in metadata |
-n, --name TEXT |
bundle name (default: the directory name) |
--alias TEXT |
register the new folio under this alias |
datafolio add
Add a file to a folio: datafolio add [--to PATH | -a ALIAS] FILE.
datafolio add --to analysis/experiment-12 ~/Downloads/blah.parquet
datafolio add -a exp12 synapses.parquet -d "synapse table I thought was interesting"
datafolio add -a exp12 cells.csv --name cells
datafolio add -a exp12 notes.csv --as-file
datafolio add -a exp12 /data/huge.parquet --reference
The file type decides what gets stored:
| File | Stored as |
|---|---|
.parquet, .csv, .feather, .arrow |
table, as parquet (via folio.import_table) |
.npy |
numpy array |
.json |
JSON data |
| anything else | the file unchanged (artifact) |
Tables are streamed rather than loaded, so files larger than memory work.
Parquet is copied byte for byte. CSV and feather are converted a block at a
time. CSV column types are inferred from the start of the file; if a later
row doesn't fit, the command fails and suggests --as-file or
--reference.
| Option | |
|---|---|
--to PATH |
folio path, local or cloud |
-a, --alias TEXT |
folio alias (instead of --to) |
-n, --name TEXT |
item name (default: file name without extension) |
-d, --description TEXT |
item description |
--as-file |
store any file unchanged, with no conversion |
--reference |
link a parquet/csv file in place, without copying |
--overwrite |
replace an existing item |
Without --to or -a, the folio comes from the global -f/-a,
DATAFOLIO_PATH, or the current directory. add never creates a folio; use
init first.
datafolio folios
Manage the per-user registry in ~/.datafolio (set DATAFOLIO_HOME to keep
it elsewhere). It holds aliases you choose, and a list of the 50 folios most
recently used from the CLI.
datafolio folios list # aliases, then recents
datafolio folios alias exp12 analysis/experiment-12 # register an alias
datafolio folios alias shared gs://team/folios/shared
datafolio folios alias exp12 other/path --overwrite # rebind
datafolio folios unalias exp12
datafolio folios forget ~/scratch/tmp-folio # drop from recents
datafolio folios prune # drop missing local folios
folios alias NAME with no path aliases the current folio. Rebinding an
existing alias needs --overwrite. prune never removes cloud folios.
datafolio find
Find items by name across every aliased and recently used folio. Only each
folio's items.json is read, so this is fast even across many folios.
datafolio find 'cells*' # glob on the whole name
datafolio find 'synapse|soma' --regex --type table
datafolio find --in exp12 --type model # everything of a type in one folio
datafolio find 'dataset=minnie*' --metadata # folio metadata key=value
datafolio find synapse --desc # also search descriptions
Matches for 'cells*'
┏━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Folio ┃ Item ┃ Type ┃ Description ┃
┡━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ exp12 │ cells │ included_table │ cell table │
│ exp13 │ cells_raw │ numpy_array │ │
└────────┴───────────┴────────────────┴──────────────┘
| Option | |
|---|---|
--regex |
treat the pattern as a regular expression (matched anywhere) |
--type TYPE |
table, model, artifact, array, json, timestamp; repeatable |
--metadata |
match folio metadata keys (KEY or KEY=VALUE) instead of item names |
--in ALIAS_OR_PATH |
only search this folio; repeatable |
--aliases-only |
skip recent folios that have no alias |
--local-only |
skip cloud folios |
--include-archived |
include archived items |
--case-sensitive |
match case-sensitively (default: case-insensitive) |
--desc |
also match items whose description contains the pattern |
A folio that can't be opened (moved, deleted, no credentials) is reported
with a warning and skipped. Exits 1 when nothing matches; if the
pattern does appear in item descriptions, the message says so and suggests
--desc. The same search
is available in Python as datafolio.find(), which returns a DataFrame.
datafolio describe
Print the folio: metadata, snapshots, and every item with its description,
size, and lineage. Same output as folio.describe() in Python.
datafolio describe
datafolio describe --snapshot v1.0
datafolio describe --max-metadata 25
| Option | |
|---|---|
--max-metadata INTEGER |
maximum metadata fields to show |
-s, --snapshot TEXT |
describe a snapshot instead of the current state |
datafolio validate
Check that a directory is a folio and that every payload is present and matches its recorded checksum.
datafolio validate
datafolio validate /path/to/folio
✓ Valid DataFolio bundle: /Users/casey/analysis/experiment-12
Items: 12 (all payloads verified)
Snapshots: 1
✗ Invalid DataFolio bundle: /Users/casey/analysis/experiment-12
Items: 12 (1 failing)
✗ raw_measurements: missing or corrupt payload
A failing external reference usually means "unreachable from here" (no credentials, no network) rather than "gone".
datafolio snapshot
datafolio snapshot --help
create
datafolio snapshot create v1.0 -d "Baseline model" -t baseline -t production
datafolio snapshot create v2.0 --env --exec
datafolio snapshot create scratch --no-git
| Option | |
|---|---|
-d, --description TEXT |
snapshot description |
-t, --tag TEXT |
tag; repeat the flag for multiple tags |
--no-git |
skip git capture (captured by default) |
--env |
capture Python version, platform, and package versions |
--exec |
capture entry point and working directory |
Git remote URLs are sanitized of embedded credentials before storage, and environment variables are never captured.
list
datafolio snapshot list
datafolio snapshot list --tag baseline
Snapshots (1)
┏━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Name ┃ Description ┃ Items ┃ Created ┃ Tags ┃
┡━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ v1 │ baseline │ 12 │ 2026-08-14 13:45 EDT │ baseline │
└──────┴─────────────┴───────┴──────────────────────┴──────────┘
show
datafolio snapshot show v1
Snapshot: v1
============================================================
Description: baseline
Tags: baseline
Timestamp: 2026-08-14 13:45:03 EDT
Items (12):
• classifier
• embedding
• features
... and 2 more
Git:
Commit: 9d1a60b
Branch: v2-refactor
Status: dirty (uncommitted changes)
Metadata (2 fields):
• experiment: exp-12
• analyst: casey
status and diff
status is the git status of a folio: what changed since the most recent
snapshot. diff does the same against a named one.
datafolio snapshot status
datafolio snapshot diff # vs. the newest snapshot
datafolio snapshot diff v1.0
Comparing current state to snapshot 'v1'
============================================================
Modified (1):
~ features
Unchanged (11):
= classifier
= embedding
... and 6 more
Summary:
Added: 0
Removed: 0
Modified: 1
Unchanged: 11
"Modified" means the item's current version is not the one the snapshot pinned — a version comparison, not a content diff.
compare
datafolio snapshot compare v1.0 v2.0
delete and gc
datafolio snapshot delete v0.1 --cleanup
datafolio snapshot delete experimental-v5 -y
datafolio snapshot gc --dry-run
datafolio snapshot gc
| Option | |
|---|---|
--cleanup / --no-cleanup |
remove versions orphaned by the deletion (default: no) |
-y, --yes |
skip the confirmation prompt |
--dry-run (gc) |
list what would be removed, remove nothing |
Nothing is garbage-collected automatically; gc is the sweep.
Not in the CLI
restore_snapshot() and export_snapshot() are Python-only — both rewrite or
create bundles, which is not something to trigger from a one-line command by
accident. Reading data is Python-only by design; see the
API cheat sheet.
Scripting against a folio
There is no --json output. For anything scriptable, use Python — it is the
same information without a parsing layer:
python -c "
from datafolio import DataFolio
f = DataFolio('analysis/experiment-12', read_only=True)
print('\n'.join(f.tables))
"
Or read items.json directly — it is a documented, stable format. See
Reading a folio without DataFolio.