anscom documentation

Overview

anscom is a Python C extension that scans directories at raw OS speed.

It uses direct kernel syscalls — getdents64 on Linux, FindFirstFileW on Windows, readdir/lstat on macOS — a multi-threaded work queue, and per-thread statistics accumulation. It never loads file contents into memory (except an optional bounded read for duplicate detection), never follows symlinks, and its memory footprint does not grow with the number of files on disk.

Unlike os.walk or pathlib.rglob, the entire traversal runs in C with the GIL released, so scanning tens of millions of files uses no more memory than scanning tens of thousands. The result is always a plain Python dict.

Installation

anscom is published on PyPI and works on Python 3.6 and later. Pick your platform — macOS ships its own system Python, so use pip3 there; on Linux and Windows pip3 is the usual entry point.

Build with CPU-specific optimizations

On Linux and macOS the default source build is portable. To compile tuned to your own machine (best when you build and run on the same host), opt in with an environment variable:

Quick Start

The entire public surface is two functions: scan() and pwd().

import anscom

# Scan the current directory — no argument needed
result = anscom.scan()

# Silent scan — suppress all terminal output, just get the dict
result = anscom.scan(".", silent=True)
print(result["total_files"], "files in", round(result["duration_seconds"], 3), "s")
print("Scanned:", result["scanned_path"])

# Scan a specific path, deeper
result = anscom.scan("/home/user/projects", max_depth=20, silent=True)

for cat, count in result["categories"].items():
    if count:
        print(f"{cat:20s} {count:>10,}")

Running that script from a shell:

Verify

One line confirms the extension imported and the C engine is live.

import anscom
r = anscom.scan(".", silent=True)
print(f"anscom is working. Files found: {r['total_files']}")

anscom.scan()

Recursively scans path and returns a result dictionary. All arguments except path are keyword-friendly, and path itself is optional.

anscom.scan(
    path=".",            # str  — optional, defaults to the current directory
    max_depth=6,         # int  — recursion depth, clamped to [0, 64]
    show_tree=False,     # bool — print a DFS tree (forces workers=1)
    workers=0,           # int  — 0 = auto-detect CPU count
    min_size=0,          # int  — skip files smaller than N bytes
    extensions=None,     # list — whitelist of extensions to count
    callback=None,       # func — progress callback (~1/sec)
    silent=False,        # bool — suppress all terminal output
    ignore_junk=False,   # bool — skip node_modules, .git, etc.
    export_json=None,    # str  — write the result dict to JSON
    export_tree=None,    # str  — write the tree to a .txt (needs show_tree)
    return_files=False,  # bool — include a per-file list in the dict
    export_csv=None,     # str  — write a per-file CSV inventory
    largest_n=0,         # int  — report the top-N largest files
    find_duplicates=False, # bool — detect duplicate files
    regex_filter=None,   # str  — only count paths matching this pattern
) -> dict

anscom.pwd() new

Returns the current working directory as an absolute path string — the Unix pwd, as a value. It returns (never prints), so it stays composable.

import anscom

root = anscom.pwd()      # e.g. '/home/user/project'
anscom.scan(root)          # feed it straight back in

Signature: anscom.pwd() -> str. Takes no arguments. Raises OSError if the working directory cannot be read (for example, if it has been deleted).

Parameters

Return Value

Always a dict. Six keys are present on every scan; three more appear on demand.

result = anscom.scan("/data", silent=True)
# {
#   'total_files': 21008,
#   'scan_errors': 0,
#   'duration_seconds': 1.5186,
#   'scanned_path': '/data',
#   'categories': {'Code/Source': 5955, 'Documents': 203, ...},
#   'extensions': {'py': 5955, 'pyc': 5707, ...},
# }

Features

All heavier features are strictly opt-in — a plain anscom.scan(".") runs the fast counting path with zero per-file allocation.

Current Directory new in 1.5.1

Two conveniences for working with the current directory, both returning values (nothing is printed):

# pwd() — the current directory as a string
root = anscom.pwd()               # '/home/user/project'

# scan() with no path scans the current directory
r = anscom.scan(silent=True)
print(r["scanned_path"])          # resolved absolute path

# scanned_path is resolved even for relative input
r = anscom.scan("../logs", silent=True)
print(r["scanned_path"])          # e.g. '/home/user/logs'

If a path cannot be resolved (missing or unreadable), scanned_path falls back to the raw input so the key is always present.

Largest Files

Set largest_n=N to get the top-N files by size across the whole scan. Each worker keeps a min-heap of capacity N — O(log N) per file, no extra pass, no full sort. Heaps merge after the threads join.

result = anscom.scan("/mnt/storage", largest_n=20, silent=True)

for f in result["largest_files"]:
    print(f"{f['size'] / 1024**3:.2f} GB  {f['path']}")

Duplicate Detection

Set find_duplicates=True for a two-phase scan: files are bucketed by size (unique sizes are skipped with zero I/O), then each same-size group has the first 4 KB read and CRC32-compared. Matching groups are returned as lists of paths.

result = anscom.scan("/media", find_duplicates=True, silent=True)
print(f"Duplicate groups: {len(result['duplicates'])}")

# Reclaimable space (combine with return_files=True)
result = anscom.scan("/media", find_duplicates=True, return_files=True, silent=True)
size = {f["path"]: f["size"] for f in result["files"]}
wasted = sum(sum(size[p] for p in g[1:]) for g in result["duplicates"])
print(f"Reclaimable: {wasted / 1024**3:.2f} GB")

Regex Filter

Pass regex_filter to count only files whose full path matches the pattern. On Linux and macOS this uses POSIX regcomp/regexec entirely in C with no GIL; on Windows it falls back to Python's re. Invalid patterns raise ValueError before the scan starts.

# Only Python files under a tests/ directory
result = anscom.scan("/repo", regex_filter=r"/tests/.*\.py$", silent=True)
print(f"Matched: {result['total_files']}")

Extensions & Size

Combine an extension whitelist with a minimum size and junk exclusion for fast, targeted audits.

# Only web source, over 1 KB, skipping node_modules/.git/etc.
result = anscom.scan(
    "/project",
    extensions=["js", "ts", "jsx", "css", "html"],
    min_size=1024,
    ignore_junk=True,
    silent=True,
)

Directories skipped by ignore_junk=True (case-insensitive, at any depth):

.git .svn .hg .idea .vscode node_modules bower_components site-packages .venv venv env build dist target __pycache__ temp tmp .cache .pytest_cache .mypy_cache

CSV / JSON Export

Exports are parameters of scan(). One traversal can write multiple files at once, with no re-scan and no third-party dependency.

anscom.scan(
    "/data",
    export_json="report.json",     # full result dict as JSON
    export_csv="inventory.csv",    # per-file: path,size,ext,category,mtime
    silent=True,
)

Load the CSV downstream, or convert to Excel through pandas:

import pandas as pd
df = pd.read_csv("inventory.csv")
print(df.groupby("category")["size"].sum())
df.to_excel("report.xlsx", index=False)   # Excel via pandas

Tree View

With show_tree=True, anscom prints a DFS-ordered tree as each entry is discovered and, optionally, mirrors it to a file. Workers are forced to 1 for strict ordering; lines are streamed with no buffer, so a 50-million-entry filesystem produces 50 million lines without memory growth.

anscom.scan(".", show_tree=True, max_depth=4,
            export_tree="tree.txt", silent=True)

[brackets] mark directories; unbracketed names are files.

Report Format

When silent=False (the default), anscom prints a summary report and an extension breakdown on completion. silent=True suppresses everything — banner, live counter, and report.

With largest_n or find_duplicates enabled, a "TOP N LARGEST FILES" and/or "DUPLICATES SUMMARY" section is appended.

File Categories

170+ extensions across 9 categories, in a table validated at module load. Any extension not in the table falls into Other/Unknown.

Architecture

Six engineering choices determine correctness and throughput:

• Direct OS syscalls. getdents64 on Linux (128 KB read buffer, d_type avoids a stat), FindFirstFileW on Windows, readdir/lstat on macOS.
• Multi-threaded worker pool. N threads auto-detected from the core count, all blocked on a condition variable before the first directory opens, pulling from a shared 131,072-slot ring queue.
• Slab path allocator. Each thread allocates one contiguous block up front; path assembly during traversal does zero heap allocation.
• Per-thread statistics. No shared counters during counting — each thread writes its own struct; results merge in one serial pass after join.
• FNV-1a hash table. A 512-slot open-addressing table gives O(1) extension lookup, built once at import and never modified.
• GIL released for the whole scan. Workers run under Py_BEGIN_ALLOW_THREADS; the GIL is re-acquired only for the progress callback and final dict construction.

On a plain count-only scan (v1.5.1), the per-thread file buffer is no longer pre-allocated, so the hot path performs zero heap work.

Security & Safety

• No file contents read — only directory entries and metadata. The one exception is find_duplicates=True, which reads up to 4 KB per candidate (bounded, opt-in, read-only).
• Symlinks never followed — fstatat(AT_SYMLINK_NOFOLLOW) / lstat on POSIX, reparse points skipped on Windows.
• Depth hard-capped at 64 — enforced in C at the top of every recursion.
• Bounded path assembly — every path is built with snprintf into a fixed buffer.
• Errors counted, not silenced — every failed open increments scan_errors and the scan continues.
• Zero mandatory dependencies — the core is pure C, so there is no supply-chain surface.

Additional Recipes

Everything at once

import anscom

print("Scanning:", anscom.pwd())

result = anscom.scan(                # no path -> current directory
    max_depth=20,
    workers=32,
    ignore_junk=True,
    silent=True,
    return_files=True,
    largest_n=50,
    find_duplicates=True,
    export_json="audit.json",
    export_csv="inventory.csv",
)

print(f"Root       : {result['scanned_path']}")
print(f"Files      : {result['total_files']:,}")
print(f"Duration   : {result['duration_seconds']:.3f}s")
print(f"Dup groups : {len(result['duplicates'])}")

CI/CD policy gate

import anscom, sys
r = anscom.scan("./repo", silent=True, ignore_junk=True)
if r["categories"]["Executables"] > 0:
    print("POLICY VIOLATION: executables in repo")
    sys.exit(1)

Changelog

Ten releases since January 2026. Open the release history for the full timeline in a side panel.

Mascot new in 1.5.2

Meet Chuhi — the anscom mascot, the house mouse of a Kolkata apartment. Run anscom.about() to print the original ASCII art with its credit line in your own terminal; the call returns None and touches nothing else.

Chuhi — the anscom mascot. Prints the same on macOS (python3) and on Windows (python).

License

anscom is released under the MIT License — free for personal and commercial use. Copyright © 2026 Aditya Narayan Singh.