File System Operations
Overview
Apply safe path handling and temporary file patterns so reads and writes are reliable, even with untrusted input or large intermediate data.
When to Use
- Use this skill only after the frontmatter triggers; otherwise start with a simple read/write path.
Decision Tree
- Normalize or resolve the path?
- Normalize only: use
os.path.normpath()for lexical cleanup. - Resolve + check existence: use
os.path.realpath(path, strict=True).
- Normalize only: use
- Are you writing to a target path that may already exist?
- Yes: write to a temp file in the same directory and replace.
- Is the file size unknown or potentially large?
- Yes: use
tempfile.SpooledTemporaryFileto avoid unnecessary disk writes.
- Yes: use
Workflows
1. Normalize vs Resolve Paths
- Normalize user input with
os.path.normpath()to collapse redundant segments. - Resolve real paths with
os.path.realpath(path, strict=True)when you must confirm the file exists. - Reject paths that fail resolution or escape your allowed base directory.
2. Atomic Replace Write
- Create a temporary file in the destination directory.
- Write and flush content to the temp file.
- Replace the destination with
os.replace/os.renameafter the write completes. - Ensure the temp file and destination are on the same filesystem.
3. Spooled Temporary Buffer
- Create
tempfile.SpooledTemporaryFile(max_size=...). - Stream data into the spooled file as you process it.
- Call
.rollover()to force the buffer onto disk when needed.
Non-Obvious Insights
normpathis lexical only and can change the meaning of a path with symlinks.realpath(..., strict=True)raisesFileNotFoundErrorwhen the path does not exist.os.renameis atomic on POSIX when successful; keep temp and destination on the same filesystem.- Temporary files may not have a visible name; do not rely on name visibility.
- Spooled temp files can be forced to disk with
rollover()when a file grows.
Evidence
- "Normalize a pathname by collapsing redundant separators and up-level references so that A//B, A/B/, A/./B and A/foo/../B all become A/B. This string manipulation may change the meaning of a path that contains symbolic links." - Python Docs
- "FileNotFoundError is raised if path does not exist, or another OSError if it is otherwise inaccessible." - Python Docs
- "If successful, the renaming will be an atomic operation (this is a POSIX requirement)." - Python Docs
- "Rename the file or directory src to dst. If dst exists and is a file, it will be replaced silently if the user has permission. The operation may fail if src and dst are on different filesystems." - Python Docs
- "TemporaryFile, NamedTemporaryFile, TemporaryDirectory, and SpooledTemporaryFile are high-level interfaces which provide automatic cleanup and can be used as context managers." - Python Docs
- "should not rely on a temporary file created using this function having or not having a visible name in the file system." - Python Docs
- "rollover() ... causes the file to roll over to an on-disk file regardless of its size." - Python Docs
Scripts
scripts/file-system_tool.py: CLI for resolve, atomic write, copy, and safe reads.scripts/file-system_tool.js: Node.js CLI equivalents.
Dependencies
- Python standard library (
os,tempfile,pathlib,shutil) or Node standard library (fs,path,os).