SeaweedFS
This skill is a practical router for deploying and operating SeaweedFS from the upstream repository and wiki.
Prefer production guidance from multi-component setups over weed mini shortcuts.
Quick Navigation
| Situation | Open |
| ----------------------------------------------------- | ------------------------------------------------------------------ |
| Learn the system shape and bootstrap paths | references/getting-started.md |
| Stand up a local all-in-one sandbox | references/quick-start-mini.md |
| Review control-plane, volume, and collection topology | references/topology-and-setup.md |
| Check master, volume, filer, and client API surfaces | references/api-surfaces.md |
| Set replication, TTL, failover masters, and env vars | references/configuration.md |
| Work with performance notes, FAQ topics, and examples | references/benchmarks-and-use-cases.md |
| Work with filer metadata, uploads, JWT, and TUS | references/filer-core.md |
| Choose and scale filer metadata stores | references/filer-stores.md |
| Operate S3 buckets, auth, and IAM/OIDC | references/s3-gateway.md |
| Plan Cloud Drive and remote storage mounts | references/cloud-drive.md |
| Run backups, metrics, repairs, and shell workflows | references/backup-and-replication.md, references/operations.md |
| Choose S3 encryption and client tooling | references/encryption.md, references/s3-client-tools.md |
| Review transport, JWT, TLS, and exposure controls | references/security.md |
When to Use
- Planning a SeaweedFS deployment
- Running
weedcomponents in development or production - Designing filer, S3, or cloud-tier topologies
- Choosing metadata stores and replication patterns
- Hardening SeaweedFS for public or multi-tenant use
- Operating backups, metrics, and cluster repair workflows
Core Mental Model
- SeaweedFS separates volume management from file and object access paths.
- The filer layer adds directories, metadata stores, and higher-level protocols.
- S3, WebDAV, FUSE, and other interfaces are front doors on top of the same storage services.
- Production deployments should document topology, credentials, persistence, monitoring, and recovery paths explicitly.
Release Highlights (4.25)
- Security/admin path:
4.24-4.25tightens admin auth on destructive/admin endpoints and fixes Admin UI behavior undersecurity.tomlby attaching admin-signed auth on filer IAM gRPC calls. - Erasure coding / multi-disk ops: the release line fixes several EC planner/recovery cases across multi-disk and cross-server layouts, including stale-shard cleanup and safer source-volume deletion.
- S3/IAM hardening: IAM users without policies are now denied instead of implicitly over-permitted, while OIDC/web-identity and audit surfaces continue to mature.
Release Highlights (4.26 -> 4.28)
- Erasure coding / multi-disk ops: EC planning now packs shards across disks more reliably, includes
disk_idin execution planning, and can rebuild lost.ecx/.vifmetadata from local shards during recovery. - Integrity checks: scrubbing/fsck paths now account for zero-sized volumes instead of silently skipping them, which matters for sparse or recently created topologies.
- Filer backend reliability: Redis3 avoids a skiplist-end panic path, and SQL-based filer stores no longer force-disable idle connection pooling.
- S3 audit trail: requester identity is populated more consistently for GET/HEAD/IAM operations, improving compliance and incident triage.
- HA heartbeat path: masters now accept volume-server ping targets on follower replicas, which improves failover visibility in multi-master deployments.
Release Highlights (4.31 -> 4.33)
- S3 versioning/IAM: atomic versioned
PutObjectcommits, properNoSuchVersionerrors, latest-pointer repair and recovery, suspended-versioning delete handling, managed IAM policy versions, and bucket read-only quota enforcement;4.33also fixes HTTP-date conditionals, checksum trailer headers, and empty v4 signed header names. - Erasure coding integrity: per-shard checksum sidecars for bitrot detection (
4.31), even shard spreading across machines with co-located servers treated as a single fault domain and pre-deletion shard-landing verification (4.32), and guards preventing EC shards from different encode runs mixing or leaving phantom.daton restart (4.33). - Volume server: fixes
maxVolumeCountdead zone on auto-sized disks, keeps volumes writable after deletion-tail compaction, resolves EC data-shard count from.vifon reboot (4.31), and addsCheckDiskdisk-health detection (4.32). - Filer scalability: bounded BFS-metadata memory and byte-lexicographic S3 list order (
4.31), streamed metadata-subscription log files (4.32), and object-size distribution metrics, mount-reconnect OOM fixes, per-chunk replay, a bounded flush queue, and Elasticsearch 8 (elastic8) support (4.33). - Admin/worker: lane-aware scheduler pruning,
-dataDirdefaulting for persisted state and in-flight task reload on startup (4.32), plus full cluster volume list JSON export (4.33). - Rust volume server: bounded request body / stored-content expansion to prevent OOM under load (
4.31); stops EC-shard-deletion phantom.daton restart (4.33).
Release Highlights (4.29 -> 4.30)
- S3 write/auth path: object writes move toward filer-side owner routing and
ObjectTransactionbatches, reducing distributed-lock pressure;4.30also tightens path traversal rejection, ownership-control validation,MetadataDirective=REPLACE, and JWT handling for unsigned-streaming uploads. - FUSE and filer coordination: POSIX advisory locks are now routed through filer ownership under
-dlm, with session leases, keepalive, ring-change cooling, and writeback-cache crash fixes for cross-mount write workloads. - Erasure coding and volume repair: EC placement uses the shared
ecbalancer.Placepath and placement snapshots, while4.30improves credible-replica selection, empty-stub cleanup,.vifpreservation, writable-volume notification after vacuum, and shell safety around stuck read-only volumes and merge verification. - Operations surfaces: admin exposes Prometheus metrics, and S3/IAM/volume/filer/master processes add
/healthzand/readyzprobes for orchestration checks. - Filer, sync, and remote sinks: Redis2 now applies
keyPrefixin KV methods, Postgres filer writes default toON CONFLICTupsert, dropped filers are pruned from discovery, and sync/remote-storage paths preserve chunk size, manifest, offset, and MIME metadata more reliably.
Release Highlights (4.20)
- S3/IAM: embedded IAM flows gained
ListUserPolicies, group inline policy actions, safer user-policy round trips, and bucket-scoped cleanup onDeleteBucket. - Mount/FUSE:
weed mountadds-dlmfor cross-mount write coordination and improves POSIX metadata behavior,nlinkaccounting, and filer RPC efficiency. - Master placement: volume assignment is more size-aware, readonly transitions drain pending size first, and a topology bug that could cause endless growth in some DC/rack layouts was fixed.
- Filer reliability: PgBouncer/Postgres compatibility improved, graceful shutdown corruption was fixed, and redundant filer disk reads that caused memory/CPU regressions were removed.
- Ops surfaces:
weed shellgained group-management helpers, S3 user provisioning handles existing users more safely, and master/volume now exportstart_time_secondsmetrics.
Prohibitions
- Do not use
weed minifor production. - Do not treat single-binary defaults as production-safe configuration.
- Do not expose S3 or filer endpoints publicly before reviewing auth, TLS, and network boundaries.
- Do not choose a filer store without validating HA, scaling, and backup properties.
- Do not design backup or replication flows without restore validation.