Development Guide

s3fs is a community-maintained project. We welcome and encourage contributions in the form of bug reports, documentation updates, code patches, and design proposals.

This guide outlines how to set up your environment, run the test suite, and prepare your code for submission.

Creating a Development Environment

To start contributing, fork the repository on GitHub, then clone it to your local machine. It is highly recommended to use a virtual environment or conda environment to avoid dependency conflicts.

# Clone your fork
git clone https://github.com/<your-username>/s3fs
cd s3fs

# Create and activate a clean conda environment
conda create -n s3fs-dev python=3.11
conda activate s3fs-dev

# Install the package dependencies and testing requirements
pip install -r requirements.txt -r test_requirements.txt

# Install s3fs in editable mode
pip install -e .

Code Style & Linting

This project enforces strict code formatting to keep the codebase clean and readable. We use pre-commit to automatically run formatters and linters before you commit your code.

The project uses:

black for deterministic code formatting.
flake8 for linting and style checks.

To install the hooks locally:

pip install pre-commit
pre-commit install

Now, whenever you run git commit, your code will be automatically checked and formatted. If a hook fails or modifies files, you will need to stage the changes and run git commit again.

Running Tests

Tests are written using the pytest framework.

Crucially, the test suite heavily relies on moto (specifically the ThreadedMotoServer) to mock an S3 backend locally. You do not need real AWS credentials or internet access to run most of the tests. moto simulates the S3 API directly on your local machine.

To execute the entire test suite:

pytest -vv -s s3fs

Writing New Tests

If you are adding a new feature or fixing a bug, please ensure you add corresponding test cases in the s3fs/tests/ directory.

Use the s3 or s3_base pytest fixtures. These fixtures automatically stand up the moto server, clear out old mock buckets, and provide a pre-authenticated S3FileSystem instance.
For tests that require simulating network errors, look at test_custom_error_handler.py for examples on how to mock botocore exception responses.

Building Documentation

The documentation is built using Sphinx. To build the docs locally and preview your changes:

# Install sphinx and the read-the-docs theme
pip install sphinx sphinx_rtd_theme

# Build the HTML output
cd docs
make html

The generated HTML will be located in docs/build/html/. Open index.html in your browser to view it.

Release Procedure (Maintainers Only)

If you are a core maintainer responsible for releasing a new version of s3fs to PyPI, follow these steps:

Verify Tests: Ensure all CI tests pass across Linux, macOS, and Windows.
Update Changelog: Complete entries in docs/source/changelog.rst. Ensure the date and version number are correct. Note: There's no need to manually change version numbers in the source Python files. The release version is dynamically determined from the git tag via versioneer.
Tag the Commit: Create an annotated git tag matching the version (e.g., using CalVer YYYY.MM.MINOR).
```
git tag 2026.2.0 -m "Version 2026.2.0"
```
Push the Tag:
```
git push fsspec main --tags
```
Build Packages: Clean old builds, then build the source distribution and universal wheel.
```
rm -rf dist/
python setup.py sdist bdist_wheel --universal
```
Upload to PyPI: Use twine to publish the artifacts.
```
twine upload dist/*
```