S3-Compatible Storage

File shares can also be mounted as network drives via NFS - see the NFS documentation.

Overview

The S3 storage service:

• Is only accessible from within the mesh network
• Uses the same authentication as other mesh services
• Supports standard S3 API operations
• Stores coordinator internal state (users, roles, stats)

Configuration

S3 is automatically enabled on all coordinators (hardcoded port: 9000).

Optional S3 configuration in coordinator config:

# S3 is always enabled on coordinators (port 9000, mesh IP only)
coordinator:
  s3:
    max_size: "100Gi"                  # Storage quota (required)
    data_dir: /var/lib/tunnelmesh/s3   # Storage directory

Configuration Options

Size Format

The max_size option accepts Kubernetes-style size notation:

• Ki, Mi, Gi, Ti - binary units (1Ki = 1024 bytes)
• K, M, G, T - also supported as aliases
• Plain numbers are interpreted as bytes

API Endpoints

The S3 API is available at https://this.tm:9000 (or your configured port).

Supported Operations

Authentication

The S3 service supports multiple authentication methods:

1. AWS Signature V4 - Standard S3 authentication

   Authorization: AWS4-HMAC-SHA256 Credential=ACCESS_KEY/...
   

2. Basic Auth - Simple username/password

   Authorization: Basic base64(access_key:secret_key)
   

3. Bearer Token - Access key only

   Authorization: Bearer access_key
   

CLI Commands

Bucket Operations

# Create a bucket
tunnelmesh bucket create my-bucket

# List buckets
tunnelmesh bucket list

# Delete a bucket
tunnelmesh bucket delete my-bucket

# Get bucket info
tunnelmesh bucket info my-bucket

Object Operations

# Upload a file
tunnelmesh object put my-bucket/path/to/file.txt local-file.txt

# Download a file
tunnelmesh object get my-bucket/path/to/file.txt output.txt

# List objects
tunnelmesh object list my-bucket
tunnelmesh object list my-bucket --prefix docs/

# Delete an object
tunnelmesh object delete my-bucket/path/to/file.txt

System Bucket

The coordinator uses a reserved _tunnelmesh bucket for internal state:

_tunnelmesh/
  auth/
    users.json      # Registered users
    roles.json      # Custom roles
    bindings.json   # Role bindings
  stats/
    history.json    # Stats history

This bucket is only accessible to service users with the system role.

File Shares

File shares are user-accessible storage areas backed by S3 buckets. They provide an easy way to share files across the mesh with automatic permission management.

Creating File Shares

File shares can be created via the admin panel's Data tab or the API:

# Via API
curl -X POST https://this.tm/api/shares \

  -H "Content-Type: application/json" \
  -d '{"name": "team-files", "description": "Shared team files", "quota_bytes": 10737418240}'

Share Properties

Default Permissions

When a file share is created:

• The creator becomes the owner with bucket-admin role
• If "Guest Read" is enabled (default): all mesh users (everyone group) get bucket-read access
• If "Guest Read" is disabled: access is controlled entirely via RBAC bindings
• Additional permissions can be granted via role bindings in the Data tab

Accessing Share Contents

File shares are backed by S3 buckets with a fs+ prefix. Access them via:

S3 API:

aws s3 ls s3://fs+team-files/ --endpoint-url https://this.tm:9000

NFS Mount:

sudo mount -t nfs this.tm:/team-files /mnt/team-files

S3 Explorer: Browse in the admin panel's Data tab.

See the NFS documentation for detailed mount instructions.

Soft Delete (Tombstoning)

Deleted file shares are tombstoned rather than immediately removed:

• Share data is retained for tombstone_retention_days (default: 90)
• Recreating a share with the same name restores the previous content
• After the retention period, data is permanently purged

Using with AWS CLI

Configure the AWS CLI to use your mesh S3:

# Configure credentials
aws configure --profile tunnelmesh
# Enter your access key and secret key

# Set endpoint
export AWS_ENDPOINT_URL=https://this.tm:9000

# List buckets
aws s3 ls --profile tunnelmesh --endpoint-url $AWS_ENDPOINT_URL

# Upload a file
aws s3 cp file.txt s3://my-bucket/file.txt --profile tunnelmesh --endpoint-url $AWS_ENDPOINT_URL

Using with SDKs

Go

import (
    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/s3"
)

cfg, _ := config.LoadDefaultConfig(context.TODO(),
    config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(
        accessKey, secretKey, "",
    )),
)

client := s3.NewFromConfig(cfg, func(o *s3.Options) {
    o.BaseEndpoint = aws.String("https://this.tm:9000")
    o.UsePathStyle = true
})

Python (boto3)

import boto3

s3 = boto3.client('s3',
    endpoint_url='https://this.tm:9000',
    aws_access_key_id='ACCESS_KEY',
    aws_secret_access_key='SECRET_KEY',
)

# List buckets
response = s3.list_buckets()

Storage Quotas

When max_size is configured, the service enforces storage limits:

• Object uploads are rejected if quota would be exceeded
• Quota is tracked per-bucket and total
• Stats available via tunnelmesh bucket info

Check quota usage:

tunnelmesh storage status
# Output:
# Total: 45.2 GB / 100 GB (45.2%)
# Buckets:
#   my-bucket:     30.1 GB
#   backups:       15.1 GB

Data Persistence

All S3 data is stored in the configured data_dir:

{data_dir}/
  buckets/
    {bucket}/
      _meta.json           # Bucket metadata
      objects/
        {key}              # Object data
      meta/
        {key}.json         # Object metadata

Backup

To backup S3 data, copy the entire data_dir:

# Stop coordinator first
sudo systemctl stop tunnelmesh

# Backup data
tar -czf s3-backup-$(date +%Y%m%d).tar.gz /var/lib/tunnelmesh/s3

# Restart
sudo systemctl start tunnelmesh

Restore

tar -xzf s3-backup.tar.gz -C /
systemctl restart tunnelmesh-server

TunnelMesh is released under the AGPL-3.0 License.