Storage building block

The Storage block is the kit’s blob-storage abstraction. One interface (IStorageService), two implementations - LocalStorageService (filesystem) and S3StorageService (AWS S3 and any S3-compatible store - the kit runs RustFS locally) - with presigned URL support and an optional QuotaMeteredStorageService decorator that meters per-tenant usage against the Quota block.

What it ships

Extensions

• AddHeroLocalFileStorage(services) - registers LocalStorageService for wwwroot-based file I/O. Useful for dev.
• AddHeroStorage(services, configuration) - reads Storage:Provider eagerly at registration time. "s3" registers a shared IAmazonS3 client + S3StorageService; anything else falls back to LocalStorageService. Wraps the chosen provider with QuotaMeteredStorageService when QuotaOptions:Enabled is true (also read at registration).

Interface

public interface IStorageService
{
    // Returns the stored object's relative path/key
    Task<string> UploadAsync<T>(FileUploadRequest request, FileType fileType,
        CancellationToken cancellationToken = default) where T : class;

    Task<FileDownloadResponse?> DownloadAsync(string path, CancellationToken cancellationToken = default);
    Task<bool> ExistsAsync(string path, CancellationToken cancellationToken = default);
    Task<long> GetSizeAsync(string path, CancellationToken cancellationToken = default);   // 0 if missing
    Task RemoveAsync(string path, CancellationToken cancellationToken = default);   // swallows store errors

    // Server-side copy, overwriting the destination; the source stays. Throws on failure (incl. a missing source)
    Task CopyAsync(string sourceKey, string destinationKey, CancellationToken cancellationToken = default);

    Task<PresignedUploadUrl> GenerateUploadUrlAsync(
        string storageKey, string contentType, long maxBytes, TimeSpan ttl,
        CancellationToken cancellationToken = default);

    Task<Uri> GenerateDownloadUrlAsync(
        string storageKey, TimeSpan ttl, string? responseContentDisposition = null,
        CancellationToken cancellationToken = default);

    Task<StoredObjectMetadata?> HeadObjectAsync(string storageKey, CancellationToken cancellationToken = default);

    string BuildPublicUrl(string storageKey);   // durable non-expiring URL (or server-relative path on local)
}

Implementations

• LocalStorageService - stores under wwwroot/uploads/{owner-type}/{guid}_{sanitized-filename} and validates extension + size against FileTypeMetadata.GetRules(fileType). CopyAsync is a file copy that throws FileNotFoundException when the source is missing. Presigning is a dev-only token fallback: GenerateUploadUrlAsync issues a local://upload/{token} URL backed by LocalPresignTokenStore; GenerateDownloadUrlAsync and BuildPublicUrl return server-relative /uploads/... paths (no signing needed).
• S3StorageService - uses AWSSDK.S3 with a singleton IAmazonS3 client for every real S3 call. UploadAsync<T> writes under public/uploads/{owner-type}/..., because its result is always used as a public URL; CopyAsync is an S3 CopyObject within the bucket (content type and metadata carry over). A custom ServiceUrl (RustFS, MinIO, etc.) switches to path-style addressing per ForcePathStyle; presigned PUT/GET URLs come from the SDK’s request signer, through a second, keyed IAmazonS3 (S3StorageService.PresignClientKey) aimed at PresignServiceUrl when it is set and at ServiceUrl otherwise. Signing itself is offline; the only network call that client can make is fetching ambient credentials when no keys are set.
• QuotaMeteredStorageService - decorator. CheckAndRecordAsync(tenantId, QuotaResource.StorageBytes, bytes, ct) on upload (throws 507 when exceeded, rolls back the charge if the write fails). CopyAsync charges the copied object’s size without a quota check, so a move (copy, then delete) is net-zero and never fails because the tenant is at its limit. RemoveAsync refunds the object’s size only when the object is gone afterwards: providers swallow delete errors, so a failed delete keeps both the bytes and the charge. Requests with no resolved tenant pass through unmetered.

Request / response

• FileUploadRequest - FileName, ContentType, Data.
• FileDownloadResponse - Stream, ContentType, FileName, ContentLength?.
• PresignedUploadUrl - Url, RequiredHeaders (headers the browser must send verbatim, e.g. Content-Type), ExpiresAt.
• StoredObjectMetadata - SizeBytes, ContentType, LastModified, ETag (from HEAD).
• FileType enum - Image, Document, Pdf; FileTypeMetadata maps each to an extension whitelist + max size.

Options

• S3StorageOptions - Bucket, Region, Prefix, PublicRead (default true), PublicBaseUrl (for non-expiring public URLs), ServiceUrl (custom endpoint for RustFS, MinIO, etc.), PresignServiceUrl (optional host that presigned upload/download URLs are signed for, when browsers reach the store on a different address than the API does; PublicBaseUrl only rewrites the plain public-read URLs, never a presigned one), AccessKey / SecretKey (leave empty to use the AWS SDK credential chain), ForcePathStyle (applies to each client with a custom endpoint, ServiceUrl or PresignServiceUrl). PresignServiceUrl must be an absolute http(s) URL with no path, checked at startup: a path would be signed into every URL and break behind a proxy.

Key layout and visibility

Visibility is part of the object key. The first segment is a visibility root (StorageVisibilityRoot.Public = public/, StorageVisibilityRoot.Private = private/), and every stack the kit ships grants anonymous s3:GetObject on public/* only. An object is publicly readable exactly when its key starts with public/; everything else needs a presigned URL.

StorageKeyBuilder is the one place that decides a Files key; don’t build one by hand. Changing a file’s visibility moves the object to the other root - see Files module.

Anonymous read per stack

PublicBaseUrl

BuildPublicUrl(key) returns {PublicBaseUrl}/{key} when PublicBaseUrl is set. Without it, a client with a ServiceUrl falls back to {ServiceUrl}/{Bucket}/{key}, and plain AWS to the bucket’s s3.amazonaws.com URL. Set it to the base a browser reaches that maps to the bucket root: for a path-style store that includes the bucket name (https://s3.example.com/fsh), for CloudFront it is the distribution’s domain. The Compose stack talks to RustFS as http://rustfs:9000, so without PublicBaseUrl it would hand out public URLs no browser can resolve.

Storage:S3:Prefix

With a prefix set, every key is written as {Prefix}/{key}, so public objects land under {Prefix}/public/.... Two rules follow:

• The bucket policy (or app_s3_public_read_prefix) must cover {Prefix}/public/*, not public/*.
• The prefix must not be public or private. A key that already starts with the prefix is not prefixed again, so with Prefix = public a private/... key is stored as public/private/... - anonymously readable. Nothing validates this at startup.

Local provider

LocalStorageService serves everything under wwwroot statically. It does not enforce private/ at all: a private file is as readable as a public one to anyone who has the path. It is for dev and tests only.

How modules consume Storage

The Files module is the canonical consumer: its request-upload-url endpoint mints presigned PUT URLs, the finalize handler verifies size + content type via HeadObjectAsync before a row leaves PendingUpload, and purge jobs clear orphans.

Direct usage (without the Files module) looks like:

public sealed class UploadAvatarHandler(IStorageService storage, ICurrentUser current)
    : ICommandHandler<UploadAvatarCommand, string>   // returns the public URL
{
    public async ValueTask<string> Handle(UploadAvatarCommand cmd, CancellationToken ct)
    {
        var storageKey = await storage.UploadAsync<UserAvatar>(
            new FileUploadRequest
            {
                FileName = $"{current.GetUserId()}-avatar.png",
                ContentType = "image/png",
                Data = cmd.Data,
            },
            FileType.Image,
            ct).ConfigureAwait(false);

        return storage.BuildPublicUrl(storageKey);
    }
}

The generic T names the owning type and becomes the folder segment (public/uploads/useravatar/... on S3, uploads/useravatar/... on local storage).

Configuration

{
  "Storage": {
    "Provider": "s3",                              // or "local"
    "S3": {
      "Bucket": "fsh-uploads",
      "ServiceUrl": "http://rustfs:9000",          // omit for AWS S3 default
      "PresignServiceUrl": "https://s3.example.com", // host browsers reach; omit when ServiceUrl is already public
      "Region": "us-east-1",
      "ForcePathStyle": true,                      // required for RustFS / MinIO
      "AccessKey": "rustfsadmin",
      "SecretKey": "set-via-secrets",
      "PublicBaseUrl": "https://cdn.example.com"   // browser-reachable base for public/* objects
    }
  }
}

How to extend

Add Azure Blob Storage

Implement IStorageService against the Azure SDK and register it in place of the S3 implementation:

services.AddSingleton<IStorageService, AzureBlobStorageService>();

Wrap with the quota decorator if you want:

services.Decorate<IStorageService, QuotaMeteredStorageService>();

(The kit doesn’t ship Scrutor; copy its Decorate pattern or register manually with IConfigureOptions.)

Use AWS IAM role for credentials

Omit AccessKey and SecretKey from S3StorageOptions; the AWS SDK falls through to the instance-profile / EKS-pod-identity / web-identity chain automatically.

Scan files post-upload

The Files module ships an IFileScanner hook. If you’re using IStorageService directly without the Files module, kick off the scan in the same handler that finalises the upload.

Gotchas

• Self-hosted S3 stores (RustFS, MinIO) need ForcePathStyle = true. Virtual-hosted-style addressing puts the bucket name in the subdomain, which they can’t service without DNS gymnastics. The option defaults to false and only takes effect when ServiceUrl is set - set it explicitly for RustFS, MinIO, and other self-hosted S3-compatible services.
• A presigned URL only works on the host it was signed for. SigV4 signs the Host header, so the URL always points at the endpoint of the client that signed it. When the API reaches the store on an internal address a browser cannot resolve (the Docker Compose stack uses http://rustfs:9000), set PresignServiceUrl to the public one: presigned PUT and GET URLs are signed for it, their scheme follows it (http or https), and every other S3 call keeps using ServiceUrl. The host refuses to start when it is set but is not an absolute http(s) URL; left empty, presigning uses ServiceUrl exactly as before. A reverse proxy in front of the public endpoint must forward the Host header unchanged, or the store answers 403 SignatureDoesNotMatch, and the store’s CORS has to allow the front-end origins, because the browser PUTs to it cross-origin.
• Presigned URLs have a TTL. Once it expires, the URL is dead. Use BuildPublicUrl for non-expiring public URLs; they only work for keys under public/ (or {Prefix}/public/), because that is all the bucket policy opens.
• CopyAsync throws, RemoveAsync doesn’t. Move an object as copy, then update the reference, then delete, and confirm the delete with ExistsAsync when it matters: RemoveAsync logs and swallows store errors.
• Docker Compose publishes the S3 port on all interfaces. Public files are served straight from FSH_S3_PUBLIC_URL, so the store has to be reachable, but only through your proxy: firewall FSH_S3_PORT, or bind the rustfs port mapping to loopback ("127.0.0.1:${FSH_S3_PORT:-9000}:9000") when the proxy runs on the same host.
• QuotaMeteredStorageService is scoped. It depends on IQuotaService which is scoped per request. Don’t resolve it from a singleton or a hosted service without creating a scope.
• Local storage paths are not tenant-segmented, and private/ is not enforced. UploadAsync files land under wwwroot/uploads/{owner-type}/, and everything under wwwroot - private/ keys included - is served statically without policy enforcement. Don’t use LocalStorageService in multi-tenant production - it exists for dev and tests.
• AddHeroStorage reads configuration eagerly. The provider choice and the quota toggle are evaluated once, at registration. Integration tests that swap config after host build must re-register/rewire IStorageService post-registration - changing Storage:Provider later does nothing. A factory that re-registers the S3 stack this way also has to register the keyed presign client (S3StorageService.PresignClientKey, as FshWebApplicationFactory does), or S3StorageService fails to resolve.

Critical files

• src/BuildingBlocks/Storage/Extensions.cs
• src/BuildingBlocks/Storage/Services/IStorageService.cs
• src/BuildingBlocks/Storage/StorageVisibilityRoot.cs
• src/BuildingBlocks/Storage/QuotaMeteredStorageService.cs
• src/BuildingBlocks/Storage/Local/LocalStorageService.cs
• src/BuildingBlocks/Storage/S3/S3StorageService.cs
• src/BuildingBlocks/Storage/S3/S3StorageOptions.cs

Related

• Files module - the policy + lifecycle layer on top of this block.
• Quota - usage metering wrapper.
• Catalog module - product images flow through here.