Use ZStandard with .NET Core + Docker (Alpine)


In a .NET Core + Docker (Alpine) environment, there is a requirement to throw data compressed with ZStandard to API. In this article, I will explain how to use ZstdNet to compress data compressed with ZStandard via HTTP.

ZStandard (zstd) and .NET Core

ZStandard is a library open sourced by facebook, and it seems to be a high-speed compression algorithm that realizes a high compression rate. The ZStandard Home Page introduces the following three libraries that can be used with .NET.

This time, we will use ZstdNet, which has the most stars on github.

Compression and HTTP requests

The code itself is compressed with the following code for the time being, referring to the sample code in ZstdNet github. The argument to the constructor of the CompressionOptions class is the compression level, which can be specified in the range 1-19 (default is 3).

ReadOnlySpan<byte> Compress(string text)
    var options = new CompressionOptions(16);
    var bytes = Encoding.UTF8.GetBytes(text);
    using var compressor = new Compressor();
    return compressor.Wrap(bytes);

After that, set the content-type of the compressed data to zstd and send it.

var requestContent = new ByteArrayContent(Compress(json).ToArray());
requestContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");
requestContent.Headers.Add("Content-Encoding", "zstd");

var response = await _httpClient.PostAsync(uri, requestContent);

Adjust ZstdNet to work with Docker.

If you try to run it as it is with the Dockerfile created by Visual Studio, you will get the exception that libzstd is not found as shown below.

zstd net Unable to load shared library 'libzstd' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: Error loading shared library liblibzstd: No such file or directory

ZstdNet is only a wrapper for the native library libzstd, so if you want to run it in a non-Windows environment, you need to install it separately (libzstd.dll for Windows is installed with nuget). In the case of Alpine, the package zstd-libs is published in the main repository, so I will apk add it.

Also, when the above package is added, two files, and, are placed in the/usr/lib/directory, but ZstdNet is libzstd. I'm going to look for a native library with the name .so, so I'll create a symbolic link and trick it.

The Dockerfile looks like this:

FROM AS base
RUN apk add --update --no-cache zstd-libs && \
    ln /usr/lib/ /usr/lib/

FROM AS build
COPY ["ZstdLibSample/ZstdLibSample.csproj", "ZstdLibSample/"]
RUN dotnet restore "ZstdLibSample/ZstdLibSample.csproj"
COPY . .
WORKDIR "/src/ZstdLibSample"
RUN dotnet build "ZstdLibSample.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "ZstdLibSample.csproj" -c Release -o /app/publish

FROM base AS final
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "ZstdLibSample.dll"]

As far as I can see the communication telegram, it seems that it is compressed properly.


--If you want to compress using ZStandard in .NET Core, it's easy to use ZstdNet --ZstdNet uses a native library, so let's install it separately with apk --ZstdNet will search for native libraries in /usr/lib/, so let's trick them with symlinks.

Recommended Posts

Use ZStandard with .NET Core + Docker (Alpine)
Use Puphpeteer with Docker
Use ngrok with Docker
Use GDAL with Python with Docker
Run TAO Core with Docker
[Docker] Use whenever with Docker + Rails
Why use orchestration tools with Docker
How to use mssql-tools with alpine
Use cuda11.0 with pytorch using Docker
[Rails] How to use rails console with docker
Use Symbolic Link with Docker multi-stage builds
Use Amazon ECR Credential Helper with Docker Desktop
How to use docker compose with NVIDIA Jetson
Use ProGuard with Gradle
Launch MariaDB with Docker
Run Pico with docker
Explode Docker with WSL2
Why we use Docker
Use XVim2 with Xcode 12.0.1
Use CentOS with LXD
Operate Emby with Docker
Try WildFly with Docker
Run Payara with Docker
Use webmock with Rspec
[Docker] Connection with MySQL
Php settings with Docker
Getting Started with Docker
Use WebJars with Gradle
Use jlink with gradle
Disposable PHP with Docker
Build bazel with alpine
Install Composer with Docker
[PHP8] Install and use PECL YAML function (YAML parser) with Docker
Use docker's in-container shell with cygwin [docker exec -it bash]
How to use mysql with M1 mac Docker preview version
You are required to use winpty with docker exec [Windows]