In a .NET Core + Docker (Alpine) environment, there is a requirement to throw data compressed with ZStandard to API. In this article, I will explain how to use ZstdNet to compress data compressed with ZStandard via HTTP.
ZStandard is a library open sourced by facebook, and it seems to be a high-speed compression algorithm that realizes a high compression rate. The ZStandard Home Page introduces the following three libraries that can be used with .NET.
This time, we will use ZstdNet, which has the most stars on github.
The code itself is compressed with the following code for the time being, referring to the sample code in ZstdNet github. The argument to the constructor of the CompressionOptions class is the compression level, which can be specified in the range 1-19 (default is 3).
ReadOnlySpan<byte> Compress(string text)
{
var options = new CompressionOptions(16);
var bytes = Encoding.UTF8.GetBytes(text);
using var compressor = new Compressor();
return compressor.Wrap(bytes);
}
After that, set the content-type of the compressed data to zstd and send it.
var requestContent = new ByteArrayContent(Compress(json).ToArray());
requestContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");
requestContent.Headers.Add("Content-Encoding", "zstd");
var response = await _httpClient.PostAsync(uri, requestContent);
If you try to run it as it is with the Dockerfile created by Visual Studio, you will get the exception that libzstd is not found as shown below.
zstd net Unable to load shared library 'libzstd' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: Error loading shared library liblibzstd: No such file or directory
ZstdNet is only a wrapper for the native library libzstd, so if you want to run it in a non-Windows environment, you need to install it separately (libzstd.dll for Windows is installed with nuget). In the case of Alpine, the package zstd-libs is published in the main repository, so I will apk add it.
Also, when the above package is added, two files, libzstd.so.1 and libzstd.so.1.4.5, are placed in the/usr/lib/directory, but ZstdNet is libzstd. I'm going to look for a native library with the name .so, so I'll create a symbolic link and trick it.
The Dockerfile looks like this:
FROM mcr.microsoft.com/dotnet/core/runtime:3.1-alpine AS base
RUN apk add --update --no-cache zstd-libs && \
ln /usr/lib/libzstd.so.1 /usr/lib/libzstd.so
WORKDIR /app
FROM mcr.microsoft.com/dotnet/core/sdk:3.1-alpine AS build
WORKDIR /src
COPY ["ZstdLibSample/ZstdLibSample.csproj", "ZstdLibSample/"]
RUN dotnet restore "ZstdLibSample/ZstdLibSample.csproj"
COPY . .
WORKDIR "/src/ZstdLibSample"
RUN dotnet build "ZstdLibSample.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "ZstdLibSample.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "ZstdLibSample.dll"]
As far as I can see the communication telegram, it seems that it is compressed properly.
--If you want to compress using ZStandard in .NET Core, it's easy to use ZstdNet --ZstdNet uses a native library, so let's install it separately with apk --ZstdNet will search for native libraries in /usr/lib/libzstd.so, so let's trick them with symlinks.
Recommended Posts