Beta 3 Is A Magic Number

Today were very happy to announce ImageSharp Beta 3.

ImageSharpReleases

It's been four months since our last beta and we've been very, very busy. Since our last release we have.

  • Created API documentation and additional articles to help developers work with the library.
  • Introduced a full memory management API that allows fine grained control of how ImageSharp manages large buffers.
    • For the default pooling memory management implementation we managed to halve the memory footprint of the library without significant loss in speed.
  • Made codecs easier to use. The performance of the Jpeg decoder is doubled since the first beta!
  • Introduced a new metadata-only decoding API with the help of the community.
  • Introduced new affine transform methods that allow the composition of affine transforms.
  • Refactored all filters (Hue, Brightness, Saturation etc) to match the behaviour described in the W3C specification.
  • Introduced static instances of reusable algorithms for use as method parameters.
  • Refactored the namespaces to logical groups based on functionality.
  • Fixed the resampling processes to correctly blend semi-transparent output.
  • Many other performance improvements and bug fixes.

I'll talk about some of the bigger changes below. While you're reading that, download the updated package from Nuget.

Documentation. #

Our new documentation, built with DocFX, is hosted at https://sixlabors.github.io/docs/. There you will find complete API documentation for all the SixLabors libraries plus additional articles describing the where's and what-how's of ImageProcessing.

Please bear in mind that this documentation is a work-in-progress, being very labor intensive to write. All pages offer edit links so please help out if you spot any errors!

Memory Management #

By default, ImageSharp uses ArrayPools for performance reasons, however, this behavior is fully configurable.
All large buffers within the API are managed by the MemoryManager implementation associated to Configuration classes MemoryManager property. We default to using ArrayPoolMemoryManager in order to utilize the benefits of array pooling.

This means:

  • Less pressure on the GC (Garbage Collector), because buffers are being reused most of the time.
  • Reduced LOH (Large Object Heap) fragmentation.
  • When working with unclean buffers is acceptable, we can spare on array cleaning behavior also.

Pooling helps us to reduce CPU work and increase throughput by the cost of a larger memory footprint.

Working in Memory Constrained Environments #

Sometimes having larger memory footprint is not an option. Let's mention a few cases:

  • When horizontal scaling is achieved by having multiple memory constrained containers in a cloud environment.
  • Mobile applications.

Before scaling down pooling behavior because of unwanted OutOfMemoryException's in a cloud or desktop environment, make sure that you are running your service in a 64 bit process.

There are several pre-defined factory methods to create an ArrayPoolMemoryManager instance for memory constrained environments. For example ArrayPoolMemoryManager.CreateWithModeratePooling might be suitable in most constrained situations:

Configuration.Default.MemoryManager = ArrayPoolMemoryManager.CreateWithModeratePooling();

Of course, you may also configure a MemoryManager on your own Configuration instance.

Filters #

Filters were originally part of the SVG specification. However, when their usefulness became evident, W3C started working on adding some common filter effects to CSS as well.

In beta 3 we've added filter methods by means of the Matrix4x4 struct found in System.Numerics.Vectors.

These methods match the behaviour of many CSS filters. You can use them to blur, brighten or saturate images among other things. They can be used alone or in combination with other filters.

One big advantage with using the filter matrix is that composite filters can be obtained by matrix multiplication; the order of multiplication determining the transformed output.

using (var image = Image.Load("in.png"))
{
    Matrix4x4 brightness = KnownFilterMatrices.CreateBrightnessFilter(0.9F);
    Matrix4x4 hue = KnownFilterMatrices.CreateHueFilter(180F);
    Matrix4x4 saturation = KnownFilterMatrices.CreateSaturateFilter(1.5F);
    Matrix4x4 m = brightness * hue * saturation;

    image.Mutate(i => i.Filter(m));

    image.Save("out.png");
}

In addition to the composable filters we have added convenience methods for each individual filter. For example:

image.Mutate(x => x.Grayscale(GrayscaleMode.Bt709));

image.Mutate(x => x.Hue(-180F));

image.Mutate(x => x.Brightness(0.9F));

Additional methods can be found in the API documentation.

Affine Transforms #

In geometry, an affine transformation is a function between affine spaces which preserves points, straight lines and planes. Also, sets of parallel lines remain parallel after an affine transformation. An affine transformation does not necessarily preserve angles between lines or distances between points, though it does preserve ratios of distances between points lying on a straight line.

Translation, scaling, rotation, and skewing are all classified as affine transforms.

In beta 3 we've added affine transformation methods by means of the Matrix3x2 struct found in System.Numerics.Vectors.

As with filters the big advantage with using the transform matrix is that composite transforms can be obtained by matrix multiplication; the order of multiplication determining the transformed output.

using (var image = Image.Load("in.png"))
{
    Matrix3x2 rotate = Matrix3x2Extensions.CreateRotationDegrees(45F);
    var translate = Matrix3x2.CreateTranslation(5, 10);
    var scale = Matrix3x2.CreateScale(.5F, .5F);
    Matrix3x2 m = rotate * scale * translate;

    image.Mutate(i => i.Transform(m, KnownResamplers.Bicubic));

    image.Save("out.png");
}

In addition to the freeform transforms we have added convenience methods for Rotate and Skew with multiple overloads, that maintain the center point of the transformed image and automatically adjust the output image dimensions to fit.

image.Mutate(x => x.Rotate(45F));

image.Mutate(x => x.Skew(-20F, 10F));

All our transform methods have multiple overloads and accept the same IResampler implementation as Resize to determine the resampling filter to assign during processing.

Metadata-only decoding #

Our new feature is very similar to magick identify from ImageMagick. To enable work with metadata-only results the types IImageInfo and PixelTypeInfo have been itroduced:

using (Stream stream = OpenMyStream())
{
    IImageInfo info = Image.Identify(stream);
    Console.WriteLine("bits/pixel: "  + info.PixelType.BitsPerPixel);
}

RoadMap #

All things going to plan this should be the last beta. We're very pleased with the shape and robustness of the API and we are not planning on making any more large-scale changes. We shall be concentrating now on bug fixes and performance improvements.

We will however be considering:

  • Projective non-affine transforms (Tilt, Warp)
  • Non-Rgb-based TPixel implementations (Cmyk, CieLab etc)
  • Adding a full suite of colorspace conversion algorithms

You Can Help #

ImageSharp is fully open source and built for the community at large. You can help us reach v1.0

We had some fantastic help over the last four months from several community members, I'd like to shout out to, in no particular order:

If I've missed you out here (I deeply apologise if I have) please has a good growl at at me in our Gitter Channel and I'll add you to the list.

We are always looking for assistance optimizing our code so if you've got performance chops please get in touch.

Additionally we've got plenty of tickets that you can help with and we'd love some help checking our spelling and grammar in the documentation, it doesn't matter how little or how much you can offer, we really appreciate every bit of assistance.

You can also really help us by downloading and testing the libraries. We really want to deliver something that is not only performant and robust, but also extremely usable.

Enjoy!