Announcing ImageSharp Beta 1

"O frabjous day! Callooh! Callay!" He chortled in his joy.

I'm brimming with excitement writing this blog post; it's a truly fantastic moment for the ImageSharp team,

We've released our first beta!!

With over 2 1/2 years of development, 4,400+ commits, and the formation of Six Labors from a dedicated team of developers, ImageSharp has come a very long way. I'm immensly proud of everyone involved and would like to thank the community at large for sticking with us for such a long time.

We've built the better part of a full managed 2D Graphics API that is truly cross-platform.

Let that sink in for a moment... Amazing!

Let's have a quick look at what we've delivered so far.

The Current State of Play #

ImageSharp is split into three major libraries:

• ImageSharp - Color and image primitives, formats, filters, transforms and other standard algorithms
• ImageSharp.Drawing - Image, shape, and line drawing primitives and algorithms
• ImageSharp.Web - Extensible middleware for ASP.NET Core projects that allows processing and caching of images

Without the constraints of System.Drawing We have been able to develop an API that is much more flexible, easier to code against, and much, much less prone to memory leaks when consuming.

The ImageSharp API #

The API centers around a single generic image class. Image<TPixel> where TPixel can be one of many supported pixel formats. The generic nature of this design allows excellent flexibility when working with the different formats with each method format-agnostic.

You want Bgr24? Use Image<Bgr24>! Alpha8? Use Image<Alpha8>, nice and easy!

Many common operations have been added to our API and we've designed it to allow these to be performed simply but with great flexibility .

Check out this example resizing and transforming the colors of a jpeg:

// Image.Load(string path) is a shortcut for our default type.
// Other pixel formats use Image.Load<TPixel>(string path)) or one of the many other available overloads.
using (Image<Rgba32> image = Image.Load("foo.jpg"))
{
    image.Mutate(ctx => ctx
         .Resize(image.Width / 2, image.Height / 2)
         .Grayscale());
    image.Save("bar.jpg"); // Automatic encoder selected based on extension.
}

Processing Images with Mutate/Clone and Extension Methods #

We've got the API pretty much locked down but there's a fair amount of cheese moving from the previous alphas. (We apologise for that and salute early testers - You Rock!) The greatest of these changes is moving all our fluent image processing methods to two main operations: Mutate and Clone:

1. Mutate applies a chain of operations on the original image, changing it's state:

image.Mutate(ctx => ctx.Operation1().Operation2() ...)

2. Clone creates a clone of the original image with subsequent operations applied to the new instance:

using (Image<Rgba32> cloneImage = image.Clone(ctx => ctx.Operation1().Operation2() ...))
{
    // Operate on cloneImage
}

The original image is left untouched. It's important to note, that you are responsible for the disposal of both images when creating a clone.

We have a few good reasons behind this API decision:

1. Throughout the alpha API we realised that it wasn't obvious that the extension methods returned the same image instance. Many developers expected image.Resize(...) to return a new resized image, leaving the original untouched. Now we have provided control over this behaviour by distinguishing Mutate() and Clone().
2. The new Clone(...) operation enables optimizations behind the scenes. This might be very useful in use cases where both the original and the modified image instances should be preserved.

For example, this operation ...

var mipmap1 = image.Clone(ctx => ctx.Resize(image.Width / 2, image.Height / 2);

... is cheaper than this one:

var mipmap1 = image.Clone();
mipmap1.Mutate(ctx => ctx.Resize(image.Width / 2, image.Height / 2));

With extension methods operating on IImageProcessingContext<T> we are able to use the same fluent methods with both Mutate and Clone semantics. There's a veritable smörgåsbord of methods to choose from, each designed to handle a full range of use cases.

This all means that there is much less noise around the main Image<T> class now and we think that's much better.

The API has also been designed to provide you the tools to allow you to implement your own algorithms and integrate the library into your own applications.

Direct Access to Image Pixels #

During the alpha we introduced fast pixel manipulation API's utilizing Span<T>, the new low level memory manipulation tool, via image.GetRowSpan(rowIdx) and image.Pixels. However, we faced several issues from exposing these methods publicly, the most difficult to overcome being that System.Memory will be in preview state until the 2.1 release of the .NET Standard.

As such, we decided to internalize all the functions exposing Span<T> for now though we still have API's to directly access pixel data for most common use-cases:

• You can save pixels to a byte[] buffer by calling image.SavePixelData(bytes)
• You can manipulate individual pixels by using indexers on Image<T> and ImageFrame<T>. This is slower than writing Spans of rows, but still much faster than System.Drawing's GetPixel/SetPixel!

We plan to reintroduce the Span<T> methods as soon as the first official release of System.Memory lands, providing a modern API for advanced users with low-level pixel manipulation needs.

Image Formats #

Out of the box we support decoding and encoding of bmp, png, gif, and jpeg formats with tiff well underway. We've provided increased functionality for several formats (animation, quantization, compression) and extensible endpoints for implementing your own image formats.

For beta-1 we managed to solve all the most difficult issues affecting our Jpeg decoder:

• Improved accuracy and standard-conformity. For most cases our output pixels are ~99.9% similar when compared to libjpeg-turbo output.
  Update: An accuracy bug has been managed to sneak into beta-1, will be fixed in the next release!
• Significantly reduced memory consumption.
• We are capable of decoding many kinds of broken Jpegs, and manage errors properly in the worst cases.
• We achieved the previous points without performance drops! Although parallel processing is disabled in Jpeg decoder for now, single-core performance has been improved since alpha releases thanks to several SIMD optimizations. We think that the lack of parallelism should not be an issue for server users, because throughput of high-load image processing services depends mostly on single core performance. However, multi-core Jpeg processing might be important for desktop/mobile users, so we plan to re-add it in future releases.

We've also implemented support for EXIF and ICC metadata.

Drawing #

We support a full suite of shape, path and image drawing algorithms from complex polygons, simple straight lines, to image blending. All of which support the full range of Porter-Duff compositing operators

Check out this example of loading an image, using the drawing API to draw a rectangle and blend it over the source images using the source blending mode. This will generate an image that looks like a frame with the outer edge 10 pixels from the source image.

using (Image<Rgba32> image = Image.Load("foo.jpg"))
{
    var borderSize = 10;
    image.Mutate(ctx => ctx
         .Fill(Rgba32.Transparent,
           new RectangleF(borderSize, borderSize, image.Width - (borderSize * 2), image.Height - (borderSize * 2)),
           new GraphicsOptions(true)
           {
               BlenderMode = PixelBlenderMode.Src // Enforces that any part of this shape that has color is punched out of the background
           }));
    image.Save("bar.jpg"); // Automatic encoder selected based on extension.
}

So What's Next? #

Future work will come at a much faster cadence. We feel like we've done most of the difficult work now and will be able to concentrate on performance, bug fixes, and API tweaks.

Roadmap #

The latest planned date for a v1.0 Final release is Q1 2018. This lines up with the scheduled release date of Net Standard 2.1 We're lining up with that release as we want to introduce fast API endpoints utilizing the System.Memory classes. If we are able to release before then we will do.

In the interim we will:

• Perform optimization of all our image formats introducing SIMD where required
• Add missing features where we can to our image formats
• Perform optimization on any manipulations algorithms where required
• Implement any missing sanitation code
• Expose a full suite of ColorSpace conversion algorithms
• Tweak public APIs to ensure they are as usable as possible
• Apply any bug fixes.

Additionaly we shall create full API documentation for the library as well as and expanded set of sample projects within the Github repository to demonstrate the functionality we offer.

You Can Help #

ImageSharp is fully open source and built for the community at large. You can help us reach v1.0

We are always looking for assistance optimizing our code so if you've got performance chops please get in touch.

Additionaly we've got plenty of tickets that you can help with and we'd love some help getting documentation together, it doesn't matter how little or how much you can offer, we really appreciate every bit of assistance.

You can also really help us by downloading and testing the libraries. We really want to deliver something that is not only performant and robust, but also extremely useable.

Cheers!

P.S. Follow us on Twitter!

• Next: Beta 3 Is A Magic Number
• Previous: A Taste of Things to Come.