New Concurrency Support in .NET 4

Ben — Fri, 20 Aug 2010 13:59:40 +0000

I’ve been reading about some of the new support for parallelism and concurrency in version 4 of the .NET Framework, and came across a really good paper on parallel design patterns by Steven Toub (most or all of which seems to also be covered in the Parallel Programming with Microsoft .NET book). There are some really convenient additions to the framework in .NET 4 that help make parallelism easier and less error-prone. The point of this post is to illuminate a good example of how this is so.

About a year ago, I was working on a class library that was supposed to do some filtering on streams – something along the lines of Boost.Iostreams which gives you the ability to create pluggable, customized “sources” and “sinks” (producers and consumers) and filters set up in a stream pipeline. One simple application of such a pipeline (and apparently Toub’s favorite) is that of compressing and encrypting some data.

Input is read from the source stream, passed through the filters, and written to the output stream.

I found Toub’s MSDN Magazine article describing how one could parallelize this, and incorporated the sample into my class library. The basic idea is that you set up a blocking queue of chunks of data, and then use that queue (or queues) to connect the inputs and outputs of the streams and filters, and just let it fly. The queue serves as the synchronization mechanism, fulfilling the producer role required by the input side of each piece, and fulfilling the consumer role required by the output side of each piece. Below is the sample code.

public class BlockingStream : Stream
{
    private object _lockForRead;
    private object _lockForAll;
    private Queue _chunks;
    private byte[] _currentChunk;
    private int _currentChunkPosition;
    private ManualResetEvent _doneWriting;
    private ManualResetEvent _dataAvailable;
    private WaitHandle[] _events;
    private int _doneWritingHandleIndex;
    private volatile bool _illegalToWrite;
 
    public BlockingStream()
    {
        _chunks = new Queue();
        _doneWriting = new ManualResetEvent(false);
        _dataAvailable = new ManualResetEvent(false);
        _events = new WaitHandle[] { _dataAvailable, _doneWriting };
        _doneWritingHandleIndex = 1;
        _lockForRead = new object();
        _lockForAll = new object();
    }
 
    public override bool CanRead { get { return true; } }
    public override bool CanSeek { get { return false; } }
    public override bool CanWrite { get { return !_illegalToWrite; } }
 
    public override void Flush() { }
    public override long Length {
        get { throw new NotSupportedException(); } }
    public override long Position {
        get { throw new NotSupportedException(); }
        set { throw new NotSupportedException(); } }
    public override long Seek(long offset, SeekOrigin origin) {
        throw new NotSupportedException(); }
    public override void SetLength(long value) {
        throw new NotSupportedException(); }
 
    public override int Read(byte[] buffer, int offset, int count)
    {
        if (buffer == null) throw new ArgumentNullException("buffer");
        if (offset &lt; 0 || offset &gt;= buffer.Length)
            throw new ArgumentOutOfRangeException("offset");
        if (count &lt; 0 || offset + count &gt; buffer.Length)
            throw new ArgumentOutOfRangeException("count");
        if (_dataAvailable == null)
            throw new ObjectDisposedException(GetType().Name);
 
        if (count == 0) return 0;
 
        while (true)
        {
            int handleIndex = WaitHandle.WaitAny(_events);
            lock (_lockForRead)
            {
                lock (_lockForAll)
                {
                    if (_currentChunk == null)
                    {
                        if (_chunks.Count == 0)
                        {
                            if (handleIndex == _doneWritingHandleIndex)
                                return 0;
                            else continue;
                        }
                        _currentChunk = _chunks.Dequeue();
                        _currentChunkPosition = 0;
                    }
                }
 
                int bytesAvailable =
                    _currentChunk.Length - _currentChunkPosition;
                int bytesToCopy;
                if (bytesAvailable &gt; count)
                {
                    bytesToCopy = count;
                    Buffer.BlockCopy(_currentChunk, _currentChunkPosition,
                        buffer, offset, count);
                    _currentChunkPosition += count;
                }
                else
                {
                    bytesToCopy = bytesAvailable;
                    Buffer.BlockCopy(_currentChunk, _currentChunkPosition,
                        buffer, offset, bytesToCopy);
                    _currentChunk = null;
                    _currentChunkPosition = 0;
                    lock (_lockForAll)
                    {
                        if (_chunks.Count == 0) _dataAvailable.Reset();
                    }
                }
                return bytesToCopy;
            }
        }
    }
 
    public override void Write(byte[] buffer, int offset, int count)
    {
        if (buffer == null) throw new ArgumentNullException("buffer");
        if (offset &lt; 0 || offset &gt;= buffer.Length)
            throw new ArgumentOutOfRangeException("offset");
        if (count &lt; 0 || offset + count &gt; buffer.Length)
            throw new ArgumentOutOfRangeException("count");
        if (_dataAvailable == null)
            throw new ObjectDisposedException(GetType().Name);
 
        if (count == 0) return;
 
        byte[] chunk = new byte[count];
        Buffer.BlockCopy(buffer, offset, chunk, 0, count);
        lock (_lockForAll)
        {
            if (_illegalToWrite)
                throw new InvalidOperationException(
                    "Writing has already been completed.");
            _chunks.Enqueue(chunk);
            _dataAvailable.Set();
        }
    }
 
    public void SetEndOfStream()
    {
        if (_dataAvailable == null)
            throw new ObjectDisposedException(GetType().Name);
        lock (_lockForAll)
        {
            _illegalToWrite = true;
            _doneWriting.Set();
        }
    }
 
    public override void Close()
    {
        base.Close();
        if (_dataAvailable != null)
        {
            _dataAvailable.Close();
            _dataAvailable = null;
        }
        if (_doneWriting != null)
        {
            _doneWriting.Close();
            _doneWriting = null;
        }
    }
}

As you can observe in the above, most of the work in the design of Toub’s BlockingStream class is invested in the synchronization mechanisms. The queue that stores up blocks of data to be worked on is a simple .NET Queue, which is not thread-safe for this application. The Read and Write operations use several synchronization primitives to prevent each component in the pipeline from stomping on the data needed by the others. When you just read the above code from top to bottom, it seems fairly straight forward, but designing something even as simple as this takes some thought. It is a non-trivial design, particularly for the inexperienced. Another downside to the BlockingStream class is that it doesn’t actually get you concurrency – it just gets you synchronization. You still have to fit it into your concurrency designs, whatever they may be. Toub does provide a StreamPipeline class to help you chain these things together.

public class StreamPipeline : IDisposable
{
    private Action[] _filters;
    private List _blockingStreams;
 
    public StreamPipeline(params Action[] filters)
    {
        if (filters == null) throw new ArgumentNullException("filters");
        if (filters.Length == 0 || Array.IndexOf(filters, null) &gt;= 0)
            throw new ArgumentException("filters");
 
        _filters = filters;
 
        _blockingStreams = new List(_filters.Length - 1);
        for (int i = 0; i &lt; filters.Length-1; i++)
        {
            _blockingStreams.Add(new BlockingStream());
        }
    }
 
    public void Run(Stream input, Stream output)
    {
        if (_blockingStreams == null)
            throw new ObjectDisposedException(GetType().Name);
        if (input == null) throw new ArgumentNullException("input");
        if (!input.CanRead) throw new ArgumentException("input");
        if (output == null) throw new ArgumentNullException("output");
        if (!output.CanWrite) throw new ArgumentException("output");
 
        ThreadStart lastStage = null;
        for (int i = 0; i &lt; _filters.Length; i++)
        {
            Stream stageInput = i == 0 ? input : _blockingStreams[i - 1];
            Stream stageOutput =
                i == _filters.Length - 1 ? output : _blockingStreams[i];
            Action filter = _filters[i];
            ThreadStart stage = delegate
            {
                filter(stageInput, stageOutput);
                if (stageOutput is BlockingStream)
                    ((BlockingStream)stageOutput).SetEndOfStream();
            };
            if (i &lt; _filters.Length - 1)
            {
                Thread t = new Thread(stage);
                t.IsBackground = true;
                t.Start();
            }
            else lastStage = stage;
        }
        lastStage();
    }
 
    public void Dispose()
    {
        if (_blockingStreams != null)
        {
            foreach (BlockingStream stream in _blockingStreams)
                stream.Dispose();
            _blockingStreams = null;
        }
    }
}

Again, note how much of this code is devoted to managing Threads and synchronization issues.

Enter .NET 4. The above class along with the concurrency you want can be more succinctly and transparently implemented using the Task and BlockingCollection classes.

using System;
using System.IO;
using System.Collections.Concurrent;
using System.Threading.Tasks;
 
namespace Git
{
    class TransferStream : Stream
    {
        private Stream m_WriteableStream;
        private BlockingCollection m_Blocks;
        private Task m_ProcessingTask;
 
        public TransferStream(Stream writeableStream)
        {
            // TODO validate arguments
            m_WriteableStream = writeableStream;
            m_Blocks = new BlockingCollection();
            m_ProcessingTask = Task.Factory.StartNew(() =&gt;
                {
                    foreach (var block in m_Blocks.GetConsumingEnumerable())
                    {
                        m_WriteableStream.Write(block, 0, block.Length);
                    }
                }, TaskCreationOptions.PreferFairness);
        }
 
        public override bool CanWrite
        {
            // TODO: check state of output stream
            get { return true; }
        }       
 
        public override void Write(byte[] buffer, int offset, int count)
        {
            var block = new byte[count];
            Buffer.BlockCopy(buffer, offset, block, 0, count);
            m_Blocks.Add(block);
        }
 
        public override void Close()
        {
            m_Blocks.CompleteAdding();
            try { m_ProcessingTask.Wait(); }
            finally { base.Close(); }
        }
 
        public override bool CanRead
        {
            get { throw new NotImplementedException(); }
        }
 
        public override bool CanSeek
        {
            get { throw new NotImplementedException(); }
        }
 
        public override void Flush()
        {
            throw new NotImplementedException();
        }
 
        public override long Length
        {
            get { throw new NotImplementedException(); }
        }
 
        public override long Position
        {
            get
            {
                throw new NotImplementedException();
            }
            set
            {
                throw new NotImplementedException();
            }
        }
 
        public override int Read(byte[] buffer, int offset, int count)
        {
            throw new NotImplementedException();
        }
 
        public override long Seek(long offset, SeekOrigin origin)
        {
            throw new NotImplementedException();
        }
 
        public override void SetLength(long value)
        {
            throw new NotImplementedException();
        }
    }
}

You can see by comparing the two that all the work done before to accomplish synchronization has been abstracted by the BlockingCollection, and we now get asynchronous pipelining through the use of the Task class.

Now this class can be used in an implementation like this:

static void FilterStream(Stream input, Stream output, Stream filter)
{
    using (var t1 = new TransferStream(output))
    using (var t0 = new TransferStream(filter))
    {
        input.CopyTo(t0);
    }
}

Using this pattern, it would be a trivial task to create a new StreamPipeline class whose sole purpose would be to provide a little syntactic sugar, making usage more like this:

static void FilterStream(Stream input, Stream output, Stream filter)
{
    StreamPipeline pipeline = new StreamPipeline(input, output, filter);
    pipeline.Run();
}

One thing you might notice about the TransferStream class. Firstly, a bunch of the required Stream methods aren’t really implemented. In our contrived example, that might be alright, but one might want to implement all the methods for production use in a class library.

bloggo ergo sum

New Concurrency Support in .NET 4