Questions tagged [tpl-dataflow]

TPL Dataflow (TDF) is a .NET library for building concurrent applications. It promotes actor/agent-oriented designs through primitives for in-process message passing, dataflow, and pipelining. TDF builds upon the TPL (Task Parallel Library) in .NET 4 and integrates with async language support in C#, Visual Basic, and F#. TDF lacks join/merge by key (like SSIS) and time-based windowing (available in Rx).

Filter by
Sorted by
Tagged with
4
votes
1answer
104 views

Handle stream data with Dataflow

I have a websocket connection with 3-rd party API. The API returns a lot of data, in a peak hours it returns hundreds messages per second. I need to process the data for two purposes: save everything ...
1
vote
1answer
227 views

Infinite producer using BlockingCollection [closed]

Please have a review of this Code chunk and let me know whether it is a good way for infinite Producer? ...
1
vote
1answer
56 views

Dispatch multiple publishers to single handler with result

This code defines a MessageBus used to handle messages from many producers to a single consumer. The bus allows multiple publishers to send messages asynchronously ...
11
votes
1answer
356 views

Testing a list of proxies for validity concurrently

I have a class that tests a list of proxies for validity concurrently, which I have created in 2 different ways, one utilizing Parallel.ForEach and the other using a TPL Dataflow ActionBlock. I am ...
2
votes
1answer
407 views

C# BufferBlock that keeps unique items

In our application we watch file changes using .NET FileSystemWatcher. After each file change we process the new content asynchronously. In order to make everything ...
2
votes
0answers
458 views

Producer Consumer Using TPL Dataflow

I have written the following class which is supposed to act as a consumer of messages that are passed in from producers and forward them onto amazon SQS, I have made a compromise by starting the ...
5
votes
2answers
697 views

Task Parallel Library and Mailkit

I have a use case where I need to process about 80,000 emails per day. The client has an existing process that takes about 1/sec per email and is running out of time before the next batch comes in. ...
3
votes
1answer
1k views

Formatting a CAN bus message as a string

I am writing a CAN bus logger application and need to format messages as quick as possible in order to prevent buffer overflows on our device. In some cases I am quick enough, but under heavy loads it ...
4
votes
1answer
1k views

Rewriting to single pipeline by using data flow blocks

I have a number of messages with a payload as a list of items. My goal is to transform the message payload without losing the relation between the message and payloads. Isn't it possible to rewrite ...
1
vote
1answer
1k views

Implementation of self-updating, asynchronous cache solution of individual (non-bulk) objects

Here I am again; referring to my previous code review request about a self-updating, asynchronous (?) cache. The previous class takes care of caching operations like "get all users registered in our ...
5
votes
2answers
2k views

High udpclient CPU use

I have a bunch of different ports I want to listen on and the iterate over the available data to send to a Dataflow pipeline. In total I'm listening on 14 ports. I'm looking for any advice on how to ...
2
votes
1answer
688 views

Batcher using Microsoft.Tpl.Dataflow

What do you think? In particular, I'm not sure about my exception handling approach to raise errors when posting objects; but I couldn't come up with anything better. ...
5
votes
1answer
4k views

TPL Dataflow BatchBlock with Batch Size & Timeout

I needed BatchBlock with support for timeout (act greedy on what it received at that time) and by some search and study, this is the outcome. What are drawbacks or ...