Questions tagged [tpl-dataflow]

TPL Dataflow (TDF) is a .NET library for building concurrent applications. It promotes actor/agent-oriented designs through primitives for in-process message passing, dataflow, and pipelining. TDF builds upon the TPL (Task Parallel Library) in .NET 4 and integrates with async language support in C#, Visual Basic, and F#. TDF lacks join/merge by key (like SSIS) and time-based windowing (available in Rx).

Filter by
Sorted by
Tagged with
1
vote
0answers
46 views

"streaming" messages to a broker from a batch-like source (eg sql database), optionally acknowledge back to the source

I am writing library code to help other developers send messages to a pub-sub event broker. In many cases, the message source will not be "stream-like". Specifically, most messages are ...
2
votes
0answers
86 views

Handle ASP.NET WebSocket connections using TPL Dataflow

I was recently introduced to the wonders of TPL Dataflow and thought it would be a good exercise to create a wrapper class which wraps around an ASP.NET WebSocket ...
6
votes
0answers
276 views

TPL Dataflow - Timer Action Block

I'm new to TPL Dataflow and was looking for an action block to basically push an object on a timer - specifically to produce heartbeats on every interval. I was unable to find anything out of the box ...
5
votes
1answer
593 views

Handle stream data with Dataflow

I have a websocket connection with 3-rd party API. The API returns a lot of data, in a peak hours it returns hundreds messages per second. I need to process the data for two purposes: save everything ...
1
vote
1answer
352 views

Infinite producer using BlockingCollection [closed]

Please have a review of this Code chunk and let me know whether it is a good way for infinite Producer? ...
1
vote
1answer
65 views

Dispatch multiple publishers to single handler with result

This code defines a MessageBus used to handle messages from many producers to a single consumer. The bus allows multiple publishers to send messages asynchronously ...
11
votes
1answer
391 views

Testing a list of proxies for validity concurrently

I have a class that tests a list of proxies for validity concurrently, which I have created in 2 different ways, one utilizing Parallel.ForEach and the other using a TPL Dataflow ActionBlock. I am ...
2
votes
1answer
659 views

C# BufferBlock that keeps unique items

In our application we watch file changes using .NET FileSystemWatcher. After each file change we process the new content asynchronously. In order to make everything ...
2
votes
0answers
508 views

Producer Consumer Using TPL Dataflow

I have written the following class which is supposed to act as a consumer of messages that are passed in from producers and forward them onto amazon SQS, I have made a compromise by starting the ...
6
votes
2answers
975 views

Task Parallel Library and Mailkit

I have a use case where I need to process about 80,000 emails per day. The client has an existing process that takes about 1/sec per email and is running out of time before the next batch comes in. ...
3
votes
1answer
2k views

Formatting a CAN bus message as a string

I am writing a CAN bus logger application and need to format messages as quick as possible in order to prevent buffer overflows on our device. In some cases I am quick enough, but under heavy loads it ...
4
votes
1answer
1k views

Rewriting to single pipeline by using data flow blocks

I have a number of messages with a payload as a list of items. My goal is to transform the message payload without losing the relation between the message and payloads. Isn't it possible to rewrite ...
1
vote
1answer
1k views

Implementation of self-updating, asynchronous cache solution of individual (non-bulk) objects

Here I am again; referring to my previous code review request about a self-updating, asynchronous (?) cache. The previous class takes care of caching operations like "get all users registered in our ...
5
votes
2answers
2k views

High udpclient CPU use

I have a bunch of different ports I want to listen on and the iterate over the available data to send to a Dataflow pipeline. In total I'm listening on 14 ports. I'm looking for any advice on how to ...
2
votes
1answer
755 views

Batcher using Microsoft.Tpl.Dataflow

What do you think? In particular, I'm not sure about my exception handling approach to raise errors when posting objects; but I couldn't come up with anything better. ...
5
votes
1answer
5k views

TPL Dataflow BatchBlock with Batch Size & Timeout

I needed BatchBlock with support for timeout (act greedy on what it received at that time) and by some search and study, this is the outcome. What are drawbacks or ...