Design with Context, Table and Functions

Question

Each class of my project has several inputs and outputs. Each input, the same as output, depends on it's own set of value-datatype.

I need to provide a mechanism to forward data streams from one instance to another, with ability to split that streams and to combine several streams into one.

There are a lot of such classes, and they are hardly grouped by their input or output type. Splitting and combining (may be other operations like filtering or simple math) will provide this classes interaction.

I came out with Table class, which can store and manage two-dimensional array of objects and also can send rows to another Table, using maps on columns (that's how data streams splits). It could be one-dimensional, but data buffer avaibility is useful. Each class become a Context with a collection of that tables. Typical work algorythm for Context is to: 1. initialize tables, 2. handle input table inserts, 3. unbox and process data, 4. insert result to output table

Everything was OK with that Table, but one day, while coding a new Context I felt that data storage and transfering is not the only thing Context must do. When an external infrastructure is hiding behind the Context, it needs to be connected/disconnected/opened/loaded etc. And that looks like I need to add a Procedure or even Function to Context.

This is first huge project of mine and I have no person to help. Questions I care about:

• this design looks like implementing internal programming language, is that correct? Can you show examples?

• this design looks like implementing in-memory database, is that correct? Can you show examples?

• is that normal not to use interfaces on data types if there are a lot of them?

• can you propose more effective and elegant design for my problem?

• are there similiar design patterns?

---

May be some code will help to understand what I need. This is very simplified version. In real project there are a lot of Context realisations, more sugar to access and create Tables, more logic on managing internal collections.

public abstract class Context
{
    public Table this[string name] { get { /* ... */ } }

    public IEnumerable<Table> Tables { get; }

    protected void AddTable(Table t) { /* ... */ }
}

public class Table
{
    object[][] rows; // Two-dimensional for buffering rows

    public Table(string name /* + columns initialisation info */ ) { /* ... */ }

    public void PushRow(params object[] row) { /* ... */ } // insert new row and send it to subscribers

    public void AddSubscriber(Table dst, params int[] columnMap) { AddSubscriber(dst.PushRow, columnMap); }
    public void AddSubscriber(Action<object[]> receiver, params int[] columnMap) { /* ... */ }
}

public class Function
{
    // Do I need this in Context, if, for example, Mathematician must be Opened/Closed?
}

class Mathematician : Context
{
    public Mathematician()
    {
        // input table
        Table t = new Table("Numbers");
        AddTable(t);
        t.AddSubscriber(Count);

        // output tables
        AddTable(new Table("Average"));
        AddTable(new Table("Variance"));
        AddTable(new Table("StDev"));
    }

    void Count(object[] row)
    {
        double newNumber = (double)row[0]; // exception here if data streams connection is not correct

        double avg = 0;
        double variance = 0;
        double stdev = 0;

        /* ... computations begin ... */

        /* Tables can be accessed, their buffer can be used */

        // Finally, publish results
        this["Average"].PushRow(avg);
        this["Variance"].PushRow(variance);
        this["StDev"].PushRow(stdev);
    }
}

Possible usage. Imagine that we have another context Secretary : Context which will receive data from Mathematician to write it somewhere and Device : Context to provide Mathematician with new numbers. Note, they're independant of each other.

static class Program
{
    public static void Main(string[] args)
    {
        Context m = new Mathematician();            
        Context d = new Device();
        Context s = new Secretary();

        d["Sensor1"].AddSubscriber(m.Numbers, /* map must correspond to Device internals */)
        m["StDev"].AddSubscriber(s.Console, /* ... */);


        // Now each sensor1 signal in Device context will be sent to Mathematician, and the results of computations will be sent to Secretary, which can write it to console

        // We can start this system manually like that
        d["Sensor1"].PushRow( /* ... signal data ... */);
    }
}

Answer 1

I think the single biggest issue with your design is that it's not type-safe and relies on strings a lot.

C# is a statically-typed language, you should take advantage of that and let the compiler check for possible errors. This means:

1. Making Table generic, so that a row is an object of a specific type, instead of object[]. If you need to “map” data between different types, you can use lambdas.
2. Tables in a specific context should be properties, instead of accessing them by a string.
3. You should separate input tables and output tables into separate types (at least on the outside). Mathematician should be able to read from the Numbers table, but others should only write to it. It's a good idea to enforce this by making Numbers something like InputTable<T>.

In general, your requirements are quite similar to TPL Dataflow, you should consider looking at it for inspiration, or even use it in your implementation (though it's .Net 4.5 only).