1
\$\begingroup\$

I have a piece of code that has a bit of a problem. Not that it doesn't work, though. The problem I am having is figuring out the optimal way of working with the MemoryStream, which I find quite difficult to do in this case.

while (checkBox1.Checked && tt1.Connected)
   {

       tcpstream.Read(lenArray, 0, 4);
       read = false;
       Int32 length = BitConverter.ToInt32(lenArray, 0);
       var tempBytes = new byte[length];
       texturestream = new MemoryStream(tempBytes);
       int currentPosition = 0;
       while (currentPosition < length && checkBox1.Checked)
       {
           currentPosition += tcpstream.Read(tempBytes, currentPosition, length - currentPosition);
       }
       AutoReset.Set();
   }
}

As you can see, I'm making the texturestream over and over again (which contains tempBytes).

From my perspective, I can't understand why I need to even have a byte array. Isn't it possible to just write to the MemoryStream immediately? And just have the MemoryStream made outside the loop with a Using, and then reuse the same thing over and over (as they are dynamic with their size)?

I have tried making this possible, but it doesn't work.

I was thinking something like this:

tcpstream.read(texturestream.GetBuffer(),...,...)

I thought that would make it read to the MemoryStream instead of the byte array, which then goes into the MemoryStream.

But, it didn't work, so I am at a loss at what can be done with these loops.

Any ideas?

EDIT:

So if you check the first code, it´s a simply while loop, it´s reading 4 byte, then get´s the int data from that (BitConverter).

This is the Length of the TCP stream, which is why i remake it every loop, as i send the new length etc.

Then i take the tcpstream and set it into a bytearray, then write that array to the MemoryStream.

This isn´t what i would like to do however, i don´t really need a byte array in the first place. I however probably need a MemoryStream, though not totally sure, i may be able to work without one. It depends as i use 2 threads that works in parallel, so one is reading while the other receives and fill the memorystream.

So if i use only tcp stream, i think it would take longer time, as if the total operation would have to halt to complete it.

If i use a MemoryStream the tcpstream can continue to work, by getting the length and writing to a byte array etc, until it writes to the MemoryStream (when it reaches that, the other thread will already have read the data from it).

So here is the reading Thread:

SharpDX.Windows.RenderLoop.Run(form, () =>
{
    AutoReset.WaitOne();

    if (read == true)
    {
        sprite.Begin(SharpDX.Direct3D9.SpriteFlags.None);
        device.BeginScene();
        texturestream.Position = 0;
        try
        {
        using (tx = SharpDX.Direct3D9.Texture.FromStream(device, texturestream, -2, -2, 1, SharpDX.Direct3D9.Usage.None, SharpDX.Direct3D9.Format.X8R8G8B8, SharpDX.Direct3D9.Pool.Default, SharpDX.Direct3D9.Filter.None, SharpDX.Direct3D9.Filter.None, 0))
            {
                sprite.Draw(tx, new ColorBGRA(0xffffffff));
                if (tx.GetLevelDescription(0).Width != form.Width || tx.GetLevelDescription(0).Height != form.Height)
                {
                     wid = tx.GetLevelDescription(0).Width;
                     heg = tx.GetLevelDescription(0).Height;
                     if (heg * wid > 40000)
                     {
                         form.Height = heg;
                         form.Width = wid;
                         presentParams.BackBufferHeight = heg;
                         presentParams.BackBufferWidth = wid;
                         device.Reset(presentParams);
                     }
                     Console.WriteLine("Width:" + tx.GetLevelDescription(0).Width + " Height:" + tx.GetLevelDescription(0).Height);
                }
            }
        }
        catch (Exception ex)
        {
            //if (ex is SharpDX.SharpDXException) 
            MessageBox.Show(ex.Message, "Rendering");
        }
        sprite.End();
        device.EndScene();
        device.Present();
    }
});

Sorry for the mess.

But here i have a rendering loop, and while not ideal, it does the work currently. So, it only runt after the receiving thread has told it to(right after it has written to the MemoryStream). And i also have a Bool just for safety.

And then it does it´s device things, and finally reads the Texture from the MemoryStream. And then, draws it.

So that´s basically what the MemoryStream is used for. 1 Threads writes, 1 Threads reads, pretty much like that.

Hope this is the information needed.

EDIT 2:

So here it is currently with, the MemoryStream outside the loop, and using write, and reset the position right before the write. So the MemoryStream will only be as big, as the biggest written data at one time.

                 using (texturestream = new MemoryStream())
                            {

                                while (checkBox1.Checked && tt1.Connected)
                                {


                                    tt1.GetStream().Read(lenArray, 0, 4);
                                    read = false;
                                    var length = BitConverter.ToInt32(lenArray, 0);
                                    var tempBytes = new byte[length];
                                    int currentPosition = 0;
                                    while (currentPosition < length && checkBox1.Checked && tt1.Connected)
                                    {
                                        currentPosition += tt1.GetStream().Read(tempBytes, currentPosition, (int)length - currentPosition);
                                    }
                                    texturestream.Position = 0;
                                    texturestream.Write(tempBytes, 0, (int)length);
                                    read = true;
                                    AutoReset.Set();
                                }
                            }

But, i would like to prevent having to use a byte array as a "middle hand", can´t i write to the memorystream directly?

And, about this:

while (currentPosition < length && checkBox1.Checked && tt1.Connected) { currentPosition += tt1.GetStream().Read(tempBytes, currentPosition, (int)length - currentPosition); }

if i Don´t have " (int)length - currentPosition ", it won´t work.

I have tried using this:

tt1.GetStream().Read(tempBytes,0,(int)length);

And only that, without a while loop. And that doesn´t work, i think it can work one time or so. Not sure why it doesn´t work though, as it should work in my eyes.

I mean, i tell it to read the length of the data, so i see no reason for it to fail. (The fail is System.OverflowException, Arithmetic Overflow).

EDIT 3:

Missed the first parts that you wrote.

I would gladly hear how you would embed the size, please tell.

And, CopyTo, is probably as you say, i think i tried it before sometime. But i don´t get why i have to allocate a Byte Array, if i have a MemoryStream, i should be able to simple Stream the TCPStream into the MemoryStream (at least i think so).

I tried: tt1.GetStream().CopyTo(texturestream);

And that didn´t work as i can´t tell when it´s supposed to stop (it reads on forever).

EDIT 3:

Okay as you said, the streams goes on Forever, i always sends new data, and always reads new data. So this is why embed the size at the first 4 byte. If you know a better way of doing this, please tell.

And for the MemoryStream, am i suppose to use Write? As i gess that´s the only way to write to the MemoryStream without remaking it.

Sadly, from my tests, this:

            while (currentPosition < length && checkBox1.Checked && tt1.Connected)
            {
                currentPosition += tt1.GetStream().Read(tempBytes, currentPosition, length - currentPosition);
            }
            texturestream.Position = 0;
            texturestream.Write(tempBytes, 0, length);

Which has Using texturestream outside of the loop, is slower than:

         texturestream = new MemoryStream(tempBytes);
            while (currentPosition < length && checkBox1.Checked && tt1.Connected)
            {
                currentPosition += tt1.GetStream().Read(tempBytes, currentPosition, length - currentPosition);
            }

Though, here i can also have Using outside, so it will dispose of it after the while loop.

But i think it´s faster cause the tempBytes is linked to the MemoryStream, so writing to the tempBytes will write to the MemoryStream.

But i don´t understand this:

Finally - if your application logic allows - then don't use using(textureStream = new MemoryStream()).

Why shouldn´t i use Using? Using does exactly the same thing you mention, it disposes when i am done, which is when the loop ends.

\$\endgroup\$

closed as off-topic by Jamal Nov 3 '15 at 22:50

If this question can be reworded to fit the rules in the help center, please edit the question.

  • \$\begingroup\$ I'm voting to close this question as off-topic because it has become very convoluted, but shouldn't be reversed as the answer has already addressed the updates. \$\endgroup\$ – Jamal Nov 3 '15 at 22:50
2
\$\begingroup\$

Edit - rewrite my entire answer

First off, I kind of misunderstood the purpose of lenArray. I sort of get it now, although I don't fully understand why your app is designed like that. I would perhaps use other means of getting Length instead of embedding that value into the stream, but that is besides the point.

Perhaps the true answer you are looking for then is - each stream has a CopyTo() method. So you can perhaps use that. This way you avoid having to create a seperate byte array. Although I think what you end up winning is simply code clarity (less lines of code), because the CopyTo method has to internaly do the exact same thing you are doing now. Allocate memory (byte array) for MemoryStream and write bytes into it.

The documentation isn't very clear about it though. In that I can't be certain if it will copy the entire stream (with your 4 byte embedded length that you don't need to represent your texture) or as a MSDN comment suggests, it will only copy from the current stream position, in which case you could simply seek over the embedded length if you can't remove it to begin with. So you need to test that out yourself.

Other things worth mentioning

You can avoid having to constantly reallocate the MemoryStream in the loop. Move the declaration outside the loop, at the start of the loop call the .Seek(0, SeekOrigin.Begin) method. That way you can rewrite the internal MemoryStream data with the new data and don't incur the performance penalty of allocating and garbage collecting new objects. If your stream length changes based on the texture, then also call the .SetLength() method to make your MemoryStream of the correct size. For all of this to work your app logic must be sound, in that one thread doesn't read the MemoryStream wile another is rewriting it etc.

Finally, looking at your original code example. You are wasting CPU cycles (unless the compiler is smart enough to otimize this) here:

currentPosition += tcpstream.Read(tempBytes, currentPosition, length - currentPosition);

Before this line you set currentPosition = 0;, which means you are effectively saying
length - 0, that is a pointless substraction.

Edit 2

With length - currentPosition - I stand corrected, I overlooked the while loop, sorry about that.

I tried: tt1.GetStream().CopyTo(texturestream); And that didn´t work as i can´t tell when it´s supposed to stop (it reads on forever).

It will continue on forever if your tcpStream never stops/closes. I'm thinking that is the case. That you are continually streaming textures with different sizes and it would be the only reason that I can currently think of, why you would embed the length in the stream (otherwise you could simply access stream.Length and use that)...

From my perspective, I can't understand why I need to even have a byte array. Isn't it possible to just write to the MemoryStream immediately?

But, i would like to prevent having to use a byte array as a "middle hand", can´t i write to the memorystream directly?

In that case you don't quite understand how streams work. In order to work with a stream (Read or Write) you need finite things. A byte array is a fixed finite thing. A stream in of itself doesn't convey a finite construct, it's kind of an abstract concept. I'm struggling to think of a good analogy here...

Suffice it to say, that there doesn't exist a write API/method for a stream that accepts another stream (apart from CopyTo). Thus your only option is to read into a byte array from the source stream and then write that byte array content to the destination stream. End of story.

Finally - if your application logic allows - then don't use using(textureStream = new MemoryStream()). You are (if I understand correctly what your app is doing) constantly allocating in a loop new relatively short lived MemoryStream objects that will have to be garbage collected. Simply write MemoryStream textureStream = new MemoryStream(); outside your loop. Seek to the beginning at each loop iteration, then rewrite the old information with new data from tcpStream (if necessary also call textureStream.SetLength()). Once your application closes or you stop showing your constant stream of textures, then call textureStream.Dispose(); This way you reuse your object and don't apply unnecessary memory pressure or waste CPU cycles.

Edit 3

Okay as you said, the streams goes on Forever, i always sends new data, and always reads new data. So this is why embed the size at the first 4 byte. If you know a better way of doing this, please tell.

Most likely that is the best way to do it, unless you consider opening a different tcpStream for each texture.

And for the MemoryStream, am i suppose to use Write? As i gess that´s the only way to write to the MemoryStream without remaking it.

Apart from CopyTo and WriteByte, yes Write is the only construct to write to a MemoryStream. There are other means to get the same end result, if they apply, which is discussed next.

Which has Using texturestream outside of the loop, is slower than:

We'll the two pieces of code behave very differently. In your example, where you have MemoryStream outside of loop, this is happening:
tcpStream references memory X.
tempBytes references memory Y.
new MemoryStream() references memory Z;

You are now doing 2 memory copies. One from X -> Y (tcpStream.Read(tempBytes...)), and then Y -> Z (textureStream.Write(tempBytes...)).

Now consider your other example with textureStream = new MemoryStream(tempBytes); inside the loop. This statement effectively means, that the new MemoryStream references existing memory Y not Z. So you end up doing only one direct memory copy
X -> Y. As you read from tcpStream into tempBytes, the textureStream will already contain that information because it is based on tempBytes.

That is why it is faster.

But i think it´s faster cause the tempBytes is linked to the MemoryStream, so writing to the tempBytes will write to the MemoryStream.

Just noticed this, so yes, you came to the same conclusion I was trying to make.

Why shouldn´t i use Using? Using does exactly the same thing you mention, it disposes when i am done, which is when the loop ends.

What is it with me and while loops today? Again completely overlooked that you don't go out of the outer while loop, which at first I thought you did, thus disposing the textureStream, then as you would re-enter you would have re-created it... Pardon me, ignore my previous statement about this subject.

Optimization

SharpDX.Direct3D9.Texture has a method called FromMemory(), which takes a byte array as one of its argument. It would seem if that method is more apropriate in your scenario, because that way you can completely avoid using textureStream. Simply pass in tempBytes and job done (I assume).

Race condition

You use read as a thread synchronization mechanism, but you are not using any locking. That can lead to the following race condition (which might not effect your app or maybe it needs to work like that):

After this code has taken place:

texturestream.Write(tempBytes, 0, (int)length);
read = true;
AutoReset.Set();

That same thread can go back to the beginning of the loop and set read = false; before the other thread has a chance to continue. Thus you will miss an entire texutre, because calling AutoReset.Set(); doesn't mean that the other thread will start execution immediately. Or worse yet, the second thread can start, evaluate if (read == true) and go in, then pause. Now the first thread continues, reads half the bytes from tcpStream and writes it into textureStream, then pauses. As the second thread now continues it will get a garbeled texutreStream.

A System.Collections.Concurrent.ConcurrentQueue<T> might be a better fit in this case. You put stuff in from the tcpStream and the other thread checks if anything is queued, if so then displays that stuff.

\$\endgroup\$

Not the answer you're looking for? Browse other questions tagged or ask your own question.