Loki Astari has already pointed out the counterintuitive behavior of your copy assignment operator. Assuming that this is indeed an accident and not the semantics you intended, I would like to suggest a quite radical refactoring of your code.
using Memory = std::vector<char>;
The standard library already has everything you need here.
Of course, it might still be a good learning opportunity to implement things yourself but as I understand your question, you're actually trying to get a job done here, rather than intentionally reimplementing existing functionality as an acedemic exercise.