Questions tagged [memory-management]
Memory management is the act of managing computer memory by allocating portions of memory to programs as well as freeing memory so that it can be re-used.
44
questions with no upvoted or accepted answers
6
votes
0
answers
88
views
Subtracting elements of datasets of an HDF5 file
I am trying to solve the following problem:
Input: Input is two arrays (Nx4, sorted in column-2) stored in datasets-1 and 2 in HDF5 file (input.h5). N is huge (...
- 61
5
votes
0
answers
1k
views
Polymorphic deleter for unique_ptr
There is a basic difference in the way C++ manages the deleter for std::unique_ptr and std::shared_ptr, mainly for allowing ...
- 225
4
votes
0
answers
691
views
A fixed-size dynamic array
Background
I implemented the historic std::dynarray according to the specification under the name dynamic_array in C++17. ...
- 9,456
4
votes
0
answers
1k
views
C++ shared_ptr memory pool
I put together this little memory pool class to help avoid the costs associated with heap allocation and deallocation of objects that are frequently created & destroyed. It creates C++ standard <...
- 415
4
votes
0
answers
141
views
Memory managment for RPC code in C
My team has been given care of an old embedded C project. Its main part is a security critical RPC API and I'm trying to create a new design of its internal interfaces to make it more robust against ...
- 141
4
votes
0
answers
70
views
Tracker for object construction, copy, and movement
I made an object tracker for debugging and testing purposes called ccm_counter (construction, copy, move counter). It counts constructor, copy and move calls. It ...
- 323
4
votes
0
answers
401
views
SoundBoard in C++ Using Qt
I've been working on a soundboard software for tabletop games audio immersion in C++ using Qt. The software consists of 2 tabs who provides to a game master a set of individual sounds or scenes to be ...
- 41
4
votes
0
answers
420
views
Cleaning up memory after loading Access DB into a datagrid
I wrote an app that pulls a MS Access DB into a DataTable and then displays it into a WPF datagrid. The dialog window has a search button which calls on a class ...
- 63
4
votes
0
answers
312
views
cross-platform stdlib-only aligned allocator in C++14
I'm writing a program which uses OpenCL, and the OpenCL types are mostly over-aligned. Sometimes when using stl containers with these types, I'd get segfaults, so I tried to write my own aligned ...
- 161
4
votes
0
answers
346
views
Cache with timeout per key
I wrote a general purpose library for in-memory cache with custom timeout for each key.
...
- 273
3
votes
0
answers
138
views
Custom allocator for improved cache locality and fast iteration in C++
I am writing a custom allocator for my educational game engine project where I need to iterate through a collection of components(similar to Unity3D) every frame. There are game objects and every game ...
3
votes
0
answers
273
views
Simple and safe C++ pool allocator
I have written a simple pool allocator for C++ and I'm looking for ways to improve it, in speed, usability or safety. For example, I don't know how to allocate a buffer larger than the pool size.
<...
- 31
3
votes
0
answers
1k
views
Memory release after joblib.Parallel [python]
Stuck with the issue with memory consumption - after running joblib's Parallel, deleting results and gc.collect() -ing I still have increased memory (checking by htop for process line). Found no way ...
- 31
3
votes
0
answers
506
views
Pandas data extraction task taking too much memory. How to optimize for memory usage?
I need to process some data (one of its columns contains a json/dict with params- I need to extract those params to individual columns of their own; catch- some rows have some parameters, others have ...
- 131
3
votes
0
answers
315
views
C++ object pool using C memory pool as base
I started out with this question on Stack Overflow. I have since then gotten an official answer from the mbed developers that the MemoryPool that I was asking about only issued raw C malloc/free type ...
- 31
3
votes
0
answers
444
views
Applying Scikit-Learn Linear Kernel with a generator for low memory pressure
matrix = # a sparse matrix of TFIDF vectors - 28458x3218988
My aim was to create a similarity matrix comparing all documents to all documents using ...
3
votes
0
answers
636
views
Populate MongoDB with random documents using a Go channel
I recently created a small app in Go that generates random bson documents and insert them in a database (MongoDB). See this question
for details.
After doing some profiling, I realised that garbage ...
- 588
3
votes
0
answers
245
views
A way to automatically reorder struct fields by their sizes in descending order
Using constexpr and preprocessor magic it is possible to cook a struct which have a minimal size between all possible sizes, keeping all the data members properly ...
- 624
3
votes
0
answers
37
views
Using STORAGE macro using QSAM for input/output
I was wondering if you can help me out. I wanted to know if there is an easier way to do the following. My code works but I feel like there is a lot things going out with multiple loops and wanted to ...
- 31
3
votes
0
answers
571
views
Parsing and producing sets of financial CSV files
I have this small Haskell script that I wrote some time ago to parse a set of financial CSV files to produce other CSV files. I've recently had a problem with large input files (around 300Mb) that I ...
- 131
2
votes
0
answers
40
views
Reusable storage for array of objects V4
Here is a thirdfollow up on Reusable storage for array of objects, Reusable storage for array of objects V2 and Reusable storage for array of objects V3, taking into account the provided answers.
The ...
- 299
2
votes
0
answers
62
views
Virtual memory manager, physical memory manager and buddy allocator
I'm writing memory manager for my toy operating system and I would like to get some feedback. There is physical memory manager, which uses bitmap, virtual memory manager which uses buddy algorithm for ...
- 21
2
votes
0
answers
229
views
OSDEV Physical Memory Manager (PMM) in C
I ported the Physical Memory Manager from Vinix into C for a little OS I'm working on. I'd love to get some feedback.
Here's my code:
...
- 21
2
votes
0
answers
90
views
Dynamic heterogeneous container
I want to have dynamic continuous block of memory for different types, that inherit from single base class. I've written structure for that purpose. I know I play with a fire here, but I would like to ...
- 21
2
votes
0
answers
100
views
A lightweight version of std::shared_ptr<T> V2
This is a lightweight version of std::shared_ptr<T> called res_ptr<T>
This post is a continuation of:
A lightweight ...
- 2,253
2
votes
0
answers
27
views
Thread safe Shareable and Splittable Buffer with Safe Memory (Project)
Backstory
This is a buffer I wrote for use in multiple personal projects. It is designed
to handle pretty much any data in pretty much any way. In particular I intend to
use it in such instances as ...
- 131
2
votes
0
answers
49
views
Fixed Size Resource Array with Versioned Indexes
I'm trying to implement a fixed sized array that uses versioned keys for dangling safety. This structure is similar to a slot map data structure, with the exception that this does not automatically ...
- 53
2
votes
0
answers
1k
views
Wrapping C++ library in C++/CLI using shared_ptr
Background: I'm writing C++ code after a fairly long time, so I may not be up to date on the best practices, so please bear with me. I'm only trying to improve and learn.
Problem I need to write a ...
- 121
2
votes
0
answers
305
views
Yet Another Non-Intrusive Reference Counted Smart Pointer Implementation
I needed a reference counted smart pointer for my project, and for some reason early in my project, I decided that I did not like the std::shared_ptr. I can't ...
- 21
2
votes
0
answers
502
views
Partial file encryption with AES
Task is to encrypt part of the file using aes encryption algorithm. I did
...
- 201
2
votes
0
answers
492
views
Get process max memory usage while copying large data
I want to check the memory usage of a process before and after I copy a large file or folder, and also, I want to get the maximum memory usage during file operation. Is this a good method?
...
user127689
2
votes
0
answers
194
views
Custom checked_ptr (like std::shared_ptr) for projects without STL
I have written a class to encapsulate pointers for a platform, where the STL is not available. The idea is to have something similar to std::shared_ptr.
However, I ...
2
votes
0
answers
266
views
Fetching a weather report on an Arduino from a web service
I'm writing some code which is basically supposed to http get some information from my own website and parse it, which (finally) works. However, when I compile the code, it shows this:
Sketch uses ...
- 1,189
2
votes
0
answers
395
views
Implementation of a lock-free fixed-sized allocator - follow-up - with commented code
The following implementation of a lock-free fixed-size allocator is a follow-up of:
Implementation of a lock-free fixed-sized allocator
§1 - Introduction
The purpose is self-learning of atomic ...
- 565
2
votes
0
answers
88
views
Platform independent leak/cyclic dependency detection of objects
Sometime back I came across a bug where a particular object was leaking but it was very hard to identify the root cause. I knew that it was due to a cyclic dependency (which usually indicates a bug in ...
- 121
2
votes
0
answers
102
views
D3D COM Object Pooling
I'm using the following pattern to pool D3D objects globally in my application. Is it a good idea (alternatives)? Is it thread-safe?
...
- 576
1
vote
0
answers
60
views
Buddy - Algorithm in Rust
I wrote my first code in Rust and decided to recall the buddy algorithm, which I suggested at work for our embedded software.
I hope you can give me some advise on things like:
How to make it safe (...
- 21
1
vote
0
answers
29
views
Inserting captions under photographs in word based on entries in excel spreadsheet
I have a macro written in word for transferring captions written in excel to a corresponding photograph in a word document. At the moment it runs however I am finding that it slows down over time the ...
- 11
1
vote
0
answers
174
views
Reduce memory usage/lag with multiple large JSON files coming in through XHR
I need to show a map of the United States using Leaflet, where if the user clicks on a state, the map zooms in to view all of the state's census tracts. I need to do this without a significant delay ...
- 243
1
vote
0
answers
23
views
Initializing and managing many references in multithreaded control application
I have a large multi-window GUI program (implemented with MFC) that controls a variety of physics experiment apparatus. When the user starts the main function of the program (the "experiment" run), a ...
- 111
1
vote
0
answers
1k
views
Purging elements from the map if it reaches a memory size limit
I have implemented a LRU cache using ConcurrentLinkedHashMap. In the same map, I am purging events if my map reaches a particular limit as shown below.
I have a ...
- 551
1
vote
0
answers
939
views
Optimize Python script for memory which opens and reads multiple times the same files
I have code that works perfectly, but it uses too much memory.
Essentially this code takes an input file (lets call it an index, that is 2 column tab-separated) that searches in a second input file (...
- 147
0
votes
0
answers
74
views
Ewald summation in Julia
I am writing a small MD (molecular dynamics) engine in Julia and I noticed that 67% of the time spent running the following Ewald summation function (to calculate coulomb energy of an infinite crystal ...
0
votes
0
answers
135
views
creating worksheets for unique values parsed
I have built some code that should parse the data according to a unique value and then create a new Worksheet for each unique value. My initial table has 10 Columns and around 25K rows. The code works ...
- 1