Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [compression]

The name given to the process of encoding data such that it uses lesser number of bits as compared to the original representation.

0
votes
0answers
41 views

Huffman Coding API in Swift

This question is a follow up question to: this post I'd love some advice on my implementation and the API of the Huffman class. I'm also not sure how to test if ...
1
vote
1answer
70 views

Huffman Coding in Swift

I’m learning Swift and decided the Huffman Coding algorithm would be a good exercise while learning this new language. Below is a working version that encodes a ...
1
vote
0answers
12 views

“Function” to extract a file in Batch

Recently, I've been doing more and more Batch "programming". I always find myself in the need to extract a file and do something with the files. This is not my first version, and a few non-general-...
8
votes
4answers
1k views

Screen recorder producing huge AVI files

I'm currently trying to make a screen recorder in C#, and so far it works but the problem is that something as simple as a 20second video will take about 1GB of space. I have it setup so a timer ...
1
vote
2answers
68 views

String Compression

Implement a method to perform basic string compression using the counts of repeated characters. For example, the string "aabcccccaaa" would become "a2b1c5a3". If the "compressed" string ...
-1
votes
0answers
18 views

Find lengths of strings of repeated characters [duplicate]

This function takes in an array, and outputs it a character with its count if the element's length is greater than one Is there a way I can do this same thing, perhaps with a different javascript ...
2
votes
1answer
95 views

String compression algorithm using functional JavaScript

I've written an algo to compress a string eg. aabcbbba to a2bcb3a I'm pretty new to functional programming, I feel it's easier to debug and learn from functional code if you're good at it already, ...
2
votes
3answers
67 views

Javascript run length encoding algorithm

I have implemented a custom run length encoding algorithm that converts an m x n array into an p x (n+1) array. The extra column is to record counts. Probably easier to show through examples: <...
3
votes
0answers
53 views

Variable bit length lossy floating point compression

I am implementing a new compression algorithm for the weights of a neural network for the Leela Chess project. the weights are roughly 100Mb of ...
2
votes
2answers
70 views

Reading a gzipped file with JSON data for thousands of users

I'm a newbie in Python. I need to read a GZ file contains thousands of users. But I think my code is not optimized in this case. Could anyone can help me? What should I do to improve performance? Or ...
2
votes
1answer
86 views

Read gzipped JSON file from URL

I am requesting a JSON file that's gzipped. First, I download file: ...
4
votes
1answer
151 views

Huffman Coding library implemented in C

I have written a small library for C that implements the Huffman coding algorithm as outlined in David Huffman's paper on Minimum-Redundancy Codes, and a small test program to implement it. The ...
6
votes
2answers
419 views

Append to compressed tar file with performance

As you might know, Python's tarfile does not have the ability to append to files compressed with e.g. gz or ...
4
votes
0answers
563 views

Compression/decompression using Huffman Coding Algorithm

I am writing a program that compresses and decompresses data using the Huffman Coding algorithm. About compression: this program will export 2 files: header file (contains characters' frequency for ...
4
votes
2answers
65 views

Bernoulli Sequence Lab

I'm writing some Python 2 code with which to analyze the compressibility of random bitstrings. It's working pretty well right now, and now I'd like to request some help making sure it follows PEP 8 ...
1
vote
0answers
32 views

Bat file that can create a text file of any exact size and it will be compression resistant

I'm making a bat file that can create a text file of any exact size and it will be compression resistant just for fun. It's currently in 2 parts because that's the only way I could figure out how to ...
3
votes
2answers
148 views

Naive string compression

Description: Implement a method to perform basic compression using counts of repeated characters. If the compressed string is not smaller then return the original string. Code: ...
3
votes
0answers
51 views

Huffman encoding in Rust

An implementation of Huffman encoding / decoding as a learning exercise. I'm still new to Rust, so any tips about making things more natural for the language are appreciated. ...
1
vote
2answers
682 views

C++ huffman encoding - constructing encoded bytes efficiently

I've written a huffman compression program that works as intended, however in order to intuitively lay out the processes required to write the compressed file, I used strings in critical functions ...
2
votes
2answers
449 views

Modified compressed sparse row matrix code

I've implemented a Modified Compressed Sparse Row matrix class! The class works, but I think that some part of the code is quite crap. There are two passages and the whole code became too expansive ......
1
vote
0answers
705 views

Apache spark compaction script to handle small files in hdfs

I have some use cases where I have small parquet files in hadoop say 10-100mb and I would to compact them so as to have files at least say 100mb or 200mb. The logic of my code is find a partition to ...
-1
votes
2answers
901 views

Basic string compression implementation in C [closed]

I tried to implement a string compression algorithm that is mentioned in CTCI-5th 1.5 using C. Does this code follow common best practices? Performance and correctness in unanticipated cases of this ...
4
votes
1answer
322 views

Python read/write pickled file

Is there a way in Python 2.7 to improve reading/writing speed (or memory consumption of the file) compared to this version? ...
4
votes
2answers
1k views

Program to compress a string of characters

I was given this small task in an interview recently. I'm usually terrible at coding questions in interviews, because with the time constraint and the lack of google I usually overthink and rush stuff ...
2
votes
1answer
58 views

Extracting large tar.bz2 folders in parallel

The bash below utilizes parallel to extract tar.bz2 folders with 9 files in them. The total size of each is around 60GB. Below ...
4
votes
2answers
8k views

Huffman Coding in C++

I have written this code after studying from Introduction to Algorithm and from GeeksForGeeks. I know there is a lot to improve because I don't know much C++11. Please help me to improve this code. <...
10
votes
3answers
1k views

A simple C++ class for packing DNA strings

Introduction I have this small C++ program that packs a DNA string over alphabet ACGT into a bit vector, two bits per character. Code dnapack.hpp ...
3
votes
1answer
106 views

Exporting an object to a ZIP file

I have my own type of object, which implements Serializable. Here I have one called "extendedFormat" (not a terribly helpful name, yes, I know). I'm writing it out ...
15
votes
1answer
2k views

Compressing a 'char' array using bit packing

I have a large array (around 1 MB) of type unsigned char (i.e. uint8_t). I know that the bytes in it can have only one of 5 ...
1
vote
0answers
107 views

Micro-optimize variable byte encoding/decoding

Disclaimer: This question was originally asked on Stack Overflow but the community advised to post it here. I have written routines to encode and decode signed 32-bit-integers into variable byte code:...
7
votes
1answer
400 views

Python 3.x Hexdump

I posted a small hexdump generator function from a program I've been writing not long ago and applied what a reviewer suggested since then. The goal was to lazily hexdump bytes objects (byte strings, ...
6
votes
2answers
277 views

Creating / Uncompressing zip files in VBA

I work for a company that makes industrial secondary packaging machines. (Our machines put stuff in cardboard cases.) The machine can run several different sets of parameters for various product and ...
3
votes
1answer
353 views

Algorithm to transform a string according to occurrence count of each character

I have this algorithm that counts the frequency of character occurrence in a string, and outputs a new string based on that. For example, ...
2
votes
1answer
135 views

Minify and send HTML response in Go

I need to have some static HTML pages on a project I am making with Go. I used the http.FileServer, but I noticed it was slower than rendering with ...
2
votes
0answers
90 views

Huffman encoding implementation -follow up

Follow up of this question. I implemented most suggestions from the answers of the previous question. Code: ...
3
votes
2answers
2k views

C++ LZ77 compression algorithm

I had a hard time with int / size_t as we can't really substract a size_t. So I don't know ...
8
votes
1answer
865 views

C++ Huffman coder and decoder

I've implemented a Huffman encoder and decoder. The code is a bit long, I'm worrying if it is easy to grasp it. Also it is my first time using ...
7
votes
1answer
580 views

Run Length Encoding (RLE) and Move To Front (MTF) in C++

I have made two quite basic algorithms used for lossless data compression. This is my RLE: ...
4
votes
2answers
174 views

Optimize Huffman Compressor in C++

I'm currently writing a Huffman compressor for educational purposes. I want it to compress/decompress files less then 5mb. The time limit is 5 seconds. Now it compresses a 5mb file in almost 8 ...
3
votes
1answer
408 views

Compress svg files in PHP

I wrote something to "compress" svg files. The svg files I am using often have comments and empty <g> tags, and I want to remove them. My main goal is not the ...
1
vote
0answers
148 views

Optimizing BZ2 decompression code for big data set

I have the following fully working code that: Walks through all sub-folders in a given directory; Captures dirpath and filename for the compressed file (i.e., Dir/*.json.bz2); Creates a decompressed ...
4
votes
1answer
600 views

Rijndael compression/encryption class

Out of personal interest and as a learning exercise I've written a C# class (.NET 4) to perform encryption/decryption of a file along with some compression upon encryption. Most of my understanding of ...
1
vote
2answers
814 views

Image Compression Using the Singular Value Decomposition (SVD) with MATLAB

I'm new to both linear algebra and MATLAB. I need help with this code with the objective of compressing image using the Singular Value Decomposition (SVD). The code is currently working, if I run with ...
-3
votes
1answer
369 views

Huffman tree node

I've implemented a Huffman tree for one of my academic coursework. For each of the nodes in the tree, there will be a left child, right child, and a parent. For each ...
6
votes
1answer
108 views

CGI output gzip compression module

Edit: How should I interpret the silence? On a scale from 0 to 10 where 0 means "Bloody awful" and 10 means "Nothing to complain about". I'm mainly concerned about readability and things I don't ...
5
votes
2answers
907 views

Perform basic string compression using the counts of repeated characters

I managed to implement this question from Cracking the Coding Interview: Implement a method to perform basic string compression using the counts of repeated characters. For example, the string <...
4
votes
1answer
3k views

Improving gzip function for huge files

I have created a python system that runs Linux core files through the crash debugger with some python extensions. This all works fine but one bit of is problematic. These files are send to the ...
2
votes
1answer
90 views

Compression unit test data “easy case”

In another post I created a piece of data, which make compressor annoyed. Create 'worst case test' data for compression test Now I am trying to make it happy. I generate another data set, ...
8
votes
2answers
138 views

Create 'worst case test' data for compression test

I am going to prepare some test data for a compression test. One of the them is the 'worst case test', which should make the compressor work worst. Use random number to generate such a file is an ...
4
votes
1answer
484 views

LZ-77 text compression algorithm

This is my version of a LZ-77 lossless compression algorithm for text. It maintains a sliding window of 4095 characters and can pick up patterns up to 15 characters long. Basically, the compressed ...