Questions tagged [compression]

The name given to the process of encoding data such that it uses lesser number of bits as compared to the original representation.

Filter by
Sorted by
Tagged with
2
votes
2answers
50 views

Priority queue implementation on C. (For Huffman Coding)

I trying to implement Huffman Codes on C. And, since my previous attempt failed, I decided to approach the issue more responsibly. So I'm asking for feedback on my implementation of the priority queue ...
2
votes
0answers
33 views

SevenZipCompressor Utility class that uses Commons-Compress to compress and uncompress 7zip archives

I wrote a Compressor utility class to generate 7zip files containing everything in a specific directory. The problem is that it's an order of magnitude slower than both archiving the directory using ...
3
votes
2answers
123 views

Slow Huffman Code in pure Python

I was working on writing a fast implementation of a simple Huffman code compression of text. The idea was to write it using only the standard library, but I can't seem to find a way to make it faster. ...
5
votes
1answer
80 views

LZ77 compressor and decompressor in C++

I've written working LZ77 algorithm implementation which uses linked lists (they help to look for matching substrings faster). I'd like to get some feedback on my code's quality and also information ...
3
votes
1answer
94 views

LZW Compression Library

I wrote a library that implements LZW compression and decompression. A goal of this project was to help me acquaint myself with modern C++ development practices (I primarily come from a Java ...
9
votes
2answers
219 views

Huffman encoding

The Huffman-Algorithm The Huffman-Algorithm, named after David A. Huffman who first published this algorithm in 1952, is a algorithm for lossless data compression. As most encoding-methods, the words ...
4
votes
1answer
45 views

Huffman encoding as Unix style command line utility

After watching Tom Scott explain Huffman coding in this YouTube video, I wanted to implement it myself. I want to use this project to further my understanding of Python. Additionally this tool should ...
4
votes
0answers
41 views

Fast compression using a thread pool

This is my latest effort on RFC 1951 compression ( the widely-supported compression standard ) in Rust. It uses 2 extra threads to compute the LZ77 matches and the checksum in parallel with the ...
1
vote
0answers
22 views

RFC 1951 “Inflate” (de-compression)

My exploration of Rust continues. I'm quite pleased with the 2-step table lookup for decoding variable bit-length codes here, for some reason I found this tricky to work out, but now it's done I like ...
10
votes
3answers
283 views

Compression Library for C using Huffman Coding

This is an update to a question I posed nearly two years ago about my implementation of Huffman Coding which I had written in C. Since then I have had time to grow as a programmer and have managed to ...
2
votes
0answers
43 views

LZ77 Encoding and Decoding in Haskell

I am trying to learn Haskell and figured I would kill two birds with one stone and also learn about data compression. My ultimate goal is to implement the DEFLATE compression algorithm in Haskell. To ...
6
votes
0answers
54 views

Feedback on LZSS implementation

Recently, I've been trying to get back into Haskell, so I tried coding up a LZSS implementation. I realize this is much more easily implemented imperatively, but I wanted to challenge myself. The last ...
3
votes
2answers
87 views

Huffman Encoding in Javascript

So, I've implemented Huffman Encoding in JavaScript, you can see it live here. Here is the code: ...
1
vote
0answers
39 views

How can I make my LZ77 algorithm to compress faster or to compress more?

So, I am trying to create a PNG compression library, and to do that I need to use the DEFLATE algorithm; DEFLATE consists of two parts, the LZ77 compression, and the Huffman Compression. Currently, my ...
1
vote
0answers
11 views

Compressing and encrypting file systems for backups

I am currently backing up remote OS' while preserving the permissions using rsync: rsync -aAX --numeric-ids --delete ... root@X.X.X.X:/ /backup/server/ Now I ...
3
votes
0answers
39 views

Doing image compression with Neural Network AutoEncoders

I wanted to create an image compressor using Machine Learning and started work on an "AutoEncoder". This is a type of Neural Network which takes in the image and creates a compressed vector form. It ...
4
votes
1answer
58 views

Minimalist context compressor

I am working on an implementation of the minimalist compression method. The method is based on context modeling, followed by a simple entropy coder (Golomb-Rice coder). The context model uses a single ...
5
votes
1answer
121 views

Minimalist Golomb-Rice coder

I have implemented a very simple but robust implementation of the Golomb-Rice coding. To understand my motivation, see the minimalist data compressor that is based on this. At the moment, the ...
6
votes
2answers
114 views

Compressing large jpeg images

I'm working with thousands of large image files in a regularly updated library. The following script does the job (on average reduces my file size ~95%) but costs me around 25 seconds to compress one ...
3
votes
0answers
50 views

Powershell optimization (scripts to run simultaneously)

I need some help here, I posted on stackoverflow and was directed here, just to put things in context, I am new to PowerShell, I have a task, that in simple terms, would take a csv with more than ...
6
votes
1answer
269 views

LZ77 compression algorithm (general code efficiency)

I'm implementing LZ77 compression algorithm. To compress any file type, I use its binary representation and then read it as chars (because 1 ...
8
votes
1answer
336 views

Brainfuck compressor in Python

Just for fun, I created a script called TinyBF which can compress Brainfuck programs into a format half their size. For example, take a famous hello world program clocking in at 78 bytes: ...
7
votes
1answer
221 views

LZ77 compression (also longest string match)

So, I'm implementing LZ77 compression algorithm. To compress any file type, I use its binary representation and then read it as chars (because 1 ...
7
votes
1answer
99 views

Loading BitmapImages out of a Zip file

I'm working on a personal project to get better with C#, and as part of that I have a zip file with 20 or so images saved inside. I want to load those images out of the zip file and display them in my ...
5
votes
3answers
137 views

File compression project

I wrote a general purpose file compression program out of curiosity, but I am concerned about readability. I don't know if the variable names are useful, and whether most of the comments are unneeded ...
3
votes
1answer
51 views

File compressor

I have written a file compressor in python. The program generates a file filled with sequences of the same character with a random length, and compresses that file. I would like feedback on code ...
2
votes
2answers
378 views

Run length encoding in C#

This is the original question: https://www.geeksforgeeks.org/run-length-encoding/ Given an input string, write a function that returns the Run Length Encoded string for the input string. For ...
2
votes
0answers
87 views

Google drive backup robot

I have a lot of files. I care about those files. I also care about good programming practices and learning proper python. Given the described context, it seemed appropriate to implement my own ...
0
votes
1answer
29 views

Uncompressing a list with some run-length-encoded elements

Regarding the following code ...
3
votes
1answer
77 views

Print a string of characters with their occurrences

I have worked out a solution for this problem, however I am trying to reach an O(1) solution without the use of two for loops. The output should read as a3b2c4d1 ...
3
votes
1answer
123 views

First Huffman Compression Algorithm in C++

I decided to take up learning c++ recently, so I coded a Huffman compression algorithm. I'm particularly interested in critique of my c++ techniques (pointers, references, best practices, etc), but I ...
6
votes
3answers
4k views

Huffman Code in C++

This is my version of the Huffman Codes. ...
4
votes
2answers
296 views

WPF Bitmap / BitmapFrame to multi-page / multi-frame TIFF with JPEG encoding

I was searching for a long time how to write multi-page TIFFs with the JPEG encoding. TIFFs support JPEG encoded frames but the built-in encoder in .NET Framework does not have JPEG as a compression ...
5
votes
2answers
119 views

Using two threads to increase RFC1951 Deflate compression performance

I have modified my implementation of RFC 1951 "Deflate" compression to use two threads. One thread performs the LZ77 matching, looking for sequences of at least three bytes that occur earlier in the ...
1
vote
2answers
97 views

Optimised code to compute Huffman codes when implementing RFC 1951

I found a trick to significantly reduce the time needed to compute Huffman codes in my implementation of RFC1951. I was slightly surprised to find it slightly improves compression as well, something ...
6
votes
3answers
179 views

Improved RFC 1951 Deflate compression

I have substantially revised my earlier implementation, it now seems to be competitive with the standard C# ZLib implementation, increasing compression while being faster for moderate length inputs, ...
8
votes
2answers
190 views

RFC 1951 compression LZ77 re-hashing approach

I have written a C# class to perform LZ77 compression ( per RFC 1951 ), to incorporate in my RFC 1951 deflate implementation ( posted on Jan 2 ). The usual method (per RFC 1951) is to use a 3-byte ...
3
votes
2answers
270 views

RFC 1951 “Deflate” compression

This is a little project I did over Christmas, I wanted to understand a bit more about how RFC 1951 (https://www.ietf.org/rfc/rfc1951.txt) compression worked, mostly out of curiosity really. RFC 1951 ...
12
votes
3answers
2k views

Huffman tree compressing/decompressing in C

In a past course one of the assignments was to write a program that can compress files using Huffman Tree algorithm, and uncompress the files that the program generates. My design is to count the ...
2
votes
1answer
240 views

Huffman Coding in Swift

I’m learning Swift and decided the Huffman Coding algorithm would be a good exercise while learning this new language. Below is a working version that encodes a ...
1
vote
0answers
83 views

“Function” to extract a file in Batch

Recently, I've been doing more and more Batch "programming". I always find myself in the need to extract a file and do something with the files. This is not my first version, and a few non-general-...
8
votes
4answers
2k views

Screen recorder producing huge AVI files

I'm currently trying to make a screen recorder in C#, and so far it works but the problem is that something as simple as a 20second video will take about 1GB of space. I have it setup so a timer ...
1
vote
2answers
421 views

String Compression

Implement a method to perform basic string compression using the counts of repeated characters. For example, the string "aabcccccaaa" would become "a2b1c5a3". If the "...
2
votes
1answer
652 views

String compression algorithm using functional JavaScript

I've written an algo to compress a string eg. aabcbbba to a2bcb3a I'm pretty new to functional programming, I feel it's easier to debug and learn from functional code if you're good at it already, ...
3
votes
3answers
356 views

Javascript run length encoding algorithm

I have implemented a custom run length encoding algorithm that converts an m x n array into an p x (n+1) array. The extra column is to record counts. Probably easier to show through examples: <...
3
votes
0answers
77 views

Variable bit length lossy floating point compression

I am implementing a new compression algorithm for the weights of a neural network for the Leela Chess project. the weights are roughly 100Mb of ...
2
votes
2answers
561 views

Reading a gzipped file with JSON data for thousands of users

I'm a newbie in Python. I need to read a GZ file contains thousands of users. But I think my code is not optimized in this case. Could anyone can help me? What should I do to improve performance? Or ...
2
votes
1answer
3k views

Read gzipped JSON file from URL

I am requesting a JSON file that's gzipped. First, I download file: ...
7
votes
1answer
904 views

Huffman Coding library implemented in C

I have written a small library for C that implements the Huffman coding algorithm as outlined in David Huffman's paper on Minimum-Redundancy Codes, and a small test program to implement it. The ...
6
votes
2answers
2k views

Append to compressed tar file with performance

As you might know, Python's tarfile does not have the ability to append to files compressed with e.g. gz or ...