5
\$\begingroup\$

I created a function to read lines from an file into chunks. My hidden agenda in creation this script was in python the yield function in interaction with chunks. The script works fine, but now i want to know if anyone has improvements?

#!/usr/bin/env perl
use strict;
use warnings;
use Data::Dumper;


sub read_in_chunks {
    my $args = shift;
    my $self = {
        fd => $args->{fd} || undef,
        chunk_size => $args->{chunk_size} || 10,
        chunks => [],
    };
    my $fh = $self->{fd};
    return unless defined(my $line=<$fh>);
    while(<$fh>){
        chomp($_);
        # maybe the following line could be written nicer :)
        ($self->{chunk_size} == 0) ? return $self->{chunks} : (push @{$self->{chunks}}, $_);
        $self->{chunk_size}--;
    }
    return $self->{chunks};
}

open my $fh, 'dump.txt' or die $!;

my $opts = {
    fd => $fh,
    chunk_size => 4
};

while(my $chunk = read_in_chunks($opts)) {
    print Dumper($chunk);
    # process data
}

close $fh;
\$\endgroup\$
  • \$\begingroup\$ What happens to the line read by $line=<$fh>? \$\endgroup\$ – choroba May 16 '17 at 14:59
  • \$\begingroup\$ it returns if $line=<$fh> is not defined \$\endgroup\$ – Patrick85 May 16 '17 at 15:21
  • 1
    \$\begingroup\$ Yes, but if it's defined, it gets lost. \$\endgroup\$ – choroba May 16 '17 at 15:50
  • \$\begingroup\$ @choroba: maybe you have an Idea how i can improve my code snippet? \$\endgroup\$ – Patrick85 May 16 '17 at 19:53
3
\$\begingroup\$

Here's my take. I removed the $self structure, as it gives you no advantage, and fixed the problem with missing lines.

#! /usr/bin/perl
use warnings;
use strict;

use Data::Dumper;

sub read_in_chunks {
    my %args = @_;
    my $fh = $args{fh} or die "No filehandle given.\n";
    my $size = $args{chunk_size} || 10;
    my @chunk;
    while (@chunk < $size && defined(my $line = <$fh>)) {
        chomp $line;
        push @chunk, $line;
    }
    return @chunk
}

open my $fh, shift or die $!;

my %opts = (
    fh         => $fh,
    chunk_size => 4
);

while (my @chunk = read_in_chunks(%opts)) {
    print Dumper(\@chunk);
    # ...
}
\$\endgroup\$
  • \$\begingroup\$ Thanks to You @choroba . Do. You think Reading big Files is better and faster when Reading in Chucky? I Know this answer can only be answered if you make an Time measure. \$\endgroup\$ – Patrick85 May 16 '17 at 20:19
  • \$\begingroup\$ @Patrick85: Definitely not this way. You can try read or sysread or reading with $/ set to a reference to a number. \$\endgroup\$ – choroba May 16 '17 at 20:41
0
\$\begingroup\$

Untested,

use strict;
use warnings;
use Data::Dumper;

sub read_in_chunks {
    my ($args) = @_;

    my $chunk_size = $args->{chunk_size};
    my $fh = $args->{fd} or die "no filehandle";

    return sub {
        $chunk_size ||= 10;
        my $chunks = [];
        # my $line = <$fh> // return;
        while (my $line = <$fh>) {
            chomp($line);

            last if !$chunk_size;
            push @$chunks, $line;
            $chunk_size--;
        }
        return @$chunks ? $chunks : undef;
    };
}

open my $fh, '<', 'dump.txt' or die $!;

my $opts = {
    fd => $fh,
    chunk_size => 4,
};
my $iter = read_in_chunks($opts);

while (my $chunk = $iter->()) {
    print Dumper($chunk);
    # process data
}
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.