The Stack Overflow podcast is back! Listen to an interview with our new CEO.

# Tag Info

9

Be sure to read the relevant RFCs that govern e-mail headers! Specifically: RFC 2822, Section 1.2.2: Header names are case-insensitive. RFC 2822, Section 2.2.3: Header fields may be line-folded: 2.2.3. Long Header Fields Each header field is logically a single line of characters comprising the field name, the colon, and the field body. For ...

7

What's wrong for filename in $(ls -1 ./*); do Ouch! As a general rule (with very few exceptions), do not use ls in scripts. What you wrote is almost equivalent to for filename in ./*; do, except that if there are any non-printable characters, whitespace characters, or $?* in the file names, they will be mangled if you use ls. You don't need the ./ (except ... 7 Generally good code - plus points for good use of stdout/stderr and exit status. Shellcheck reported some issues: shellcheck -f gcc 214327.sh 214327.sh:4:19: warning: Expanding an array without an index only gives the first element. [SC2128] 214327.sh:4:19: note: Double quote to prevent globbing and word splitting. [SC2086] 214327.sh:31:61: note: ... 6 This should be pretty speedy: awk '/bar/ {n++} n==2 {exit} {print}' file To pass the pattern from the shell: awk -v p="pattern" '0 ~ p {n++} n==2 {exit} {print}' file 5 It's important to understand the purpose of every single symbol in a command: The g flag in sed's s/// commands is unnecessary when the pattern is anchored with ^: there will only be one match or no matches, never more The -s flag of cut is pointless: all lines produced by history will have a separator character You can do this with a single regular ... 5 Your regex looks complicated. What's more, it doesn't work: cat fortune | sed -r 's/^[/|$$/extract_itex]\s?([-A-Za-z ()!,?".;'a']*)[/|\$$?/\1/' Abou Ben Adhem (may his tribe increase!) Awoke one night from a deep dream of peace, And saw, within the moonlight in his room, Making it rich, and like a lily in bloom, An angel writing in a book of gold. Exceeding peace ... 5 Your idea to transform the table so that the implied keys explicitly appear on every row is an interesting one, but I think that it is overcomplicated. The backslashes to indicate continuation lines are actually superfluous here, since an unfinished$( substitution automatically causes the command to be incomplete. Similarly, ending a line with a | pipe ...

4

Assuming the "Output" section is stored in "filename.txt", and assuming you have GNU tools: grep -oP '(?<=\.\d\d)\d+\.\d\d(?=disconnect)' filename.txt That's a perl regex meaning: prececeded dot and two digits, find some digits, a dot and two digits, followed by "disconnect". grep's -o option means "output only the matched text".

4

You are parsing a JSON file using some UNIX tools. This is risky, since those can have so many edge cases that you will end up refactoring over and over. For example, you mention that sed s/\\[Placeholder.A]/${MY_VALUE}/ is giving errors when$MY_VALUE has slashes. This is normal, since this is how it gets expanded: sed s/\[Placeholder.A]/${MY_VALUE}/ ... 4 What about this? cat input | tr ' ' '\n' | grep -i "^http" Put every string on new line. Filter urls. 4 UUOC. pandoc takes the input file name as an argument. echo$file | pandoc -o outfile is equivalent to pandoc -o outfile $file. One process invocation down. UUOS (sed in this case). bash has very rich string transformation features built in. For example,${file%md} (see Parameter expansion section of man bash) will strip the md suffix form the filename. So ...

4

In the general case, the answer to any question regarding sed is "use something else." Not that sed is not useful! It definitely is! It's just that it's best for simple tasks that have compact, obvious solutions. In particular, Perl is good at replacing sed scripts, thanks to its more expressive regular expressions and uncomplicated access to ...

3

Whenever chaining awk together with grep, sed, head, tail or other text processor tools, consider the possibility of doing it all with just awk, as it's very powerful. Reducing the number of processes in the pipeline makes sense, and in some examples can drastically improve performance. Here's one way to write the two lines of pipelines as a single line (...

3

There's a gross inefficiency in this script: not only are we opening each file n times (where n is the number of lines), we're also running n find instances. Instead, consider running find just once, and using a single sed command to perform the substitutions. The easiest way to do that is to make it an executable script (with a #! line): #!/bin/sed -f s%...

3

Since we're using Bash, we can combine the echo and read by using -p (prompt) argument to read like this: read -p 'Enable aggregation [on/off] ' agg I slightly adapted the form of the prompt to be more like common conventions from other interactive programs. It's a good idea to validate the user input. If On is entered instead of on, then the user might ...

3

When processing tabular data in columns, awk is often a more appropriate tool to use. The equivalent command would be awk '{ sub("\.$", "",$NF); print $NF }' test … which I think is more readable. Explanation: NF is the number of fields: for this text, 5.$NF is the content of the last (5th) field. sub("\.$", "",$NF) strips the trailing dot from the ...

3

1. Comments on your code Your example input file is not a well-formed XML document. An XML document must have a single root element, but your example has two. What tool did you use to generate this file? You should ask yourself how it went wrong. (If you're just writing it "by hand" in some other program using a bunch of print statements, you might ask ...

3

Your suspicions are correct. The amount of work that the operating system does for you to spawn a process is formidable — much more work than the string processing work itself. For every line in the log file, you are spawning 13 processes: cut tr cut tr xargs date bash — due to the use of $( ) substitution cut tr cut tr xargs date Therefore, the ... 3 It's a couple of decades (!) since I did any CGI, but I'll cast a quick eye over this. I hope other reviewers will pitch in and fill the gaps (or correct my errors). It's an interesting choice to use the same TITLE for success and failure. It might be better to write a small function: header() { cat <<EOT Content-type: text/html; charset=us-ascii &... 2 filelist="$( IFS=' ' echo "$*" | sed -n '$!H;${x;s/.//;s/\n/ /g;p;}' )" to="$( IFS=' ' echo "$*" | sed '$!d' )" This should do, assuming there are no new lines in the parameter/s. Your code assume there is only 1 word per argument. This have nearly no impact unless the last argument is a multi word (miss interpreted by your code taking ...

2

Not bad so far! You seem to like ${name} instead of$name. That's fine, but it's not needed anywhere in the posted code. Usually it's needed in situations like ${name}_suffix, where without the${...} the "_suffix" would become part of the variable name. In general, the best way to find files and do something to them is using the -exec flag of find, for ...

2

Rewriting it in AWK would definitely result in a huge improvement, enough to say that writing it in Bash was a poor choice. Many of the considerations for this problem favour AWK: The input is line-oriented. Nearly every line has the same key = value format, except for the headers with [key = value] instead. Most importantly, they all share the same = ...

2

Consider removing tags with sed 's/<[^>]*>/ /g'

2

From the GNU sed documentation: If no -e, -f, --expression, or --file options are given on the command-line, then the first non-option argument on the command line is taken to be the script to be executed. Your two sed commands each has one non-option argument, which gets treated as the script. It would be better practice to always explicitly put a -e ...

2

This yet another way to do it: sed -r '1{s/$/ /;s/ / \n/g};:a;$!N;s/$/ /;:b;s/\n(.*\n+)(\S+\s)/\2@!@\1/;tb;s/@!@/\n/g;${s/ \n/\n/g;s/\n+$//;q};ba' This method is somewhat faster and only uses single delimiter which may be crafted to be unique. i.e. @!@ in this example 2 In addition to the other excellent suggestions, Instead of using sed:$(echo $file | sed "s/\/home\/stanek\/Downloads\///g") try bash parameter substitution: "${file//\/home\/stanek\/Downloads\//}" And remember to quote - there's a few places that have a bare $variable that shouldn't be unquoted. 2 "/home/stanek/Downloads/" is used often, consider putting that as a script variable, if it's not already present as an environment variable provided by your shell user profile? Let's assume USER_DOWNLOADS="/home/stanek/Downloads", MUSIC_DIR="/home/stanek/Music" for the suggestions below. You are invoking find three times, the problem being that new files ... 2 N;/7/!{P;D} Nice job there, this very concisely (and clearly) allows you to capture both lines $n$ and $n+1$. :b N;s/\n/&/3;Tb;d Now the loop that follows it is mostly redundant. You are essentially appending lines of input until you are left with $4$ lines in total. You already have $2$ in your pattern space and need $2$ more, i.e. lines \$...

2

Yes, sed can be used for a range of lines, for example: sed -e '3,5s/^/!/' scm_code_ex.txt This inserts a ! at the start of line 3, 4, and 5. I would write your script like this, and then I'll explain the details: #!/bin/bash # This script will comment in or out sections in the code text file file=scm_code_ex.txt read -p "- Aggregation (on) or (off) (...

2

I wouldn't use regular expressions at all here. I'd use printf to pad the digits, and bash glob-patterns to extract the digits. shopt -s extglob for filename in *; do tmp=${filename%.txt} # remove the ".txt" extension digits=${tmp##*_} # remove everything up to the final underscore case \$digits in +([0-9])) # 'digits' contains only ...

Only top voted, non community-wiki answers of a minimum length are eligible