Questions tagged [awk]

AWK is an interpreted programming language designed for text processing and typically used as a data extraction and reporting tool.

Filter by
Sorted by
Tagged with
4
votes
1answer
66 views

Removing duplicate field entries from sorted csv data

Given the following input (cat i.txt), I want to remove duplicate field entries in each of the first three columns and none of the others. ...
0
votes
2answers
48 views

Select a specific line after matching a pattern

I just recently started using AWK and I'm still learning about it. I have solved the problem I'm about to show but I feel it's not the best solution and I'm trying to find a solution to fit within an ...
4
votes
1answer
67 views

Select second field in text table with shell script

I am writing a bash script to mount openshift service accounts into kubernetes objects. The right secret to use is highlighted in this text: $ oc describe sa sa-...
3
votes
2answers
64 views

Code to get the IP address and format it to run in nmap

For scanning my subnet using my IP address, I need to run the following command nmap -sn 192.168.100.* I want to extend this functionality such that I don't have ...
2
votes
0answers
32 views

Summary data using csv file

Please, Is there the possibilty to improve the code below. I am sure the result i got, can be get with shorted code. code ...
6
votes
1answer
328 views

Updating my package version using only a Makefile

So I have written a Makefile that has a release function that can be run like: make release bump=patch and I want it to be fully ...
7
votes
3answers
581 views

Print code-fenced sections of a Markdown document

Original code and demo at this gist. Given a Markdown document like ...
2
votes
1answer
47 views

Counts and average code

Using the columns 4 and 2 will create a report like the output file showed below. My code works fine but I believe it can be done shorter. I have a doubt in the part of the split: ...
3
votes
2answers
50 views

Bash script for managing hashtag notes

I have written a simple script for managing hashtag notes. I am not familiar with bash scripting so I would really appreciate it if someone could give me some pointers about it. ...
4
votes
1answer
42 views

Remove every line that include and follows the second occurrence of a given pattern

From a file, I am trying to remove every line that include and follows the second occurrence of a given pattern in Bash (Mac OSX). Note, that a file may have only one match of the given pattern (and ...
4
votes
1answer
73 views

Try every option for a command and make anagrams out of the successful ones

What I have: I am trying to find what words you can make out of Linux command options like ls -cat or ps -elf. To do that I am ...
2
votes
1answer
87 views

Extract thoughput value from JMeter log file

I use JMeter for load testing and want to report error on CI pipeline when application though put is less than 1/sec. JMeter logs summary of results as - ...
3
votes
1answer
73 views

ECG Bash selection tool

I made the following bash script for extracting a group of ECG signals from ECG files. I would like to know if there is any mistakes and/or weaknesses. I have experienced difficulties in integrating ...
5
votes
1answer
1k views

Python library for awk-like file manipulation

I recently published a library for advanced awk-like file manipulation in Python 3. The code can be found here and here is the documentation. It is also available for download from pip (...
2
votes
1answer
139 views

Parsing GTF file using command-line

I am extracting exons details from a GTF file using command line in Unix like cut, awk, grep or sed. input file.gtf: ...
4
votes
2answers
228 views

Generate SQL UPDATE from Excel CSV file

I must write updates in a database from a flat file (CSV). I want to do that in the shell, with tools such as AWK. ...
4
votes
1answer
183 views

Find and report all occurrences of duplicate lines in a text file

I would like you to take a look at this simple script I wrote that's supposed to emulate the Unix sort [FILE] | uniq -cd command. What's different about my script ...
0
votes
1answer
36 views

Loading a file in vim via piped output of ag

I feel as if my scenario is fairly simple but that I might be overcomplicating it. I am searching through files looking for a special condition, that is where the method ...
2
votes
1answer
466 views

Finding hidden field values in a web page before submitting a login form

In order to submit a login form using wget, I need to find the values for the hidden field that were added to the form recently. Grepping for the names gives the ...
2
votes
1answer
56 views

Listing Docker Images without registry URI

I have a private container registry behind a firewall that will need some Google registry images. To make sure things are updated sanely, I'm writing a mirror script in Bash. I have one line that ...
3
votes
2answers
220 views

Advanced CSV-to-SQLite converter

I just wrote an advanced CSV parser that translates CSV to SQL(SQLite Compatible) statements, which are then inserted into the database. It translates variable names in the CSV to values defined in ...
1
vote
1answer
34 views

Print first element of the last line of files when the second element does not match a pattern

I am relatively new to Bash, regular expression and stuff. In a directory, I need to read the last line of all the files that contain the word "fitness", and list (on one line and separated with ...
6
votes
3answers
209 views

Shell script to count chess game outcomes

I came across this blog post by Adam Drake from around a year ago which is now making the rounds again. I made some improvements to his code, but wish to see if there are additional tweaks that could ...
7
votes
1answer
72 views

Calculating percentages in arbitrary number of columns

Given this sample input: ...
4
votes
2answers
2k views

Checking percentage of free memory using top and awk

The following Awk code was created in order to verify if free memory from the top command is less than 20% of the total. I've noticed that I actually do not need the ...
3
votes
1answer
1k views

Analyse an Nginx access log with awk and produce a report

I'm looking for your opinions on anything that I am doing wrong in the application below, such as best practices, glaringly horrible errors or even just your own personal opinion. Task: Grab an Nginx ...
10
votes
3answers
19k views

Extracting data from text file in bash using awk, grep, head and tail

I've been writing bash script on and off, with pretty good results in terms of getting the job done. However, I'm worried that my script might be very ugly, as I am a beginner. I'm looking for advice ...
4
votes
2answers
95 views

Filter shell script to find lines that contain all specified patterns

I wrote a script that does the following: Run another script on the system Filter the output to find lines that contain ALL of the given patterns Pipe the output to a second script on the system I ...
3
votes
1answer
72 views

Extracting performance statistics from FTP session transcript

I've borrowed and written the following code to output the disconnect time. All works well but I'm curious as to how I could tighten/ shorten the code. If anyone feels like having some fun then I'd ...
8
votes
1answer
739 views

Bash script - automate an upload image + description process on a server

I was tasked at work to improve a system where through a web interface a client, which eventually became us, uploads a bunch of images with descriptions (latter coming from csv files), the images are ...
4
votes
3answers
190 views

Removing NULL / empty fields

just wanted to check with you could this be done better: ...
4
votes
1answer
70 views

Input Columns with Awk

I wrote a small script to manipulate fields of a file based on the input of a user. Is there something I could've done better, or perhaps a better refactor? I do know however that I need a way to ...
6
votes
2answers
190 views

Bash log monitoring

I've made a Bash script to monitor some server log files for certain data and my method probably isn't the most efficient. One section specifically bugs me is that I have to write a newline to the ...
5
votes
1answer
231 views

Directory's disk usage list

For my classes, I had to finish this task: Directory's disk usage list For indicated directory print a list of files and subdirectories in descending order according to their total disk ...
2
votes
1answer
124 views

Linking input/output states and conditions in an input file

Consider the following CSV file: A; B; ; B; ; A; C; ; E F; D; ; E; E; C; ; The fields: $1: the ...