Developing One Liners with Command Substitution

Understanding differences between the output of one command being taken as the input (the argument) for another, OR whether the output of a command is read by the shell (an unseen "subshell" if you like) and interpreted (or not) as a command or an argument in itself, can be very confusing when piping commands, depending what commands, where they are in the chain, and whether they accept STDIN or not.

The difference between variable expansion using the $() format or whether backtics `$5`; double quotes "$5"; single quotes '$5' or no quotes $5; can be easily mistaken - especially if many braces {}, brackets () or slashes // etc are involved.

The content of the backtics example is taken as a command for the shell to run first, whose results may become an argument, but the $() expansion needs to be run by a command so it becomes the argument. Confused? These example should summarise some variations:

stevee@AMDA8 ~/Awk $ b=5; echo $b
5
stevee@AMDA8 ~/Awk $ b=5; echo "$b"
5
stevee@AMDA8 ~/Awk $ b=5; echo '$b'
$b
stevee@AMDA8 ~/Awk $ b=5; echo `$b`
5: command not found 

stevee@AMDA8 ~/Awk $ $(b)
b: command not found

stevee@AMDA8 ~/Awk $ $b
5: command not found

stevee@AMDA8 ~/Awk $ printf $b
5

stevee@AMDA8 ~/Awk $ printf $(b)
b: command not found

stevee@AMDA8 ~/Cprogs $ a=1; b=2; expr $a+$b
1+2
stevee@AMDA8 ~/Cprogs $ a=1; b=2; expr a+b
a+b
stevee@AMDA8 ~/Cprogs $ echo `expr $a + $b`
3
stevee@AMDA8 ~/Cprogs $ echo `expr $a - $b`
-1
stevee@AMDA8 ~/Cprogs $ echo `expr $a \* $b`
2
stevee@AMDA8 ~/Cprogs $ echo `expr $a / $b`
0
stevee@AMDA8 ~/Cprogs $ echo `expr $b / $a`
2

stevee@AMDA8 ~/Awk $ $USERNAME
stevee: command not found

It pays to experiment and practise to get answers. For example, nmap can take a list of IP addresses to scan, if separated by whitespace, including tabs, e.g.

nmap 192.168.1.2     192.168.1.3

Normally, you may feed nmap a prepared IP list from a file if there are many targets, but what if another command outputs some IPs you wanted to use, but were not in a space delimited format, like arp -a for example? This gives the MAC and IP addresses for all devices up on a network. On a class C /24 network that's up to 254 device addresses. You could direct them into a text file and edit them, then feed the list to nmap that way:

nmap -iL iplist.txt

but depending where you got the list from, say a DHCP list, it may still need formatting to be suitable anyway, such as the DHCP list from my router accessed by webpage:

1 192.168.1.2 00-1A-A0-5D-1D-AD FIXED IP
2 192.168.1.3 B8-27-EB-E8-6B-1D FIXED IP
3 192.168.1.4 B8-27-EB-71-14-27 FIXED IP
4 192.168.1.5 00-23-54-3A-EB-9A FIXED IP
5 192.168.1.7 B8-27-EB-73-A8-A5 FIXED IP
6 192.168.1.8 B8-27-EB-B9-D1-C0 FIXED IP
7 192.168.1.14 5C-93-A2-D5-83-3B 23:41:21

A better solution may be to create a one-liner to use anywhere else in future, from another generally system available source/cmd like:

arp -a
? (192.168.1.3) at b8:27:eb:e8:6b:1d [ether] on wlan0
? (192.168.1.4) at b8:27:eb:71:14:27 [ether] on wlan0

Practise inventing situations like this to get to know how to output what you want for the commands you know, then look at optimising it after it works - whether one command can act in place of itself in one instance rather than 2, or whether it has better options than you know by reading it's man page.

To get the IPs in a format that nmap can accept, I need the IPs on a single line, so building in stages: first isolate the 2nd field:

arp -a | awk '{print $2}'
(192.168.1.3)
(192.168.1.1)
(192.168.1.4)
(192.168.1.7)
(192.168.1.2)
(192.168.1.5)
(192.168.1.8)

Remove the brackets by substituting the first and last chars between the "anchors"; the first (^) and last ($) non printable line delimiters - that define a line. The dot represents ANY single char before and after them so matches each bracket ():

arp -a | awk '{print $2}' | sed 's/^.//g' | sed 's/.$//g'
192.168.1.3
192.168.1.1
192.168.1.4
192.168.1.7
192.168.1.2
192.168.1.5
192.168.1.8

Now they need to be on one line, by having the newline (\n) stripped and replaced with whitespace or a tab (\t), but the paste command was designed for this already:

arp -a | awk '{print $2}' | sed 's/^.//g' | sed 's/.$//g' | paste -s
192.168.1.3 192.168.1.1 192.168.1.4 192.168.1.7 192.168.1.2 192.168.1.5 192.168.1.8

Or tr and perl are better to use when learning concepts, as sed and awk don't work in the same simple search and replace format for "\n" type chars in Mint.

This seems to be what paste -s uses anyway, so you could also use 1 white space " " instead of a tab \t, also:

arp -a | awk '{print $2}' | sed 's/^.//g' | sed 's/.$//g' | tr "\n" "\t"
192.168.1.2 192.168.1.7 192.168.1.1 192.168.1.3 192.168.1.8 192.168.1.4 192.168.1.5

arp -a | awk '{print $2}' | sed 's/^.//g' | sed 's/.$//g' | perl -pe 's/\n/\t/g'
192.168.1.2 192.168.1.7 192.168.1.1 192.168.1.3 192.168.1.8 192.168.1.4 192.168.1.5

sed and awk fail for this simple format requiring more cryptic parameters:

arp -a | awk '{print $2}' | sed 's/^.//g' | sed 's/.$//g' | sed 's/\n/\t/g'
192.168.1.2
192.168.1.7...

arp -a | awk '{print $2}' | sed 's/^.//g' | sed 's/.$//g' | sed '{:q;N;s/\n/ /g;t q}'
192.168.1.2 192.168.1.7 192.168.1.1 192.168.1.3 192.168.1.8 192.168.1.4 192.168.1.5

arp -a | awk '{print $2}' | sed 's/^.//g' | sed 's/.$//g' | awk 'gsub("\n","\t")'

no OP

arp -a | awk '{print $2}' | sed 's/^.//g' | sed 's/.$//g' | awk '{printf "%s ",$0} END {print ""}'
192.168.1.2 192.168.1.7 192.168.1.1 192.168.1.3 192.168.1.8 192.168.1.4 192.168.1.5

Now the result is in the right list form as an argument that nmap requires to read multiple IPs, but how to get all this on the command line AFTER the nmap command when nmap cannot take input from a pipe end, such as 192.xxx | nmap.

This is where you see how the shell handles different output as an argument or a command, so works or not, if either any form of quotes is used (fail) or whether variable expansion or backtics is used (pass):

nmap 'arp -a | awk '{print $2}' | sed 's/^.//g' | sed 's/.$//g' | paste -s'

Starting Nmap 6.40 ( http://nmap.org ) at 2016-09-04 23:02 BST
Failed to resolve "arp -a | awk {print".
Unable to split netmask from target expression: "} | sed s/^.//g | sed s/.$//g | paste -s"
WARNING: No targets were specified, so 0 hosts scanned.
Nmap done: 0 IP addresses (0 hosts up) scanned in 0.30 seconds

nmap "arp -a | awk '{print $2}' | sed 's/^.//g' | sed 's/.$//g' | paste -s"

Starting Nmap 6.40 ( http://nmap.org ) at 2016-09-04 23:03 BST
Unable to split netmask from target expression: "arp -a | awk '{print }' | sed 's/^.//g' | sed 's/.$//g' | paste -s"
WARNING: No targets were specified, so 0 hosts scanned.
Nmap done: 0 IP addresses (0 hosts up) scanned in 0.13 seconds

nmap $(arp -a | awk '{print $2}' | sed 's/^.//g' | sed 's/.$//g' | paste -s)

Starting Nmap 6.40 ( http://nmap.org ) at 2016-09-04 23:00 BST
Nmap scan report for 192.168.1.3
Host is up (0.013s latency).
Not shown: 995 closed ports
PORT STATE SERVICE
22/tcp open ssh
139/tcp open netbios-ssn
445/tcp open microsoft-ds
8080/tcp open http-proxy
8081/tcp open blackice-icecap....

nmap `arp -a | awk '{print $2}' | sed 's/^.//g' | sed 's/.$//g' | paste -s`

Starting Nmap 6.40 ( http://nmap.org ) at 2016-09-04 23:00 BST
Nmap scan report for 192.168.1.3
Host is up (0.013s latency).
Not shown: 995 closed ports
PORT STATE SERVICE
22/tcp open ssh
139/tcp open netbios-ssn
445/tcp open microsoft-ds
8080/tcp open http-proxy
8081/tcp open blackice-icecap...

You can see how cryptic looking this can get for a working example of backtics and tr, and being aware of what spaces are required or not:

nmap `arp -a|awk '{print $2}'|sed 's/^.//g'|sed 's/.$//g'|tr "\n" " " `

There are some oddities that can occur if you investigate these variations, for example, the live ASCII bar chart example has uptime embedded in a perl function inside backtics, and works as intended:

perl -e 'while(1) {`uptime` =~ /average: ([\d.]+)/; printf("% 5s %s\n", $1, "#" x ($1 * 10)); sleep 1 }'

3.41 ##################################
3.41 ##################################
3.45 ##################################
3.45 ##################################
3.26 ################################

BUT..If you replace the backtics with single quotes, an error is not thrown, and the command does not "fail" to work completely, but it gives no output, and loops until stopped:

perl -e 'while(1) {'uptime' =~ /average: ([\d.]+)/; printf("% 5s %s\n", $1, "#" x ($1 * 10)); sleep 1 }'

no OP

no OP...

Yet if you run uptime alone in single quotes, it gives the same output as it would without single quotes, yet that data does not become an argument for the function as before:

'uptime'
23:50:05 up 6:27, 3 users, load average: 3.60, 3.81, 4.63

The same occurs for double quotes.

For an attempt at variable expansion, it fails noisily:

perl -e 'while(1) {$(uptime) =~ /average: ([\d.]+)/; printf("% 5s %s\n", $1, "#" x ($1 * 10)); sleep 1 }'
Bareword found where operator expected at -e line 1, near "$(uptime"
(Missing operator before uptime?)
syntax error at -e line 1, near "$(uptime"
Execution of -e aborted due to compilation errors.

The cause is shown by trying to run the expansion alone, as the first space delimited output of the uptime command - the time of day - is read by the shell as a potential command, which of course, does not exist.

$(uptime)
23:54:07: command not found

Yet it can be run as a variable expansion argument for the right command:

echo $(uptime)
23:58:39 up 6:37, 5 users, load average: 0.96, 0.85, 1.09

BUT, it cannot be run in the function like that inside backtics, even though echo is present to try to expand it:

perl -e 'while(1) { `echo $(uptime)` =~ /average: ([\d.]+)/; printf("% 5s %s\n", $1, "#" x ($1 * 10)); sleep 1 }'
sh: 1: Syntax error: ")" unexpected

perl -e 'while(1) { `echo` $(uptime) =~ /average: ([\d.]+)/; printf("% 5s %s\n", $1, "#" x ($1 * 10)); sleep 1 }'
Scalar found where operator expected at -e line 1, near "`echo` $("
(Missing operator before $(?)
Bareword found where operator expected at -e line 1, near "$(uptime"
(Missing operator before uptime?)
syntax error at -e line 1, near "`echo` $("
Execution of -e aborted due to compilation errors.

Comments are closed.

Post Navigation