Here’s a couple one-liner shell commands collected over the last couple months when it occurred to me to record them. Each of these I thought were somewhat notable at the time I did so.
Nested “backquotes”
I often have to run commands where it is convenient to have the parameters of the commands in a file. A simple example is to edit all the files in a list of files, say:
A useful variation of this is to do the same using the output of a command that also takes its input from a file. Here’s one to edit all the “ancestor” files in the version control system, assuming a command vcsancestor that produces such filenames
Observe how two different methods of embedding shell commands can be combined into one command. In the past I often used for loops for something like this, say:
(because backquotes can’t be nested). It only recently occurred to me that this isn’t a limitation if $() style subshells are used.
Batching commands with xargs
When working in a version control system, it’s often useful to do a batch checkout of all the files that have compilation errors. Suppose that you made changes that produced the following compilation error output:
$ cat compile.errors "satauth.C", line 978.30: 1540-0274 (S) The name lookup for "sqlorest" did not find a declaration. "scrutil.C", line 142.52: 1540-0274 (S) The name lookup for "SQLNLS_SAME_STRING" did not find a declaration. "testdrv.C", line 1146.16: 1540-0274 (S) The name lookup for "SQLO_OK" did not find a declaration. "testdrv.C", line 183.15: 1540-0274 (S) The name lookup for "SQLO_OK" did not find a declaration.
Here’s a one liner to checkout all the files in this list of compilation errors (this is AIX xlC error output):
The cut command selects just the (first) double-quote delimited text, then dups are removed with sort -u, and finally xargs is used to run a command on each of the files in the resulting output
Looking for a subset of information delimited by markers on separate lines
grep works nicely for matching patterns that are constrained to a single line. If you are using gnu-grep you can use the -A and -B options to find stuff after and before the pattern of interest. As an example, in our stacktrace files (a post mortem crash dump format), we have output that includes:
<pre>
<StackTrace>
—–FUNC-ADDR—- ——FUNCTION + OFFSET——
0x00002AAAC74EF263 ossDumpStackTraceInternal(unsigned long, OSSTrapFile&, int, siginfo*, void*, unsigned long, unsigned long) + 0x06e3
0x00002AAAC74EFE89 ossDumpStackTraceV98 + 0x007f
0x00002AAAC74E5C5F OSSTrapFile::dumpEx(unsigned long, int, siginfo*, void*, unsigned long) + 0x04db
0x00002AAABA6EB313 sqlo_trce + 0x0a6f
0x00002AAABA9C52B5 sqloDumpDiagInfoHandler + 0x047b
0x00002AAAAABD5E00 address: 0x00002AAAAABD5E00 ; dladdress: 0x00002AAAAABC8000 ; offset in lib: 0x000000000000DE00 ;
0x00002AAAAABD30A5 pthread_kill + 0x0035
0x00002AAAB5D828DF ossPthreadKill(unsigned long, unsigned int) + 0x0053
0x00002AAABA9C6CA1 sqloDumpEDU + 0x0091
0x00002AAABED7A853 sqlzerdm + 0x149b
0x00002AAAB5D7D745 sqle_remap_errors(int, sqlca*, sqeAgent*) + 0x01c9
0x00002AAAB5DE8717 sqeApplication::AppStopUsing(sqeAgent*, unsigned char, sqlca*) + 0x10b1
0x00002AAAB5D46FF5 address: 0x00002AAAB5D46FF5 ; dladdress: 0x00002AAAAACE1000 ; offset in lib: 0x000000000B065FF5 ;
0x00002AAAB5D4073F address: 0x00002AAAB5D4073F ; dladdress: 0x00002AAAAACE1000 ; offset in lib: 0x000000000B05F73F ;
0x00002AAAB5D44F35 sqleIndCoordProcessRequest(sqeAgent*) + 0x3959
0x00002AAAB5DA8E55 sqeAgent::RunEDU() + 0x061b
0x00002AAABEDAC2C7 sqzEDUObj::EDUDriver() + 0x035d
0x00002AAABEDABBD7 sqlzRunEDU(char*, unsigned int) + 0x0053
0x00002AAABA9BFC62 sqloEDUEntry + 0x1460
0x00002AAAAABCE2A3 address: 0x00002AAAAABCE2A3 ; dladdress: 0x00002AAAAABC8000 ; offset in lib: 0x00000000000062A3 ;
0x00002AAAC7F376DD __clone + 0x006d
</StackTrace>
</pre>
Here’s a one-liner to grab just the portions of these files within the delimiters (with some other filtering that isn’t of terrible interest to describe)
for i in *stack* ; do grep -A40 ‘<StackTrace’ $i | grep -v ‘(/’ | grep -B40 ‘/StackTrace’ | c++filt ; done | less
Unix to Windows path separator switching
Suppose we have some unix filenames
</pre> $ head -5 f /vbs/bin/AEDefines.pm /vbs/bin/AEMacro.pm /vbs/bin/bld_shared_lib_Darwin /vbs/bin/chglibpaths /vbs/bin/chglibpaths_Darwin
</pre> $ head -5 f | tr / '\\' bin\AEDefines.pm bin\AEMacro.pm bin\bld_shared_lib_Darwin bin\chglibpaths bin\chglibpaths_Darwin <pre>
The tr command above looks a bit like ascii barf, and will translate forward slashes to backward slashes (perhaps for input that’s a list of files).
I didn’t understand the requirement to both single quote the backslash as well as escaping it, but Darin explained it for me:
Quotes allow the backslashes to go through the shell to tr. And tr has its own backslash escape mechanism (so you can do things like transform \n into \r or something – where you’d then specify ‘\n’ or just \\n and ‘\r’ or \\r).
Vim: replace search results with contents from a file
Probably related to merging conflicting changes, I wanted to completely replace the implementation of a particular function:
This was an easy way one liner method to do that replacement, deleting the implementation of foo, and replacing it with the one that was found in the file ‘foo’
file:line: delimited output for a single file
The grep -n command is very handy for producing file:line:content delimited output. In particular, you can iterate over such output with vim -q. When you want to do this for a single file, grep -n doesn’t include the filename, defeating a subsequent vim -q (since vim then doesn’t know what file to open). Here’s an example
$ cat my_file_to_search blah patternOfInterest hi foo goo patternOfInterest bye blah patternOfInterest hi blah patternOfInterest hi foo goo patternOfInterest bye foo goo patternOfInterest bye $ grep -n patternOfInterest my_file_to_search | tee v 1:blah patternOfInterest hi 3:patternOfInterest bye 4:blah patternOfInterest hi 5:blah patternOfInterest hi 7:patternOfInterest bye 9:patternOfInterest bye
To get vim -q’able output, just include a second non-existent dummy file in the search
I usually use a very-short filename for the “does not exist file”, say, .u (which presumes I also don’t create little hidden files .u in my day-to-day work).