deletion and replacement of strings in bash

I try to record useful one-liners for future reference. I forgot to write down what this one does:

    mv $f ${f##START}.${f%%${f##START}}

This awkwardly-timed (time-zone troubles) blog post is atonement for my carelessness. Like all one-liners it looks complicated but is pretty simple. It does this:

    > f=STARTonetwothree 
    > echo ${f##START}.${f%%${f##START}}

The ${...} construction is brace expansion, which allows us to generate strings. In stages we have


This takes the string f and deletes the substring START from the beginning. In fact, it deletes the longest match to the substring, which makes sense when you allow your substring to be something interesting including wildcards (which may have numerous matches). A single # would delete the shortest match: try

    > echo ${f#S*T}

    > echo ${f##S*T}

To delete the substring from the end of the string, we would use %% or %, where once again two means longest and one means shortest.

So my horrible code fragment creates f' by removing START from the start of f, then creates f'' by removing f' from the end off, combines these with a period in the middle and renames the file named f with this new string-aberration. An easier way to achieve the same result would have been

   mv $f ${f##START}.START

but this does not generalise to the case of an arbitrary substring.

The obvious application of this is removing/modifying file extensions. To rename all .tgz files as .tar.gz, for example:

    for f in *.tgz do mv $f ${f%%tgz}tar.gz done

If the offending substring is not at the beginning or end, you could use replacement:

    > f=all_workERROR_and_ERRORno_playERROR
    > echo ${f//ERROR/}

The syntax is ${string/substring/newsubstring}, where a double first slash (as in previous example) replaces all instances of substring. We can also do partial matching

    > echo ${f/E*R/}

and of course replacement with something other than an empty string:

    > fp=${f/E*R/_and_all_play} 
    > echo ${fp/all/no}

So if you have a bunch of messy gzipped files like

the solution is

    for f in *.gz do mv $f ${f//.gz/}.gz done

and to wonder how you got into that mess in the first place.