Over the last 3 days I have been trying to learn how to do bulk edits of files. I’ve decided to change the license of my programs to the Affero GPL v3, and wanted to change the notices in my files to reflect that. So I learned that in CL the only way to do this is to read the files into a string, and then run cl-ppcre on that string, and then output that string back into the file.

That was proving to be a headache for me, and I didn’t want to risk polluting my files. So I decided to go with a more tried and tested system. I found that for bulk edits, sed is a useful tool, and that perl is used a lot for this too. The link that I found to be the most useful is here. It gave me the basic template that I used in the end. But first lets have a look at the options I looked at.

Sed

Very nice system. Seems to be used a lot. Seems to be liked a lot. The problem I had with it was that it didn’t work on a multi-line basis. I looked at the reference manual and this helpful page. Lots of good stuff there, but I wanted something that would work multi-line.

In the process, I also came across this page, which I think would be quite useful to know. Good amount of information on this site.

Emacs

Emacs is the text-editor that I use. Its amazing. And it has a regex-replace function. Awesome. Thats just what the doctor ordered. So lets fire it up in batch-mode (Which I’ve never done before, and was looking forward to learning how) and get this done. No need to learn anything else apart from some Emacs Lisp, and that doesn’t seem so difficult.

One problem, though. I couldn’t find any reference as to how to get the regex-replace working in a non-interactive manner. So I figure that it can’t make any sense to do it through batch-mode. Too bad, I was looking forward to learning how to use that. Now, I’m not saying that its not possible, just that I couldn’t find any guide on the internet that showed me how to do it. I’m fully expecting someone to comment on this and show me how its done in emacs-lisp in a few lines of code. I hope that happens. It would be good to learn something.

Perl

And that brings me to perl. I looked at the link that I gave at the top of the post, and it seemed to have the basic information I wanted/needed. I looked up the Perl regular expressions page, and its got a lot of great information. Perl’s regex system seems to be the most featureful out there. Enough people refer to it, which certainly satisfies my ‘tried and tested’ requirement.

I don’t know any perl. This has been my introduction to it. I have found it to be very useful. It took a few tries to get it working just the way I wanted it to, and the find utility is amazing. I have gotten to learn more about it in the past few days, and it rocks.

Anyway, the final code I came up with to do my bulk-editing is

find -name '*.lisp' -execdir perl -0777 -pi.bak -e 's/^;; Copyright.*n;; Distribu.*$/;; Copyright 2011-2012 Ravi Desai rd7190@gmail.comn;; Distributed under the terms of the GNU Affero GPL version 3 or any later version./mi' '{}' '+'

The find utility spits out a list of all files with ‘.lisp’ in the last part of their name and feeds it to the perl system. The -0777 flag tells perl to just ‘slurp’ in the entire file (As opposed to reading in only one line at a time, which seems to be default). ‘Slurp’ing the file is important since I’m doing a multiline substitution. The ‘i.bak’ flag says that the files should be edited in-place. The ‘.bak’ part is optional, and says that the original files should be copied with a ‘.bak’ extension. This serves as a backup. Very good idea. The in-place modification flag is very nice. I like it. It certainly saved me a lot of work compared to doing the read-into-string, change-string, output-string-back-into-file thing that I would have had to do in CL. The ‘p’ flag does a looping thing that I don’t quite fully understand yet. This information is available here. I have to hand it to the perl people; they certainly know how to keep all the help files in one organized place. Kudos.

Wrap up

In conclusion, I think the perl system rocks (In this domain, at least). I don’t even know where to begin with having something this simple (1 line of bash code) in CL. If there is a way, please let me know. Thanks.