[COFF] Re: Requesting thoughts on extended regular expressions in grep.

Thanks, Grant and contributors in this thread, Great thread on RE's. I bought and read the book (it's on the floor over there in the corner and I'm not getting up). My task was finding dates in binary and text files. It turns out RE's work just fine for that. Because I was looking at both text files and binary files, I wrote my stuff using 8-bit python "bytes" rather than python "text" which is, I think, 7-bit in python. (I use python because it works on both Linux, Macs and Windows and reduces the number of RE implementations I have to deal with to 1). I finished my first round of the program late fall of 2022. Then I put it down and now I am revisiting it. I was creating: A Python program to search for media files (pictures and movies) and copy them to another directory tree, copying only the unique ones (deduplication), and renaming each with *YYYY-MM-DD-* as a prefix. Here is a list of observations from my programming. 1. RE's are quite unreadable. I defined a lot of python variables and simply added them together in python to make a larger byte string (see below). The resulting expressions were shorter on screen and more readable. Furthermore, I could construct them incrementally. I insist on readable code because I frequently put things down for a month or more. A while back it was a sad day when I restarted something and simply had to throw it away, moaning, "What was that programmer thinking?". Here is an example RE for YYYY-MM-DD # FR = front BA = back # ymdt is text version ymdt = FRSEP + Y_ + SEP + M_ + SEP + D_ + BASEP ymdc = re.compile( ymdt ) 1a. I also had a time defining delimiters. There are delimiters for the beginning, delimiters for internal separation, and delimiters for the end. The significant thing is I have to find the RE if it is the very first string in the file or the very last. That also complicates buffered reading immensely. Hence, I wrote the whole program by reading the file into a single python variable. However, when files become much larger than memory, python simply ground to a halt as did my Windows machine. I then rewrote it using a memory mapped file (for all files) and the problem was fixed. 2. Dates are formatted in a number of ways. I chose exactly one format to learn about RE's and how to construct them and use them. Even the book didn't elaborate everything. I could not find detailed documentation on some of the interfaces in the book. On a whim, I asked chatGPT to write a python module that returns a list of offsets and dates in a file. Surprisingly, it wrote one that was quite credible. It had bugs but it knew more about how to use the various functional interfaces in RE's than I did. 3. Testing an RE is maybe even more difficult than writing one. I have not given any serious effort to verification testing yet. I would like to extend my program to any date format. That would require a much bigger RE. I have been led to believe that a 50Kbyte or 500Kbyte RE works just as well (if not as fast) as a 100 byte RE. I think with parentheses and pipe-symbols suitably used, one could match Monday, March 6, 2023 2023-03-06 Mar 6, 2023 or ... I'm just guessing, though. This thread has been very informative. I have much to read. Thank all of you. Ed Bradford Pflugerville, TX On Thu, Mar 2, 2023 at 12:55 PM Grant Taylor via COFF <coff(a)tuhs.org> wrote: -- Advice is judged by results, not by intentions. Cicero