Manage all those notes

2019-11-13 @Technology

At some point note management became a wild-west enterprise. I annotated information in an appalling variety of sources and mediums. Journal entries, idea lists, dedicated notes files, calendar entry notes, emails, blog posts, source code, and web bookmarks constitute some of the disparate digital sources.

Among the analog sources come to mind notebooks, leaflets of paper, napkins, bookmarks (for books) that double as note-taking parchment, and index cards as the most recent and most frequently interfaced for all my work.

The situation is actually much better than appears. In fact, I feel quiet content and laid-back with regard to the present system. The reasons are the following.

One, absolutely all of my digital content constitutes offline plain text, which means easily searchable, revision-control maintainable, and versatile. No binary data, no complex formats, no internet dependency. I store backups online, but interface directly with all local content, email included. This offline plain-text strategy alone eliminates most of the information storage anxiety that may otherwise have resulted.

Two, I recognize that the more recent and mission-critical content spans a small, palatable number of sources. Among the analog it comprises the fairly well categorized index cards. Among the digital, it matters even less for reasons I’m about to demonstrate.

Once the digital content not only dispersed among too many sources, but caused the task of recalling them ever more challenging, I wrote a CLI utility srchx. You can find it in the repository automations.

With srchx you can abstract a series of disparate sources (files and/or directories) into descriptive content labels (notes = ~/.notes.txt, ~/journal, /ideas, /research, etc). It also optionally abstracts the search terms under tags (ie linux = shell, bash, sed, awk, grep, cli, vim, mutt).

Therefore, instead of relying on something like grep with the respective flags, and the remembrance of all relevant content sources, I can conduct a simple label/tag based search, such as ‘srchx notes linux’ to search all the respective sources for anything linux related. Alternatively, srchx enables free form search keywords.

It really made my day once I found myself conducting the following kinds of searches, and seeing all relevant results appear in one output. (What you don’t see here is the convenient color coding, which highlights the sources as well as the specific search terms.) Bear in mind that the search results display only the relevant line of text.

Search all notes for ‘potassium’:

$ srchx notes potassium

~/journal.txt:Need to supplement with sodium, magnesium, potassium...
~/notes/nutrition.txt:Banana ... Also Vitamin B6, C, potassium, manganese

Search for anything related to Toastmasters:

$ srchx notes toastmasters

~/.notes:@url https://www.toastmasters.org/
~/journal.txt:2018-08-21 22:22 @idea @write @Toastmasters speech ideas
~/journal.txt:2018-08-23 09:38 @toastmasters lessons learned.
~/journal.txt:2018-08-26 10:25 @link @Toastmasters SpeakEasy
~/notes/lists.txt:    - About Toastmasters
~/notes/presentation_ideas.txt:    Keys: TM = Toastmasters, PSC = public speaking club...
~/notes/research/public-speaking-clubs.txt:Toastmasters International – for clubs in the US and internationally.

How about the (case-insensitive) search of all notes, blog entries and bookmarks for anything containing ‘lisp’, displaying just the number of matches next to each source:

$ srchx -r -c blog bookmarks notes lisp

   1 : ~/blog/posts/2018/06/meta-writing-language-conflict.md
   1 : ~/blog/posts/2018/08/alternative-things.md
   1 : ~/blog/posts/2019/03/creating-diagrams.md
   1 : ~/blog/posts/2019/03/creating-diagrams.md
   1 : ~/notes/lists.txt
   1 : ~/notes/reading_list.md
   2 : ~/notes/linux/irc-notes.txt
   4 : ~/blog/posts/2018/08/jrnl.md
  10 : ~/journal.txt
  10 : ~/blog/posts/2018/07/about-lisp-and-programming-languages.md
  11 : ~/blog/posts/2019/11/meta-programming-languages.md
  11 : ~/notes/linux/arch.txt
  19 : ~/notes/tech/lisp.txt
  38 : ~/.w3m/bookmark.html

See the repository for more detailed instructions as well as an example configuration file.

Questions, comments? Connect.