jrnl, from plain CLI journaling to time reporting

2018-08-18 @Technology

What I love about simple CLI Linux utilities is that one, many are old, well-documented and extremely light weight, two, they are modular and can easily interact with other CLIs to expand their functionality, and three, with some imagination, you can develop simple, cheap solutions to rather elaborate problems.

jrnl is an example of such a wickedly simple, yet powerful CLI utility. It may resemble a pure journaling system on the surface, but the few of its basic features allow it to serve in the course of many time-sensitive documentation, management and reporting tasks.

jrnl stores all content in one or a number of plain text files (journals), depending on your method of interaction. Initially, journal.txt is the default configured journal. I find the plain text format fantastic, as such files are not only notoriously simple to interact with, but you don’t face risk of suddenly losing access to the application and becoming unable to access the content.

At its simplest, you can input some entries, prefixed with the current timestamp by default, as follows:

$ jrnl Today\'s plan. Write, buy vegetables, skateboard.
[Entry added to default journal]

$ jrnl Afternoon break. Ordered coffee and explored new ideas.
[Entry added to default journal]

$ jrnl Evening reflection. Exercised and discovered a new forest trail.
[Entry added to default journal]

The above demonstrates the quick, one-liner entry mechanism. If we invoke jrnl with no arguments, it opens a text buffer allowing us to enter more complex multi-line entries. jrnl automatically interprets as the body any content that follows an initial period of the title.

$ jrnl

Today's plan.
- Write
- Buy vegetables
- Skateboard
- Attend Haskell conference

To view only the current day’s entries, we invoke jrnl with today as the timestamp:

$ jrnl -on today
2018-08-18 14:59 Today's plan.
| Write, buy vegetables, skateboard.

2018-08-18 14:59 Afternoon break.
| Ordered coffee and explored new ideas.

2018-08-18 14:59 Evening reflection.
| Exercised and discovered a new forest trail.

2018-08-18 15:00 Today's plan.
| - Write
| - Buy vegetables
| - Skateboard
| - Attend Haskell conference

$ jrnl -on today --short
2018-08-18 14:59 Today's plan.
2018-08-18 14:59 Afternoon break.
2018-08-18 14:59 Evening reflection.
2018-08-18 15:00 Today's plan.

We can also edit existing entries by appending ‘–edit’ to any filter. To edit the last of today’s entries:

$ jrnl today -n 1 --edit

2018-08-18 15:00 Today's plan.
- Write
- Buy vegetables
- Skateboard
- Attend Haskell conference
- Ignore the above # New entry

[1 entry modified]

We tag entries as follows. The -and directive enables conjunctive filtering, whereas the default tag combination returns a disjunctive (or-based) selection.

$ jrnl @todo Build a simple idea tracking app.
[Entry added to default journal]

$ jrnl @todo @high Meditate.
[Entry added to default journal]

$ jrnl @todo @low Photosynthesis research.
[Entry added to default journal]

$ jrnl @todo --short
2018-08-18 15:06 @todo Build a simple idea tracking app.
2018-08-18 15:06 @todo @high Meditate.
2018-08-18 15:06 @todo @low Photosynthesis research.

$ jrnl -and @todo @high
2018-08-18 15:06 @todo @high Meditate.

We replace all occurrences of @high with @low and save the buffer.

$ jrnl -and @todo @high --edit

# Before
2018-08-18 15:06 @todo @high Meditate.

# After
2018-08-18 15:06 @todo @low Meditate.
[1 entry modified]

$ jrnl -and @todo @low --short
2018-08-18 15:06 @todo @low Meditate.
2018-08-18 15:06 @todo @low Photosynthesis research.

Using a combination of timestamps and tagging, we can perform a variety of bookkeeping tasks.

Literature notes (for Tale of Two Cities)

$ jrnl 
@book @ttc Book 1, Chapter 2.
Jarvis Lorry endures an all-night journey in a carriage with two other cautious and suspicious travelers. The author employs much complex imagery to paint the journey. The protagonist is burdened by necromancy motifs.

[Entry added to default journal]

$ jrnl @book --short
2018-08-18 15:12 @book @ttc Book 1, Chapter 2.

Link processing

We can store links and pipe the filtered list into the urlview utility to quickly open the chosen URL in a configured browser.

$ jrnl @link http://jrnl.sh/
[Entry added to default journal]

$ jrnl @link https://www.goodreads.com/
[Entry added to default journal]

$ jrnl @link --short
2018-08-18 15:14 @link http://jrnl.sh/
2018-08-18 15:14 @link https://www.goodreads.com/

$ jrnl @link | urlview
UrlView 0.9: (2 matches) Press Q or Ctrl-C to Quit!

->    1 http://jrnl.sh/
      2 https://www.goodreads.com/

urlview, in fact, scrapes all urls from your input stream, so you need not necessarily provide a @link tag. Note, we must pass some argument to jrnl to avoid opening a text buffer for a new entry.

$ jrnl --short | urlview 
UrlView 0.9: (2 matches) Press Q or Ctrl-C to Quit!

->    1 http://jrnl.sh/
      2 https://www.goodreads.com/

Idea gathering

$ jrnl @idea @prog Rewrite CLI inteface in @lisp
[Entry added to default journal]

$ jrnl @idea @prog Explore the merits of @prolog
[Entry added to default journal]

$ jrnl @idea @leisure Learn to skateboard
[Entry added to default journal]

$ jrnl @idea --short
2018-08-18 15:16 @idea @prog Rewrite CLI inteface in @lisp
2018-08-18 15:21 @idea @prog Explore the merits of @prolog
2018-08-18 15:21 @idea @leisure Learn to skateboard

Reporting of tags

We view a tally of all tag occurrences, or combine with additional filters.

$ jrnl --tags
@todo                : 3
@idea                : 3
@prog                : 2
@link                : 2
@ttc                 : 1
@prolog              : 1
@low                 : 1
@lisp                : 1
@leisure             : 1
@high                : 1
@book                : 1

$ jrnl --tags @idea
@idea                : 3
@prog                : 2
@prolog              : 1
@lisp                : 1
@leisure             : 1

Timestamps

Let’s modify the default timestamp of ‘now’ (15:35 in this case) to a custom time in the past or future as follows, appending it with a colon.

$ jrnl yesterday 22:50: @consulting @start
[Entry added to default journal]

$ jrnl 02:30: @consulting @end
[Entry added to default journal]

$ jrnl 15:00: @consulting @start
[Entry added to default journal]

$ jrnl now: @consulting @end
[Entry added to default journal]

$ jrnl @consulting
2018-08-17 22:50 @consulting @start
2018-08-18 02:30 @consulting @end
2018-08-18 15:00 @consulting @start
2018-08-18 15:35 @consulting @end

Time tracking system

The following not necessarily optimal piped combination of Bash script, reports the total time spent on the @consulting project. It requires the dateutils Unix package installed.

Let’s proceed step by step. The first part simply filters out the timestamps from the above output. dateutils.dgrep requires some filter present, so I indicated anything from the beginning of 2018. (It turns out dgrep doubles jrnl as a timestamp filtering mechanism, such that we could indicate a custom time range in either utility.)

$ jrnl @consulting --short |\
dateutils.dgrep '>2018-01-01' -o

2018-08-17 22:50
2018-08-18 02:30
2018-08-18 15:00
2018-08-18 15:35

The next part calculates the durations between each pair of start/end timestamps in hours and minutes.

$ jrnl @consulting --short |\
dateutils.dgrep '>2018-01-01' -o |\
while read -r start_time && read -r end_time ; do 
    dateutils.ddiff -f "%Hh%Mm" "$start_time" "$end_time" 
done

3h40m
0h35m

Finally, we sum the time durations. Because dateutils.dadd aims to add time intervals to an initial time stamp, we provide the start of the year, and note, with the day of 0, in order for the output to provide the correct total amount of days, hours, and minutes. (xargs passes the piped input stream as parameters to the indicated application.)

$ jrnl @consulting --short |\
dateutils.dgrep '>2018-01-01' -o |\
while read -r start_time && read -r end_time ; do 
    dateutils.ddiff -f "%Hh%Mm" "$start_time" "$end_time" 
done |\
xargs dateutils.dadd '2018-01-00 00:00' -f "%dd %Hh %Mm"

00d 04h 15m

Enhanced, multi-project time tracking and reporting

Let’s document the time worked for a series of projects. Rather than generate the @start and @end tags, let’s simply append the corresponding regular text to avoid superfluous reporting output. In reality, we only append ‘start’ and ‘end’ for descriptive reasons. Our time-reporting system relies strictly on pairs of non-overlapping entries for each task, and this is crucial.

$ jrnl 16:00: @time-track @little-company @project1 start
[Entry added to default journal]

$ jrnl 16:15: @time-track @little-company @project1 end
[Entry added to default journal]

$ jrnl 16:30: @time-track @big-company @projectA start
[Entry added to default journal]

$ jrnl 17:00: @time-track @big-company @projectA end
[Entry added to default journal]

$ jrnl 17:15: @time-track @little-company @project2 start
[Entry added to default journal]

$ jrnl 18:00: @time-track @little-company @project2 end
[Entry added to default journal]

$ jrnl 18:30: @time-track @big-company @projectB start
[Entry added to default journal]

$ jrnl 19:00: @time-track @big-company @projectB end
[Entry added to default journal]

$ jrnl @time-track --short
2018-08-18 16:00 @time-track @little-company @project1 start
2018-08-18 16:15 @time-track @little-company @project1 end
2018-08-18 16:30 @time-track @big-company @projectA start
2018-08-18 17:00 @time-track @big-company @projectA end
2018-08-18 17:15 @time-track @little-company @project2 start
2018-08-18 18:00 @time-track @little-company @project2 end
2018-08-18 18:30 @time-track @big-company @projectB start
2018-08-18 19:00 @time-track @big-company @projectB end

# Let's view all time-tracking related tags:
$ jrnl --tags @time-track
@time-track          : 8
@little-company      : 4
@big-company         : 4
@projectb            : 2
@projecta            : 2
@project2            : 2
@project1            : 2

Let’s iterate through all relevant tags and report the respective time spent, including the main @time-track, which corresponds to the grand total. This time we’ll abstract the procedure into a function for ease of invocation, adding a check for no time-tracking related tags found.

function report_time
{
    output=$(jrnl --tags @time-track)
    grep 'No tags found' <<< $output && return
    awk '{print $1}' <<< $output |\
    while read -r tag ; do
        printf '%20s: ' "$tag";
        jrnl -and @time-track $tag --short |\
        dateutils.dgrep '>2018-01-01' -o |\
        while read -r start_time && read -r end_time ; do 
            dateutils.ddiff -f "%Hh%Mm" "$start_time" "$end_time" 
        done |\
        xargs dateutils.dadd '2018-01-00 00:00' -f "%dd %Hh %Mm"
    done
}

$ report_time
         @time-track: 00d 02h 00m
     @little-company: 00d 01h 00m
        @big-company: 00d 01h 00m
           @projectb: 00d 00h 30m
           @projecta: 00d 00h 30m
           @project2: 00d 00h 45m
           @project1: 00d 00h 15m

It appears we worked for a grand total of two hours, one hour dedicated to each respective company, further divided by individual projects.

To further demonstrate the ease of invoking the above functionality, let’s generate the aliases to correspond to each task.

alias time-start-bc='jrnl @time-track @big-company start'
alias time-end-bc='jrnl @time-track @big-company end'
alias time-start-lc='jrnl @time-track @little-company start'
alias time-end-lc='jrnl @time-track @little-company end'

We designed our aliases to prefix the default, current timestamp. We could, however, edit the timestamps after the fact, which I did to generate some spread in the time intervals.

$ time-start-bc # No specific project indicated
[Entry added to default journal]

$ time-end-bc
[Entry added to default journal]

$ time-start-lc @projecta # Indicate a specific project
[Entry added to default journal]

$ time-end-lc @projecta
[Entry added to default journal]

The last four entries correspond to the above aliased calls, extracted via the ‘-n 4’ parameter.

$ jrnl @time-track -n 4 --short
2018-08-18 19:15 @time-track @big-company start
2018-08-18 19:30 @time-track @big-company end
2018-08-18 19:30 @time-track @little-company start @projecta
2018-08-18 20:00 @time-track @little-company end @projecta

$ report_time
         @time-track: 00d 02h 45m
     @little-company: 00d 01h 30m
        @big-company: 00d 01h 15m
           @projecta: 00d 01h 00m
           @projectb: 00d 00h 30m
           @project2: 00d 00h 45m
           @project1: 00d 00h 15m

To summarize, for the above reporting system to function as expected,

  1. Task entries should proceed in pairs, the start followed by an end timestamp.
  2. Tasks should not overlap. No multitasking!
  3. As long as a we maintain a proper tree structure between parent and child entities (ex: company->project, project->task, etc…) the hierarchical reporting makes sense. If, however, we decide to relax this requirement and allow child entities to belong to multiple parents (ex: an identically named project in multiple companies), we would need to expand report_time for custom queries.

[UPDATE]: See jrnl_helpers in my scripts repository for the more expanded routines jrnl_time_start, jrnl_time_end, and jrnl_time_report.

Questions, comments? Connect.