# The use of power tools

I’m currently reading the Apprenticeship patterns book, out of order (explained here: TODO).

I’m writing down the read chapters in a text file, to measure progress and also be able to backtrack if necessary. This also makes reading the book a “choose your own adventure”

After reading approximately half the chapters, the navigation wasn’t so easy using only the “see also” part. So I decided to break free from that constraint and start reading other chapters. But how do you know which chapters you have left to read? Knowing the whole set (`all`) and the read set (`read`) this is just the `all - read`.

Coming to a more practical way, how to know this is an automated (non-manual) way? There are `m` in `read` and n in `all`, where `m <= n`.

The simplest approach would be:

``````for each current in read:
remove current from all
``````

as read is is no particular order, that would yield some `O(m*n)`, worst case scenario (`m==n`), `O(n^2)`.

That is quite expensive, especially if you have to do it manually

If we use the same algorithm but sort the `read` set (no longer a set, as sets have no order). Also, converting it to a `O(1)` lookup table (e.g., a map), that would yield some `O(n log2 n + m)` that is already better. Taking into account that the sorting can be automated by the computer, it would yield a “manual O notation cost” of `O(1 + m)`. So, relative to the amount of chapters you have read.

But we can even do better in terms of this “manual big o notation”:

``````read <- sort read into a map ; cost O(1)
all <- sort all into a map ; cost O(1)

``````

And this is where the power tools come into place: if we use `comm` or `diff`, the computer will create the `unread` set for us, in “manual” cost of `O(1)`

## Example using vim, sort and diff

This is the `read` set (for the moment):

``````the long road
find mentors
the deep end
create feedback loops
the white belt
practice, practice, practice
concrete skills
use the source
reflect as you work
record what you learn
kindred spirits
study the classics
dig deeper
be the worst
rubbing elbows
``````

and this is the all set (copy-paste from the book):

``````INTRODUCTION 1 What Is Software Craftsmanship? 3 What Is Apprenticeship? 8 What Is an Apprenticeship Pattern? 9 Where Did the Patterns Come From? 9 Where Do We Go from Here? 10
2 EMPTYING THE CUP 11 Your First Language 13 The White Belt 18 Unleash Your Enthusiasm 22 Concrete Skills 24 Expose Your Ignorance 25 Confront Your Ignorance 28 The Deep End 30 Retreat into Competence 32 Wrapping Up 34
3 WALKING THE LONG ROAD 37 The Long Road 38 Craft over Art 40 Sustainable Motivations 43 Nurture Your Passion 45 Draw Your Own Map 47 Use Your Title 50 Stay in the Trenches 52 A Different Road 53 Wrapping Up 55
4 ACCURATE SELF-ASSESSMENT 57 Be the Worst 58 Find Mentors 61 Kindred Spirits 64 Rubbing Elbows 66 Sweep the Floor 68 Wrapping Up 71
5 PERPETUAL LEARNING 73 Expand Your Bandwidth 74 Practice, Practice, Practice 77 Breakable Toys 79 Use the Source 82 Reflect As You Work 85 Record What You Learn 87 Share What You Learn 89 Create Feedback Loops 91 Learn How You Fail 94 Wrapping Up 95
6 CONSTRUCT YOUR CURRICULUM 99 Reading List 100 Read Constantly 102 Study the Classics 104 Dig Deeper 105 Familiar Tools 109 Wrapping Up 111
``````

The first line is not interesting to our purposes, so delete it with:

go to first line (`1G`), then delete (`dd`)

After a quick inspection, there’s a number at the end of the desired line, so replace it with newline:

`%s/\d\+/\r/g` (a snippet below):

``````EMPTYING THE CUP
The White Belt
Concrete Skills
The Deep End
Retreat into Competence
Wrapping Up

``````

The title lines are all uppercase, so delete them with `:%g/^\s*\u\u/d`:

`````` Your First Language
The White Belt
Concrete Skills
The Deep End
Retreat into Competence
Wrapping Up

``````

explanation:

• `:` : command
• `%` : in all the file
• `g` : do (I remember it as ‘go’)
• `^` : starting at the beginning
• `\s*`: whitespace, as much a possible
• `\u\u`: exactly two uppercase letters. Can also be expressed as `\u\{2}`
• `d` : delete

but there are empty lines. Delete them with: `%g/^\$/d`:

`````` Your First Language
The White Belt
Concrete Skills
The Deep End
Retreat into Competence
``````

there is a space at the beginning of the line, delete it with `:%s/^ //`:

``````Your First Language
The White Belt
Concrete Skills
The Deep End
Retreat into Competence
Wrapping Up
``````

there are upper and lowercase letters. You could find a way of doing it with vi, but I prefer something that I can remember: save the document, then process it with `awk` and `sort`

``````cat all.txt | awk '{print tolower(\$0)'}| sort > all_sorted.txt
``````

(snippet):

``````a different road
be the worst
breakable toys
concrete skills
``````

If needed, sort and downcase the read.txt too.

``````cat read.txt | awk '{print tolower(\$0)'}| sort > read_sorted.txt
``````

I tried using `comm`, but it wasn’t so useful (because of whitespace).

Tried `diff`:

``````diff read_sorted.txt all_sorted.txt  -bBw > to_read.txt

0a1
1a3
> breakable toys
2a5,6
> craft over art
5a10
6a12
> familiar tools
8a15
``````

to know which are the remain chapters:

``````cat to_read.txt | grep ">"
``````

to count the number of remaining chapters:

``````cat to_read.txt | grep ">" | wc -l
``````

The lesson learned here is to master your tools. This would have cost me a few minutes if done manually as the list was short, too much time if long (>1000 lines).

On a funny note, it has taken me approximately 2 min to do this processing but ~45 minutes to write this blog post. So I don’t recommend you write as many posts as processings you do.

To quote Apprenticeship patterns, A different road:

If you walk away from software development, you will find that the habit of rigorous thinking and automating tasks involving large volumes of data will still be useful wherever you go

D Hoover, A Oshineye