At a client, we’ve done today an internal training on “QA & how to test”. In it, the most skilled person with the QA role in the dev team has explained to us some techniques and concepts for testing

My notes

Verification vs validation: building the product right vs building the right product.

Principles

Extracted from ISTQB:

• testing shows presence of defects
• exhaustive testing is impossible
• early testing is better than later testing
• defect clustering: areas with bigger defect ratio or more critical, etc should be tested more thoroughly
• pesticide paradox
• testing is context-dependent
• absence of errors fallacy: the absence of defects does not imply perfect software. There are also problems with validation.

Techniques

People-based

• bug bashes: e.g., time-constrained
• subject-matter expert testing
• eat your own dogfood
• others

Activity-based

• regression
• scripted (manual)
• smoke
• exploratory
• installation
• load
• long sequence
• performance

Coverage-based

• menu tour: exploration based on menus (especially on websites)
• functional and system testing
• integration
• logic
• state-based

Requirements-based

• Equivalence partitioning: examples in the same set are considered equivalent
• Boundary based: there are interesting examples around and on the boundaries
• Decision tables: truth table
• State transition tables: state diagram

Risk-based

• make a prioritized list: probability and impact
• perform testing exploring each risk
• after a risk disappears, another opens. Adjust your test effort to stay focused on the current crop

Use case tests

• use case: a common case that represents one of your customer’s cases
• use busines language

Structure-based

• test coverage is different than code coverage
• test coverage is based on decision tables

Defining testing priorities

• customer and contractual requirements
• regulatory
• experience-based
• “Best representative”

A friend sent this Software Gardening Manifesto to me.

This is related to the idea of growing software vs building it, greatly exemplified by the Growing Object-Oriented Software Guided by Tests book (category here).

I’ve read this paper by Nagappan, Maximilien, Bhat and Williams on the cost, advantages, pitfalls of doing TDD in industrial teams (private companies who create software).

My notes

Abstract:

little empirical evidence supports or refutes the utility of this practice in an industrial context

• they seem to be talking about inside-out tdd as in chapter 2 (page 291), they talk about small units and then integrating them in a bigger context

Benefits

taken from chapter 2 (page 292)

• Better design
• Efficiency
• Test assets
• Reducing defect injection

Conclusions

TDD seems to be applicable in various domains

[TDD] can significantly reduce the defect density of developed software without significant productivity reduction […]

[…] an important aspect of TDD is the creation of tests assets.

Recommendations (this is all quotes):

• Start TDD from the beginning of projects
• For a team new to TDD, introduce automated build test integration towards the second third of the development phase
• […] add new tests every time a problem is found
• Get the test team involved and knowledgeable about the TDD approach
• Constantly running the unit tests […]
• Encourage fast unit test execution and efficient unit testing design

I’ve watched this talk by Erik Meijer

My notes

• Agile!: The Good, the Hype and the Ugly

• The hacker way

  • code always wins vs manager always wins
  • source: http://www.wired.com/2012/02/zuck-letter/

• we’re doing maths => we’re building a proof

• reinforcement learning

  • agent is the company

• related to “build, measure, learn” by Eric Ries (the lean startup)

Feedback systems

• Open (no) feedback system, such as timed sprinklers
• Closed feedback system
• Output only system

Why everything is software

• Examples of everything is software
• Software is eating the world
• Even hardware is modeled as software

Modeling as a Finite State Machine (FSM)

• There’s memory in the feedback loop. See the finite state machine
• A lesson from “World War Z”: “When nine people agree on something, it’s the tenth man’s responsibility to disagree no matter how improbable the idea”. “You don’t believe your own [ideas]”. There’s an article here and here, but they don’t cite any official source related to this. The second one cites World War Z again, so it might be confirmation bias.
• Test in production (e.g., chaos monkey from Netflix) rather than doing TDD
• No evidence of TDD as effective, he sees it as waste
• Move fast and break things

About layered organizations

• Layered architecture. Examples: OSI, Catholic Church, Army
• Analogy: Software development as a professional team structure (also cited from Netflix)
• Book: Creativity from Constraints: the psychology of breakthrough. Focus, constraint, creativity
• Presentation: Developing Expertise: Herding Racehorses, Racing Sheep
• Analogy: Developers are like junkies, we want to be high all the time
• Answering a new proposal: instead of “no, but” (black hat), “yes and” (blue hat). Reference to Bossypants by Tina Fey
• Your knives, your tools. “How can we make millions if we don’t value our tools?”

Conclusion

About software:

software will take over the world, and developers are the ones who make it happen.

About methodologies:

“Programming, motherfucker. Do you speak it?” (Source)

Update: I’ve grouped all small posts related to the self-study from May 2015 into a single post

Programmer passion considered harmful

I’ve read this article on why programmer passion could be negative. The title is similar to a paper by Dijkstra.

Software Craftsmanship: Are Bootcamps Enough?

I’ve read this article on software craftsmanship, development bootcamps and passion by Ken Auer

Improve your development environments with virtualization

I have seen this video: Improve your development environments with virtualization by Luke Sneeringer.

My notes:

• development should be exactly like production.
• except where it is too painful. there it should be different

Buffer’s take on values

I have found these values, written by Buffer, very interesting.

Public static void

I’ve watched this video: “Public, static void” by Rob Pike. The slides can be found here

My notes

• too much bureaucracy

• fighting the compiler

• dynamic vs static typing

• language too verbose

• do not mix the two of them

• a language niche:

  • comprehensible
  • statically typed
  • light on the page
  • fast to work in
  • scales well
  • doesn’t require tools but supports them well
  • good at networking and multiprocessing

  (minute 10:09)​

• Go was born to fill this niche

Nothing is something

This video has been extracted to a post of its own.

Software is eating the world

This video has been extracted to a post of its own.

The Cost of Test Driven Development

I’ve read this article, on the cost of doing TDD, which refers to this paper: Realizing quality improvement through test driven development: results and experiences of four industrial teams (below)

Realizing quality improvement through test driven development: results and experiences of four industrial teams

This paper has been extracted to a post of its own.

Learn You Some Lisp for Great Good

I’ve watched the video Learn You Some Lisp for Great Good, by Howard Abrams.

I guess this is a reference to the popular Learn You a Haskell for Great Good!.

He suggests many interesting resources for learning Lisp dialects. The live coding is in Clojure:

• The land of Lisp
• Structure and Interpretation of Computer Programs. See also SICPQuotes
• The Little Schemer
• An Introduction to Programming in Emacs Lisp
• 4clojure Questions
• Clojure Koans

On another topic, in the Bank accounts exercise from Exercism meetup, we discussed the book Programming Clojure (2nd edition) by Stuart Halloway

Professor Frisby’s Mostly Adequate Guide to Functional Programming

I’ve read this guide to functional programming. It is still work in progress (WIP) but still interesting. Written in javascript, so you can REPL in the browser.

Learning Code Good

I’ve watched this video, by Saron Yitbarek: Learn Code Good.

Why You Don’t Get Mock Objects

I’ve watched Why You Don’t Get Mock Objects, by Gregory Moeck.

In it, they point to the Growing Object-Oriented Software Guided by Tests book (category here), the paper Mock Roles, not Objects (also present in the GOOS book).

Technical Debt: A Repayment Plan

I’ve read this article by Jeremy Jarrell on technical debt how to repay it: schedule, taxes, default, etc

This has made me think of the “Deuda técnica, origen y consecuencias” talk (Technical debt: origin and consequences) (Spanish), that was delivered by Javier Gómez at the Software Craftsmanship 2014.

Revisiting XP: be a thoughtful programmer by exercising more collective ownership

I’ve read this article by Tiago Garcia on the collective code ownership, as stated by Xtreme Programming (XP), with some example and guidelines.

Where’s the Real Bottleneck in Scientific Computing?

I’ve read this article by Greg Wilson on the tools and procedures investigators use to produce software.

Testing inside the Sprint

I’ve read this article by Ron Jeffries.

Discusses the need for testing and why this hasn’t been communicated enough, as there still are people who have not understood it.

Also about why people equal something failing when using scrum with scrum failing.

DTO vs Domain Models

I’ve read this article by Carlos Blé

Homoiconicity isn’t the point

I’ve read this article by David Herman about homoiconicity, comparing Lisp dialects (have it) to others that don’t. The main point is at the end, where it says:

[…] homoiconicity isn’t the point, read is

So You Don’t Want to be a Programmer After All

I’ve read this article by Jeff Atwood. Cites the golden handcuffs (also known as golden shackles), reasons to be and not to be a programmer, and how to be an average and outstanding programmer

The Nature of Lisp

I’ve read this article / introduction to Lisp by Slava Akhmechet

[David Pollak’s] My Thoughts on Go

I’ve read these comments on Go Lang from David Pollak

The Well-Rounded Technologist

I’ve read this article by Michael Bernstein

Should I read papers?

I’ve read this article by Michael Bernstein

Lisp-1 vs Lisp-2

I’ve read this article by @Hornbeck about the lisp 1 and lisp 2.

Science Corps

I’ve watched this TEDx talk by Aza Raskin, featuring Data Banks and Science corps

Healthy Codebase and Preparatory Refactoring

I’ve read this article by Philippe Creux about preparatory refactoring

You Are Learning Haskell Right Now (Or Anything You Want Really)

I’ve read this article by Michael Bernstein

Lisp is Abstract Syntax

I’ve read this article by Michael Bernstein

Two Things Types Have Taught Me

I’ve read this article by Michael Bernstein

Don’t Bet Against Yourself

I’ve read this article by Michael Bernstein

Learning Math: A Blank Is A Blank With A Blank

I’ve read this article by Michael Bernstein

The False Economy Of Metaprogramming

I’ve read this article by Michael Bernstein

Management and Engineering, overlapping

I’ve read this Quora answer by Adam D’Angelo. The original title is: Does Adam D’Angelo feel comfortable with the transition from an engineer to a management position?

Evolution of The Twitter Stack

I’ve read these slides about changing the technology stack in Twitter: from Rails to JVM-based

Compilers as Assistants

I’ve read this article by Evan Czaplicki on seeing the compiler as an assistant to your work, instead of an adversary. Of how the compiler should give the most useful errors and warnings.

He also explains what features have been removed from the Elm language in 0.16.

Tags: compiler, assistant, adversary, enemy, elm, elm-compiler, evan-czaplicki

After reading the GOOS¹, the more and more I get this feeling of what vs how:

• Information hiding
• The total simpler than the sum of the parts
• Extracting a concept so you can think about this. Makes you find better names:
  • test name
  • variable name
  • method name
  • interface / protocol name

I also got this feeling when reading Clean Code. But with this other book the smell is much harder, stronger. This same concept is repeated all over the book. In our discussions about the book, this concept appears again and again. I’m thinking of adding a sub-title to the book:

Growing Object-Oriented Software, Guided by Tests
What and not How

Applying this in your team

Now, there might be a problem if your team has a split opinion on ‘what vs how’ (i.e., part of the team prefers what and another prefers how)

Reasons I’ve heard to defend how:

• The ones cited above
• Concision / shortness
• Code reusability
• Easier to test, extracting to collaborators²

Reasons I’ve heard to defend what:

• Knowing exactly what the code is doing: i.e., do not have to think what the name means and to what this corresponds