software

Software Engineering Procedures

Prologue 

Often the people managing software projects have never written and maintained a large software system. Not acknowledging the time it takes to develop software is like not acknowledging the time it takes to do a clinical trial. It comes from a lack of understanding of the process. “The highest form of ignorance is when you reject something you don’t know anything about.”, Wayne Dyer.

Quotes 

I have written some of the quotes below. The ones I have written are evolving / improving (and started in June of 2017). Please e-mail me if you know of a similar quote by someone else.

Iterative Development 

“A Minimum Viable Product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.” leanstack.
“Minimum Viable Product: Build a slice across, instead of one layer at a time.” interactive-design.
“You’re selling the vision and delivering the minimum feature set to visionaries, not everyone.” minimum feature set.
Start a project with a minimum feature set and then iterate. “You should use iterative development only on projects that you want to succeed.”, UML Distilled.

Sooner is Better 

“The sooner you find a defect, the cheaper it is to fix.”, Extreme Programming Explained.
“Fail fast.”, Martin Fowler.
“Release early. Release often. And listen to your customers.”, The Cathedral and the Bazaar.

Clear Code 

“Any fool can write code that a computer can understand. Good programmers write code that humans can understand.”, Refactoring: Improving the Design of Existing Code.
“If it’s not clear that a condition is assumed true, add a comment about it in the source code. If the assumption might be false, include a check that halts the program; i.e., fail fast. If a user error can make the assumption false, include a message that helps users fix the problem.”, Bradley Bell; see assertion (software development).
“The present letter is a very long one, simply because I had no leisure to make it shorter.” Blaise Pascal. “Inside every large program is a small program struggling to get out.” Tony Hoare
“If a long section of code is not clear, identify a shorter sub-section you understand. Make the sub-section a separate documented routine. Include asserts for conditions you are uncertain of. Test that the new version functions like the old. Repeat this process until the new version is clear.” Bradley Bell.

Refactoring 

“Each significant piece of functionality in a program should be implemented in just one place in the source code. Where similar functions are carried out by distinct pieces of code, it is generally beneficial to combine them into one by abstracting out the varying parts.” Benjamin C. Pierce, abstraction principle.
“When you find you have to add a feature to a program, and the program’s code is not structured in a convenient way to add the feature, first refactor the program to make it easy to add the feature, then add the feature.”, see link above to Refactoring: Improving the Design of Existing Code.
“Find the smallest useful change you can make, make it, then test it.”, refactor in very small steps.

Verification 

“Never write a single line of code unless you have a failing automated test.” Kent Beck.
“A program’s documentation should include specifications so any expected results can be verified; i.e., reproduced by another program. Each documentation section should include a simple example that automatically checks for correctness.”, Bradley Bell; see software documentation, verification, see link above to design by contract.
“You are doing enough testing if the following is true: You rarely get bugs that escape into production. and You are rarely hesitant to change some code for fear it will cause production bugs.”, see link above to Martin Fowler.
When “You find a bug. You write an (automated) test that replicates the bug. You run the test and you check that the test is failing as expected. You fix the bug. You run the test and you check that the test is passing.” test driven bug fixing.
“Tests that demonstrate previous bugs focus on aspects of the code that are confusing to the people modifying a program and thereby prevent future bugs. They should be easy to understand, isolate specific problems, and run quickly during regression testing. Often, simplifying a test for a bug report makes it clear that it is not a bug but rather a mistake on the part of the user.” Bradley Bell; see regression testing.
“If you have a test that demonstrates a bug, add it to your automated testing system, with a check that expects it to fail, and a comment that identifies the bug. Once the bug is fixed, change the test so it is expected to pass.” Bradley Bell

Optimization 

“premature optimization is the root of all evil. … A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified.” When to optimize.

Unit Tests 

A system example and/or test is for the API of the entire system. A unit example and/or test is for the API of a piece of the system. Many programmers do not create an API for pieces of the system because this only helps developers (not users) of the system.

Test Driven Development 

A simple development process can be diagrammed as follows:

|--------------|      |----------------|      |----------------|
| Automate new |      |  Code design   |      |  All automated |
|     test     | ---> |      and       | ---> |      tests     |
|  that fails  |      | implementation |      |    now pass    |
|--------------|      |----------------|      |----------------|

see test driven development

Iterative and Incremental Development 

The following process builds a foundation that makes development and use easier in the long run:

|--------------|      |----------------|      |----------------|
|              |      | Documentation  |      |  Code design   |
| Requirements | ---> |      and       | ---> |      and       |
|              |      | specification  |      | implementation |
|--------------|      |----------------|      |----------------|
      /|\                                             |
       |                                             \|/
|--------------|      |----------------|      |----------------|
|              |      |  Distribution  |      |   Automated    |
|  Validation  | <--- |      and       | <--- | unit & system  |
|              |      |  maintenance   |      |  verification  |
|--------------|      |_---------------|      |----------------|

see iterative and incremental development

Documentation 

This is helpful if, people other than the developers will be using the program, or the program is large enough so that the developers can not remember it all. See software technical documentation. The technical documentation should be embedded in the software and kept up to date with it’s changes. Trying to do this later will take a lot more time and not be as accurate.

Specifications 

This is required for the program’s results to be reproducible by another program; e.g., for the program to be verified. It also enables coding and implementation by people who do not completely understand the program. See design by contract. This should be part of the technical documentation and kept up to date.

Automated Testing 

This is helpful when the program needs to be verified in multiple environments, or after multiple changes to the source code. See test automation. A new automated test, that is also an example, should be written when a new feature is added to the program. A new automated test should also be written to demonstrate each bug and later verify that the bug stays fixed. Writing these tests later will take a lot more time.

Distribution 

Using a distribution tool that automates configuration for different systems is helpful if the program will run in multiple environments, or if the environment will have multiple updates and changes. For example, see cmake, and autoconf.

Maintenance 

This is helpful if there are users that provide feed back and bug reports. See software maintenance.

Validation 

This is a check, by users, that the software does what they intended. See software validation.

Requirements 

This is a set of changes that are improvements in what the program does. It is a balance between what the users want and the difficulty and time required to get to the next validation. It is best to represent requirements as changes to the documentation, so users read and understand the changes and developers can make and verify the changes. See requirements analysis.