The Importance of Testing

The value of testing

"If it's worth building, it's worth testing. If it;s not worth testing, why are you wasting your time to workng on it?"

-- Scott Ambler, agiledate.org

Design principles for Apollo

• Use a High-Level Language: Employing a higher-level language simplified calculations and reduced errors.

• Divide into Jobs: Breaking down software into smaller tasks optimized memory usage, essential for systems with limited memory like the Apollo computer.

• Restart on Failure: Instantly restarting a failed job ensured continuous operation, a practice still followed in modern systems like Kubernetes.

• Checkpoint Good State: Saving and resuming from a known state minimized redundant calculations, a concept echoed in modern stateless container designs.

• Hardware Monitors the Software: Hardware oversight prevented system hang-ups, a principle akin to preemptive multitasking in contemporary systems.

• Send Telemetry: Continuous telemetry data provided real-time insights into system health, crucial for decision-making, a practice still vital in modern software monitoring.

You can only test what you know

Testing is essential for ensuring the functionality and reliability of software systems. The principles derived from historical examples like the Apollo Guidance System highlight the importance of testing in ensuring robustness and resilience to failure. Remember, you can only test what you know, emphasizing the necessity of comprehensive testing strategies.

Why Developers Don’t Test

1. I already know it works: Developers may skip testing because they believe their code is functioning correctly. However, overlooking tests can lead to future issues when others work on the code, as they may inadvertently introduce bugs without proper tests to verify functionality.

2. I don't write broken code!: Some developers may argue that they don't need to test because they write code without errors. However, the software ecosystem is dynamic, with constant updates and patches. Without tests, it's challenging to ensure compatibility with new libraries or patches, potentially leading to vulnerabilities or system failures.

3. I have no time: Time constraints are often cited as a reason for skipping testing. However, investing time in writing tests upfront can save considerable debugging time in the future. Tests provide a safety net for refactoring code and adding new features, boosting productivity in the long run.

Why do we need to test?

Testing is crucial because:

• Prevents future code breaks and compatibility issues: Tests help ensure that changes to code, libraries, or dependencies do not introduce unexpected bugs or system failures, maintaining stability and reliability.

• Reduces overall development time: While writing tests may seem time-consuming initially, they ultimately streamline the development process by catching errors early and providing confidence for future modifications or additions.

• Ensures that your code behaves as expected when others use it: In an open-source or collaborative environment, code may be reused in various contexts. Testing ensures that the code behaves predictably regardless of its application, fostering trust among developers and users alike.

Testing Levels and Release Cycle

Software Testing Levels:

1. Unit tests:

   • Test individual units or components of a software system.
   • Verify that each unit performs as designed.
   • Focus on both "happy" paths (expected behavior) and "sad" paths (error handling).
   • Typically executed in a development environment or during continuous integration (CI).

2. Integration Tests:

   • Combine individual units and test them as a group.
   • Expose flaws in the interaction between integrated units.
   • Evaluate how modules behave together with various inputs.
   • Often performed in a development environment or during Behavior Driven Development (BDD).

3. System Tests:

   • Test the entire software process as a complete, integrated system.
   • Evaluate the system's compliance with specific requirements.
   • Verify that the entire system works together as expected.
   • Usually conducted in a staging or pre-production environment.

4. Acceptance Tests:

   • Test the system for acceptability based on business requirements.
   • Assess whether the system is acceptable for delivery to end users.
   • Often performed by end users in an environment similar to or the same as the production environment.

Traditional Release Cycle:

• Development: Developers perform unit testing and store code in a source code management (SCM) system like Git.

• Build: Code artifacts are compiled, and further unit testing may be conducted in a build environment.

• Package Repo: Built artifacts such as Java jar files, Python wheels, or Docker images are stored in a package repository.

• Test, Stage, and Prod Phases:

  • Test: Build artifacts are deployed into testing environments for integration, performance, and compliance testing.
  • Stage: Testing environments become more like production, allowing for system testing and acceptance testing.
  • Prod: The final production environment where the fully tested and accepted software is deployed for end-user access.

Key Takeaways:

• The software testing process includes four levels: Unit, Integration, System, and Acceptance.

• Testing occurs at different levels throughout the phases of the traditional release cycle, starting from development and progressing through testing, staging, and production environments.