Software Testing and the Scientific Method

An overview of using the scientific method in UX design and debugging

BY JAIZE MILLER

Software design operates on computational logic, and the average person is now surrounded by software, both large and small scale, powering anything from watches, to light bulbs, to wallets and restaurant menus. Given its binary nature, there is a quick pipeline of products whose use is lost in translation to a potential user. How often have you encountered a product or application where you were frustrated by an error in functionality that was clearly caused by poor engineering?

To reach success, products need to address specific needs which can vary greatly from individual to individual. Proper design process establishes the goals of the development cycle before beginning any further development.

To apply these tools, it is extremely helpful to implement the scientific method to ensure strategies developed are accurately meeting client needs.

Observe

The first step in the scientific method is observing. It should also be the first step in software testing.

As a designer, it is crucial to use your senses at this stage to detect potential design flaws and weaknesses. It is helpful to have a notepad or note file to write down any and all responses drawn from observation, which can be revisited further.

Metaphorically walk through the product to view it in its totality. What visuals, auditory materials, and tactile input are present? Is there a general sense of missing a detail somewhere? Where in the product do you feel this? Are there elements you would like to add?

Take note of any areas that are lagging performance-wise or yielding unexpected results. As usual, within the code itself, flag any syntax errors and comment out any unnecessary sections.

Question

To reach success, products need to address specific needs which can vary greatly from individual to individual. The target audience needs to be identified in order to ensure easy accessibility for users. Observing user needs early on improves product functionality upon deployment.

The materials necessary to interact with the product need to be assessed for practicality, specifically for that of the end user. Consider disabled perspectives when interacting with the product — what materials or stimuli may not be suitable for all? What would be an alternative?

Revisit product observations to determine the underlying cause for failure. Can the defect be replicated? How large is the scope of the breach?

To resolve issues within the code itself, consider the implications of the chosen approach to structure. Did the algorithm function as expected? What other algorithm can be used to achieve the desired result?

Hypothesize

Based off questioning, begin to infer possible solutions for design. Sort the observed qualities into testable concepts, predicting desired outcomes. Address how design changes can better accommodate functionality for all target users. Revisit questions on accessibility, providing options on alternative input and feedback. The end goal at this stage is to summarize theories into a single statement that can be tested for accuracy.

For example:

If the typed input is replaced with an option for audio input as speech to text, then the software will be more accessible for those with visual impairment.

Experiment

Replicate the scenario in a controlled environment, adjusting the dependent variable x as dictated by the hypothesis. Observe and measure any changes or differences, both expected and unexpected. Iterating through tests multiple times ensures more accurate results.

Analyze

The overarching question here is what quantifies a success?

To answer that, pull quantifiable information from data. Trace patterns in data and compare statistics and measurements with known quantities. Note what was predictable in data alongside what was not expected.

Report

Report the findings by sharing results with others and requesting feedback. Compare feedback with results to compile best plan of action. Utilizing communication and team based work improves outcomes of product success especially at the testing stage.

Conclusion

Using the scientific method to test software and smart products results in stronger design, and by association, better marketability and success potential. Try this workflow for yourself in your next debugging cycle and leave a comment below on the results!