Contributed by: Scott Henwood, Director, Research Software
I joined CANARIE’s Research Software program more than three years ago after a long career in commercial/industrial software development. The experience so far has been eye-opening. At the implementation level, software development in an academic environment is pretty much the same as software development in a corporate environment. The environments that corporate and academic software developers work in, however, are very different.
As corporations focus on profitability, they continually look for ways to make software development extremely efficient. Despite the differences in environment and goals, my observation is that there are some commercial development techniques that can be easily adopted by academic software developers. There are no doubt lessons that corporate software developers can learn from the academic community as well, but I’ll leave those for someone whose career has gone in the opposite direction of mine to explore. This post focuses specifically on research software – software that is designed to support scientific research in an academic environment.
1. Use an Agile development approach
Using any structured software development approach is a good thing, and Agile is often an excellent choice for research software development. Along with being a popular buzz word, Agile is a way of managing software projects that focuses on frequent, incremental, fixed-period releases, ongoing testing throughout the project lifecycle, continuous user involvement and feedback, and the ability to adapt to changing requirements quickly with minimal impact on the project as a whole.
Agile is a collection of methodologies, rather than a single process, the most popular of which is called Scrum. In Scrum, software is developed in short, fixed length iterations called sprints. Each sprint includes review and prioritization of requirements, and software development with concurrent testing. The goal is to release, or at least demonstrate, something of value to the users at the end of each sprint.
What’s so great about Agile?
When done correctly, product and service companies can realize several benefits from an Agile approach. The reduced time-to-market that Agile allows can give companies a first-mover advantage over competitors. Software quality is often better because testing is ongoing throughout software development. Finally, the customer, whether an internal product manager or an external organization, may not understand all the requirements of the project up front. Agile allows requirements to be adjusted during the project in response to new knowledge or understanding.
Can an Agile approach benefit research software?
Absolutely! Research is by its very nature an activity with many unknowns. Even without these unknowns, scientific researchers are not typically expected to be experts in software requirements specification or user interface design, and so having touch points where the researchers (the users of the software) can evaluate the software and make course corrections is invaluable.
Should all research software development follow an Agile approach?
Absolutely not! If you’re working on a large collaboration involving multiple teams who are not following an Agile methodology, adopting Agile yourself may actually hinder the project, depending on how tightly coupled the work being done by the different teams is. More importantly, Agile is not a panacea. Agile projects in industry can and do fail, often for cultural reasons. If your team is not on board with Agile, has no members with Agile experience, or your management doesn’t believe in it, Agile may not be right for your project. The possibilities for disaster are endless, as Googling “agile project failures” can certainly demonstrate. For those not ready to fully commit to Agile, adopting the general principles of Agile such as early feedback, frequent releases and continuous testing may be beneficial.
2. Use dedicated testers
As with any type of software, bugs and defects can result in frustrated users who may choose to stop using the software. Complicating matters is the fact that collaborative research frequently results in users that are geographically distributed; yelling over a cubicle wall or going to the next office to discuss a bug may no longer be an option. In the worst case, subtle bugs in simulation or data processing components could ultimately lead to the recall of research results. No one wants that!
Many mature corporate software development departments include dedicated test groups. These groups are typically involved in integration, performance, usability, and system-level testing. Given that developers should certainly test their own code at the functional/feature level and have ultimate responsibility for the quality of the code they create, having test engineers finding bugs that should have been caught at the development level is very expensive.
So if the developers test their own code, why bother with dedicated testers? There are a few reasons:
- Testing is hard. While in theory, developers could do all of the testing, the qualities that make someone a good developer are not necessarily the same as the qualities that make someone a good tester.
- Developers have an internal view of the software that may affect how they test. No one does this on purpose, but if you wrote the code or peer-reviewed it, you may make assumptions about how it works.
- Testers are better able to represent the users, and some of the best testers are domain experts.
A concern I’ve often heard, both in the private and academic sectors, is that adding a tester to a project is not an option because it increases the project’s cost. I don’t buy this. Although there is certainly an administrative overhead for individual employees, the cost of having the developers do all the testing is about the same as using dedicated testers – the project just takes longer. Administrative overhead is more than recovered when bugs are found early.
3. Design for ease of use
Of course, even if your research software is bug-free, people won’t use it if it’s difficult to work with. Just as software developers don’t necessarily make good testers, software developers don’t necessarily make good user experience designers. Mature commercial software development organizations often include graphics artists and user experience experts as part of their software development process as well. This doesn’t have to represent a huge cost. A few days of a user experience professional’s time is all that may be required to evaluate your software for visual design, structure and organization, accessibility, mobile compatibility, error tolerance, and so on.
Proper documentation is also important in making your research software useable. But should you spend tens of thousands of dollars writing an epic user manual that is delivered in an attractive 2-volume set? No one would ever read it. Participants in CANARIE’s Research Software program have started to move towards short, narrated screen capture videos in place of written documentation. These video guides are quite effective and don’t require a huge investment to produce.
So who should create your user documentation? Stop me if you’ve heard this before, but good developers do not necessarily make good creators of user-facing documentation. Adding a documentation specialist to your team will not only result in quality documentation, but there may be another positive side effect as well. To understand the software, the documentation creator has to use it extensively. The result is that the documentation creator often becomes an informal member of both the test and usability teams.
How do we get adoption of these approaches by more research software developers?
This is by no means a comprehensive list. Dealing with long-term maintenance of research software, for example, is a topic that deserves its own article.
Funders like CANARIE have a crucial role in encouraging best practices in the development of software to support research. CANARIE’s Research Software Developers’ workshops provide a forum for the sharing of best practice for software engineering and Agile project management in a research environment. Initially available only to participants in our Research Software program, we anticipate opening up these workshops to software developers associated with research supported by other national funding institutions.
To encourage the use of multi-functional software development teams, we encourage applicants to include software testers, project managers, technical writers and user experience professionals in their project plans.
The goal is to get research software developers from across Canada working together to minimize duplication and maximize re-use so that they can concentrate on the unique, research-facing aspects of the software they create, so that researchers can do what they do best: work toward new scientific discoveries.