Patterns for Software Engineering Research Papers

What do you think science is? There’s nothing magical about science. 
It is simply a systematic way for carefully and thoroughly
observing nature and using consistent logic to evaluate results.
— Steven Novella

Scientific research papers are purposeful, designed artifacts whose function is to communicate research results clearly and convincingly. To do this, a paper must identify the result, explain the method with which the result was obtained, and present evidence that the result is valid. The details differ from one field to another and from one research method to another, but good papers often use identifiable, widely-used patterns to present and defend the results.

Scientific and engineering fields can be characterized by the kinds of questions they find interesting, by the kinds of results they find useful in answering those questions, by the kinds of methods they use to obtaining the results, and by the kinds of evidence they accept to demonstrate the validity of the results.

In software engineering, our understanding of these patterns is largely tacit. We recognize several types of research, with different styles of reasoning. Without a clear, explicit understanding of the various paradigms, the rules that govern their reasoning, and the differences between them, we often struggle to explain our results.  Over the past decade or two we have come to understand certain types of research more systematically.  As a way to capture that understanding and provide examples for the field, we are developing a pattern language for research papers in software engineering and related areas.

We expect that this pattern language will make papers easier to write and to read.  It will

  • help authors understand how to organize a scientific argument and embed it in a paper, thereby making the scientific argument actually visible, not to mention complete
  • help authors choose the right type of argument and paper for their results
  • help readers by clarifying expectations about what they will find in a paper
  • help the software engineering community clarify its own standards for evaluating research, thereby helping reviewers
  • indeed, help researchers decide how to structure their research projects

To keep the problem tractable, we state some principles for development.  In particular, we will focus initially on the 6- to 10-page conference papers that report individual results. Real research projects are much richer, of course, but the conference-scale units are useful building blocks in the larger setting and a more manageable first target.

This site has three major sections:  The pattern language itself, the papers that exemplify the patterns, and resource material such as papers on specific research methods.

  • The Pattern Language – This section of the site is for patterns. For now it’s largely skeletal, with sketches and placeholders to lay out the shape of the language. The “About” page has an index to the patterns.
  • Examples – Patterns are based on specific examples found in the wild. This pattern languages describes research papers; this section of the site links to the papers that provide examples.
  • Supporting Resources – We aren’t the first to think about these problems. This section of the site collects guidance about specific types of research.

Who’s “we”?

This is a community effort with contributions from the following.  When it makes sense to attribute work to an individual, we use these initials.

  • [JY] Joe Yoder, The Refactory
  • [MS] Mary Shaw, Carnegie Mellon University
  • [RPG] Dick Gabriel, Dreamsongs
  • [JA] Jonathan Aldrich, Carnegie Mellon University

Leave a Reply