Nine Principles for Designing a Study Program

Stefan Cronholm
stecr@ida.liu.se
Dept. of Computer and Information Science
Linköping University
Linköping, 581 83  Linköping

Abstract
This paper outlines nine key principles to consider when designing a study program. Design principles are essential when revising education programs in the field of information systems due to the rapid changes in business technique. These principles are generated from an empirical study concerning the development of a modified study program. The principles identified are: Vision; Traceability; Progression and Integration; Manning the development project and anchoring decisions; Pedagogy; Mandatory vs. free choice of courses; Implementation; Evaluation and Marketing. These are compared to a list of criteria generated by the Swedish National Agency for Higher Education and the outcome of this comparison is that several of the principles can also be used for the evaluation of study programs. The research has been carried out as action research. This means that there has been an intervention in the process of the designing the study program.  The target groups of this paper are study boards, faculty members, teachers and students that are active in different student associations.
Keywords: study program, design principles, action research


1.  NTRODUCTION
A study program can be seen as a package consisting of several courses. The idea of a program is that it should embrace several courses, the length of which can vary from one term to several years. Some can be more flexible with few or no mandatory courses, providing students with the possibility to decide the content of the program depending on their own preferences. Other study programs are pre-designed consisting mostly of mandatory courses. The main idea of a pre-designed study program is that the courses given should be coherent. This paper will focus on programs that are pre-designed.
The research question reads: ‘What principles are important to consider when designing a study program?’. One reason for asking this question is that there is a rapid change in business techniques that creates an ongoing demand to revise education programs concerning information systems.  Another reason is that there is a lack of documented recommendations concerning advices or guidelines for designing study programs.
However, there are a lot of recommendations concerning the design concept (cf. Preece et al, 1994; Norman, 1988; Shneiderman, 1998, Nielsen, 1993). Our interpretation of the design concept is that an international term that mainly refers to industrial and graphical design. The design concept is frequently used when constructing technical artifacts such as computers, mobile phones and interfaces. We can conclude that the definition of the design concept is too general. It is mostly applied to the design of artifacts and is therefore not easily applied to the design of a study program. Advice or guidelines for designing technical artifacts can however stimulate a way of thinking.
Related to the design concept is the concept of evaluation. Development is often iterative and alternates between design – evaluation – (re)design (Cronholm & Goldkuhl, 2002). There are a lot of recommendations suggesting how to perform evaluations (cf. Walsham, 1993, Remenyi, 1999, Patton, 1990; Cronholm & Goldkuhl, 2003) but, once again, the recommendations exist on a too general level and are therefore hard to apply to study programs.
A model created by the Swedish National Agency for Higher Education (2002) has been a source for inspiration. In order to evaluate higher education the National Agency uses a number of evaluation criteria such as: goals and profile; economy; number of teachers; number of students; sex equality; scientific grounds; teaching methods and international exchange. Since the National Agency uses these criteria to assess study programs they must be important to consider when designing a study program.
The aim of this paper is to suggest principles to consider when designing undergraduate study programs. In Section 2, the development project is briefly described. Section 3 describes the research method and in Section 4 the findings are presented. In Section 5 the principles are compared to a set of existing criteria used by the Swedish National Agency for Higher Education and finally in Section 6 concluding discussions are presented.
2.  BRIEFLY ABOUT THE DEVELOPMENT PROJECT
The assignment for the development project was to develop a new and modern study program in the field of Information Systems. There was dissatisfaction with an existing older program, which originated in the 1970s. The aim of the project was to suggest a new attractive program with a good balance of theory and practice, modern technology and reflected pedagogy.
The old program lasted four years, comprised 30 courses and involved approximately 200 students and 10 teachers. The main subject was Information Systems and other subjects included Economics and Sociology. Seventy-five percent of the courses were mandatory and could not be substituted by other courses; with the remaining twenty-five percent subject to free choice. The development project was led by a project leader (the Study Director) with two teachers and three students. Industrial representatives were also consulted. The projected lasted for approximately one year.
3.  RESEARCH METHOD
The research has been carried out as action research (cf Checkland, 1991; Avison et al, 2001; Baskerville, 2001). Action researchers are researchers that intervene in a change process. The reason for choosing action research was that we had both an action interest and a research interest. This means that we have both acted as researchers and as study program developers (see Figure 1).

Figure 1 Three interlinked practices (Cronholm & Goldkuhl, 2003)
The model in Figure 1 describes our view of action research and how this study has been carried out. There are three interlinked practices: The theoretical research practice (= studying the development project), the change practice/empirical research practice (= development project of a new study program) and the regular practice (= the running program business).  In this paper, the research part that has been of primary interest since the aim was to develop new knowledge. The research part took place when we reflected on the work performed in the development project. The development project has served as source for collecting empirical data. The development project has therefore been important as a source of knowledge, but is in itself of secondary interest. The running program business has acted as base for the development project. It is important to note that the three practices existed simultaneously with a continuous flow of data between the practices. The flow consisted of assignments, bases and results. A common criticism against participatory action research is that it lacks scientific rigor (Kemmis & McTaggart, 2001). In order to reduce this criticism we have been careful about the two roles researcher and study program developer.
During the development project notes were taken about the subjects discussed. These notes have served as a data source. In order to generate and motivate principles to consider when designing a study program two main questions have been asked: “what is the discussion about” and “why is this discussion important?”. In that way principles have been induced. In order to theoretically validate the principles, comparisons have been made to the criteria used by the Swedish National Agency for Higher Education.
4.  FINDINGS: NINE PRINCIPLES
The analysis of the development project has generated nine principles to consider when designing a study program. The principles are:
* Vision 
* Traceability
* Progression and Integration
* Manning the development project and anchoring decisions
* Pedagogy
* Mandatory vs. free choice of courses
* Implementation
* Evaluation
* Marketing
Principle 1: Vision
The first principle to consider is vision. If a mission is what we do or why we exist today where we are, then a vision is the view that a person takes of what the organization can and should become if the mission is followed (Davis, 1987). Conger (1989) defines vision as "...a mental image or a dream of a highly desirable future state for the organization". A vision can be understood as a collective belief in what organizations can become.
In the development project there were a lot of discussions of the vision. It was clear that the vision governed formulations of the aim and goal whilst at the same time it set the program boundaries. Formulating a vision means that you create a profile of the program. The vision was based on the questions “what properties should the “good” information system analyst acquire?” and “what competence and skills are expected from examined students?” When formulating the vision it is important to identify different clients that will benefit from the program.
The main clients are the students, but there are also other clients to consider such as employers and the faculty itself. Our experience is that there can be conflicting opinions between clients about what properties the “good” information system analyst should acquire. A request from the faculty, in particular from the research education, was to formulate the vision in more scientific terms and that the program should be based on scientific grounds. The industrial employers requested a more practical program content. Our experience is that it is important to find a balance between different requests in order to satisfy different needs. The reason for this conclusion is that the study program developed should educate students both for industry and for further research education.
Principle 2: Traceability
Once the vision is formulated, it is possible to decide what courses the program should contain. Traceability should exist between the vision and courses given. Every course in the program should contribute to fulfilling the vision and thereby motivate its existence. In the development project, the old program was analyzed through the new vision. The outcome of this exercise was that some courses:
* Contribute to the fulfilment of the vision
* Contribute with a modification request
* Were rejected since they did not contribute to the fulfilment of the vision
The aim of this exercise was to decide upon course candidates for the new program. Courses that contributed to fulfillment were accepted for the new program. To investigate if traceability exists between the courses and the vision legitimates the program content. To investigate if traceability exists means that the election of courses is reflected and problematized.
Principle 3: Progression and Integration
The idea of a study program is that it should consist of a coherent collection of courses. But what do we mean by the term coherent? The analysis of the work in the development project identified two concepts: progression and integration. Progression means that following courses on a higher level should be more advanced. Integration means that courses given on the same level should, if possible, be related to each other.
These two concepts can be related and illustrated as two dimensions in a matrix (see Table 1). Each cell in the matrix can correspond to a course. The concepts used as column headers are derived from the vision. Deriving concepts from the vision ensures that they are relevant. Every row header represents one academic year. The horizontal dimension shows the integration possibilities between courses. There is also an order between concepts that follows a systems lifecycle. This implies that the horizontal dimension represents a time order meaning that knowledge about “concept 1” is acquired before knowledge of “concept 2”. The integration possibilities consist of the knowledge acquired from an earlier course contributing to the following course. For example, in the development project “concept 1” corresponds to a course in change work and “concept 2” corresponds to a course in design work. A result from change work is often a list of measures to take. These measures can act as a requirement specification for design work in the following course.
Table 1.

Concept 1
Concept 2
Concept n
Year 1 



Year 2



Year 3



Year n



The same integration possibilities exist for the following academic years, but a progression is evident in that the knowledge acquired is more advanced.
Principle 4: Manning the development project and anchoring decisions
The fourth principle to consider is Manning and Anchoring decisions. The result from a development project is normally implies change of doing business. In order to reach a high acceptance for this change, the question of how to man a project and how to communicate the changes to the clients who are affected by the change, is important (Mumford, 1983). The clients that are affected by the changes in this case are students, teachers, employers and the faculty. Therefore, our experience is that representatives from these groups should be included in the development project. The analysis of the development project showed that a higher acceptance was reached when design issues were discussed in a participatory design spirit. This result is line with the findings of Heron & Reason (2001) and Carlshamre (1994). The students’ voices as well as the teachers’ voices were important bases for decisions. Decisions having a high support and a high acceptance among the clients of the program were easier to implement in the new study program.
Principle 5: Pedagogy
The analysis of the development project’s discussion about pedagogy generated the fifth principle. This principle declares that the choice of pedagogy should be reflective and conscious. In the old study program there was a large variety of pedagogy used, for example problem based learning, project-oriented work and other ways. Often the pedagogy used was a result of an individual teacher’s choice. Often the pedagogy chosen was one that the teacher felt most comfortable with, and not a strategic choice decided at the program level. In order to achieve more reflected decisions about pedagogy, the choice should reside at the program level and in dialogue with the teachers. The reason for this claim is that individual teachers often have detailed knowledge about his/her course and have a limited view of the whole program. At the program level there is an overview of the program and therefore relations to adjacent courses become more visible.
A lot of the pedagogical discussion in the development project concerned project-oriented work. A description of project-oriented work in relation to learning and examination can be found in Melin & Cronholm (2004).
Principle 6: Mandatory vs. free choice of courses
The sixth principle to consider is the degree to which mandatory courses are provided. There are both advantages and disadvantages with a study program that solely contains mandatory courses. The advantages are that it is easier to design in terms of progression and integration and that the students can easily follow the main thread of the program. The disadvantages are that several students would like to have flexibility and the possibility to select courses at free will. They like to add a specific education profile to their degree. For example, in addition to Information Systems, some students could be interested in education and consequently want to enroll in courses such as Pedagogy. Alternatively, others may wish to combine Management and Economics with Information Systems. Another finding from the analysis is that flexibility offers freedom and increases the students’ possibility to choose courses. The advantage is that this freedom can be used as a marketing argument when recruiting new students.
The analysis also revealed an economic aspect. It is more expensive to offer flexibility instead of standardized mandatory courses. One goal of the university is to reach a high student volume for every course. To conclude, there are three sub-categories to consider for the problem of mandatory vs. free choice of courses: ability to follow the main thread; the degree of flexibility; and economy.
Principle 7: Implementation
The seventh principle generated concerns the implementation of a study program. The analysis identified three strategies of implementation. These are:
* Successive implementation 
* All at once
* A combination of the two alternatives above
The first strategy “successive implementation” means that after one year the first year of the new study program is implemented and after two years, the second year is implemented and so on. This means that both the old and the new programs are running parallel. One can characterize this strategy as “Safe and Secure”. It is far from certain that all students are satisfied with the new program content. There are always students that claim they would like to take the old courses. Some of the students even claim that they have the right to take the old courses since they enrolled onto the program on the condition that these courses would be offered. In the case of successive implementation this problem will never occur. You can in this respect say that this strategy is “safe”. The disadvantage with this strategy is that several existing students would like to have access to the new program since it possibly contains more attractive courses. Another disadvantage with this strategy is that running two programs parallel increases the program administration.
The second strategy “all at once” means that all four years of the program are implemented at the same time. Students that have joined the old program are transferred to the new program. This strategy could be risky in terms of the progression and integration (see above) of courses. It is not certain that “old” students would have the pre-knowledge required for some of the new courses. An advantage with this strategy is that a modern, fresh program can be offered quickly without having to wait several years until the whole program is implemented.
The third strategy “a combination of the two alternatives above” means to implement parts of the new program that are not risky; the chosen parts do not therefore counteract program progression and integration. The disadvantage with this strategy is that you still have two programs running in parallel but you can partly offer a new and modern program
In the development project the second strategy “all at once” was chosen. The reason for this choice was that the existing students wanted to have access to all the new courses. Several teachers also supported the “all at once” strategy since they felt the change exciting and motivating. The industrial representatives also preferred “all at once” since they wanted the students to have an education that is as modern as possible.
Principle 8: Evaluation
Normally a program develops through the continuous use of evaluation in the form of course evaluations and continuous course modification. These activities can be characterized as smaller efforts and they are more frequent than analyzing and developing a program from a comprehensive picture. According to the literature there are two main approaches: formative and summative evaluation. Both approaches contain different measurements or criteria. According to Walsham (1993) and Scriven (1967) formative evaluation aims to provide systematic feedback to the designers and implementers during the design process; while summative evaluation is concerned with identifying and assessing the worth of program outcomes in the light of initially specified success criteria after the implementation of the change of the program is completed. The criteria or measurements used are often derived from one specific perspective or theory.
The analysis of the question as to how the new program should be evaluated resulted in a selection of both formative and summative evaluation. The reason for choosing formative evaluation is that the new program was developed in a prototyping-like way. Prototyping means to shift between design and evaluation until the product (the program) is satisfactory (Vonk, 1990). The reason for choosing summative evaluation is that the project group also wanted to obtain experiences after the new program is fully implemented. It is important to note that evaluation activities were planned in advance and that they were not carried out as an ad-hoc activity.
Principle 9: Marketing
A major reason for developing a new program is that there have been a reduced number of students in the old program. The analysis of discussions held in the developed project identified that marketing the new program was a central issue. Inspired by Kotler (1994) two main questions were raised “how should we inform our clients about the new program” and “what key concept can we use in order to attract the clients?” The process of answering these questions can be thought of as brainstorming. Several suggestions were made. The conclusion of the first question is that there is a need to be more visible in secondary schools. For some years the need for marketing was not so urgent since the number of student applications was enormously high, with maximum numbers of students easily achieved. The current situation of fewer students implies that new actions need to be taken in comparison with earlier circumstances.
The analysis of the discussions concerning key concepts for marketing is summarized as:
* Young students and practitioners must understand the concepts
* The key concepts must be popular and attractive
* The key concepts should be formulated in acceptable academic way
One potential problem with the goals is that they can be conflicting. There can be a conflict between the goals “young students must understand the concepts” and “the key concepts should be formulated in an acceptable academic way”. Organizations have often created a specific business language (Cronholm & Goldkuhl, 2002). The academic language often consists of abstract concepts that are not always easy to understand for younger people or practitioners. Formulating key concepts in an understandable way and in an academically acceptable way can be a real challenge. There can also be a conflict between “the key concepts must be popular and attractive” and the other two goals. There are a lot of buzzwords around that are easy to absorb and use in advertising in order to look popular. The opinion of the project group is that the key concepts used in marketing should correspond to the content of the program.
5.  COMPARISON WITH ESTABLISHED CRITERIA
The nine principles are generated from an empirical project. In order to validate the principles a comparison has been made with the criteria used by the Swedish National Agency for Higher Education (2002). Since this reference is written in Swedish a brief description is given for the criteria. The criteria are: goals and profile (analysis and description of local goals and local profile); economy (analysis of economical conditions to offer education in a certain subject, the distribution of laboratory work and non laboratory work); number of teachers (including title, research hours, teaching hours, number of doctoral students that is supervised) number of students (for each academic year, number of bachelors and masters, pre-knowledge, sex distribution); sex equality (describe and analyze how the equality between men and women is promoted); scientific grounds (describe and analyze the scientific grounds for the education and how critical thinking is stimulated); teaching methods (describe and analyze teaching methods and examination forms) and international exchange (describe and analyze cooperation and contacts with the other universities, both national and international).

Figure 2. Model for comparison
A simple model has been used for the comparison (see Figure 2). The outcome of the comparison can be viewed as three sets. Set “A” represents the principles generated in this paper and set “C” represents the criteria used by the Swedish National Agency for Higher Education. Set “B” represents principles and criteria that are similar. Set “A” minus Set “C” represents principles that not are similar to the criteria. This set is equal to the main contribution of this study. Set “C” minus Set “A” represents criteria that not are identified in this study.
The similarities identified (Set “B”) are:
* The criteria “Goal and profile” is similar to the principle “Vision” 
* The criteria “Scientific Grounds” is not an explicit principle, but this criteria is part of the principle “Vision” (see Principle 1: Vision)
* The criteria “Teaching Methods” corresponds to the principle “Pedagogy”
Explicit correspondences to the other six principles (traceability, progression, integration, manning the development project and anchoring decisions, mandatory vs. free choice of courses and implementation) are not found among the criteria used by the National Agency (Set “A”-Set “B”). A possible reason is that the principles are generated for a design situation whereas the criteria used are for an evaluation situation of a running program.
There are also criteria that do not explicitly correspond to the principles (Set “C”-“A”). These criteria are economy, number of students, number of teachers, sex equality and international exchange. The first criteria, ‘economy’ have been an underlying condition in the development project. One major reason for forming a project group was the reduced number of students and a weak program economy. Questions of sex equality and international exchange are important criteria that could not be identified from the analysis of the discussions in the project group.
6.  CONCLUDING DISCUSSION
The main contribution of this study is to provide nine design principles to consider when forming a study program. The principles are generated from an empirical study and compared against criteria used by the Swedish National Agency for Higher Education. When comparing the principles to the criteria it is clear that the characters of the principles are qualitative and several of the criteria are more quantitative.

Figure 3 Conceptual relations
The principles vision, traceability, progression and integration are conceptually tightly coupled and they have a direct relation to the program content. Their relations are illustrated in Figure 3. Figure 3 shows that there should be traceability between the courses and the program vision. This implies that the fulfillment of the vision should be supported by the courses given. Between courses there should also be both progression and integration (see Section 4).
In Section 4, the principles are presented as a list without any specific order. Using a process perspective raises the question of whether there is a specific time order between the principles. In order to present a possible time order the principles have been analyzed in terms of causality or in terms of conditions and consequences (cf. Strauss & Corbin, 1998). The result of this analysis should not be thought of as a sequential waterfall model; where development work requires a lot of iterations. The first principle to consider is how to man the development project and anchoring decisions. The second principle to consider is the vision. The vision acts as a condition and governs the following work. In the next step the principles progression, integration, traceability, pedagogy and mandatory vs. free choice of courses can be considered simultaneously. The concluding steps are implementation, evaluation (if formative evaluation is used this activity occurs earlier in the process) and marketing.
One way of generalizing the findings is to enlarge the scope and claim that the principles can be used in other situations than in the design situation of a study program. Can they be valid for other situations? This question can be problematized and the scope can theoretically be enlarged in at least two directions. One direction is whether the principles can be used in other activities than design related to a study program; the other direction is whether they can be used for designing something other than a study program.
As mentioned in Section 1 the concepts of design and evaluation are closely related. So far, we cannot see anything that contradicts the principles being relevant for an evaluation situation as well. This argument is based on the fact that some design principles were similar to the evaluation criteria used by the National Agency. An example that shows that the concepts design and evaluation are closely related is illustrated by the principle “manning the development project and anchoring decisions”. If decisions, in a design situation, are not anchored among teachers and students, an evaluation can explore negative effects such as weak acceptance of the program. Other principles such as “traceability”, “progression”, “integration” and “mandatory vs. free choice of courses” can easily be used in an evaluation situation. The aim of this paper is to suggest design principles but our findings also show that the principles can be added to and thereby enriching the evaluation criteria used by the National Agency. The question of the design principles can be used for designing something other than a study program needs to be further explored.
However, the principles are generated from one development project and must therefore be tested further. We suggest two areas for further research. One direction is to enlarge the study to embrace more empirical projects that can further validate the principles. Another direction is to evaluate a program that is based on the principles. The question ‘what consequences will arise?’ would be of particular interest. The principles can so far be thought of as a hypothesis; a hypothesis that is grounded in empirical data.
7.  ACKNOWLEDGEMENTS
The Faculty of Arts and Sciences, Linköping University financed the project.
8.  REFERENCES
Avison D, Baskerville R & Myers M (2001) “Controlling action research projects”, in Information Technology and People, Vol 14, No 1, MCB University Press.
Baskerville R (2001) “Conducting Action Research: High Risk and High Reward in Theory and Practice”, in Qualitative Research in IS: Issues and Trends (Trauth E M ed.), Idea Group Publishing, London.
Carlshamre P (1994), A Collaborative Approach to Usability Engineering, Dept of Computer and Information Science, Linköping University
Checkland P (1991) “From Framework through Experience to Learning: the essential nature of Action Research”, in Information Systems Research: Contemporary Approaches and Emergent Traditions (Nissen H-E ed.), Elsevier, Amsterdam.
Conger, Jay A. (1989), The Charismatic Leader: Behind the Mystique of Exceptional Leadership. Wiley
Cronholm S & Goldkuhl G (2002). “Document-Driven Systems Development - an Approach Involving Novice Users”. In proceedings of the 7th Annual Conference of United Kingdom Academy for Information Systems (UKAIS), (Howell B, Orange G, eds), 10-12 April 2002. Leeds Metropolitan University, UK
Cronholm S & Goldkuhl G (2002). “Actable Information Systems - Quality Ideals Put Into Practice”. In proceedings of the Eleventh Conference On Information Systems (ISD 2002). 12-14 September, Riga, Latvia 
Cronholm S & Goldkuhl G (2003). “Strategies for Information Systems Evaluation - Six Generic Types”. Electronic Journal of Information Systems Evaluation, Vol 6, Issue 2.
Cronholm S, Goldkuhl G (2004). “Conceptualising Participatory Action Research – Three Different Practices”. Electronic Journal of Business Research Methods, Vol 2, Issue 2.
Davis Stanley M., (1987) Future Perfect, Addison-Wesley, Reading, MA
Heron & Reason (2001) “The Practice of Co-Operative Inquiry: Research ‘with’ rather than ‘on’ people, in Handbook of Action Research (Reason P & Bradbury H eds), Sage Publications, London.
Kemmis S, McTaggart R (2001) “Participatory action research”, in Handbook of Qualitative Research, second edition, (Denzin N K, Lincoln Y S eds), Sage, Thousands Oakes.
Kotler P (1994), Marketing Management – Analysis, Planning, Implementation, and Control, Prentice Hall International, UK
Melin U & Cronholm S (2004). “Project Oriented Student Work – Learning and Examination”. In proceedings of the 9th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education (ITiCSE 2004). 28-30 June 2004. Leeds, UK
Mumford E (1983), Designing Participatively, Manchester Business School
Nielsen J (1993), Usability Engineering, Academic Press, SanDiego, CA
Norman D A (1988), The Psychology of Everyday Things, Basic Books, New York
Patton M Q (1990), Qualitative Evaluation and Research Methods, second edition, Sage Publications
Preece J, Rogers Y, Sharp H, Benyon D, Holland S & Carey T (1994), Human-Computer Interaction, Addison & Wesley
Remenyi D & Sherwood-Smith M (1999) “Maximise Information Systems Value by Continuous Participative Evaluation”, Logistics Information Management, Vol 12 No 1/2 pp 14-31
Scriven, (1967), The Methodology of Evaluation, Rand McNally, Chicago
Shneiderman B. (1998). Designing the User Interface: Strategies for Effective Human-Computer-Interaction. Addison-Wesley
Strauss A, Corbin K (1998), Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, Sage Publications, Thousand Oaks, CA
Swedish National Agency (2002) in Swedish: Anvisningar och underlag för självvärdering i informatik och datavetenskap m.m.
Vonk R (1990), Prototyping - the effective use of CASE Technology, Prentice Hall
Walsham G (1993), Interpreting Information Systems in Organisations, Wiley & Sons