Developing Computer Based Training
an essay by
Delano P Wegener, Ph.D.

Components of a CBT System:
CBT training can be delivered in individual, isolated, and freestanding modules or CBT training can be delivered as a comprehensive collection of interrelated instructional modules. In the latter case the collection is referred to as a CBT system and it typically contains components beyond the purely instructional modules.

The remarks in this document refer to CBT systems with ten or more modules. Typically such a CBT system may contain any number of the following components:

  • Instructional modules
  • Formative tests
  • Evaluative tests
  • Record management system
  • Student management system
  • Content management system

System Design:
The first step in the design of a CBT system is to decide which of the above components are to be included. The needs and desires of the client are the determining factors in this first step. However, it is wise to include the prospective developer in this early stage. The developerís role may simply be to provide clarification, but he might also suggest alternatives.

The degree of integration of the chosen components is the second decision in the design of a CBT system. A CBT system may be delivered as a single large computer program or a large collection of small computer programs. In practice, the configuration of most CBT systems fall somewhere between these two extremes. The actual configuration depends on many factors including, but not limited to, the following:

  • Intended uses of the system
  • Deployment methods
  • Computer literacy of users
  • Computer literacy of administrators

As in the first step, the needs and desires of the client are the determining factors in this step, but capabilities and limitations of the developer and the development tools are also factors. The computer system used for deployment of the finished system can also affect the decisions in this step of the design. For this reason, the prospective developer should very definitely be involved in this phase of the design of the system. The developer is able to relate the clientís needs to development effort. The developer is also able to ask questions to determine all the clientís needs as they will impact development. The developer is able to offer alternative development strategies which satisfy the clientís needs.

The third step in design of a CBT system is the design of each of the individual components of the system. This step actually consists of very many tasks which are individually detailed later in this document. It is of paramount importance that the developer be involved in each of these tasks.

The final step in system design is to construct a development schedule. To construct such a schedule, the following considerations are important:

  • Required resources
  • Degree to which the client will participate
  • Resources available from the client
  • Resources available from the developer
  • Events which are anticipated to interfere with development
  • Marketing considerations

Although the needs and desires of the client are still determining factors, the developer is able to provide realistic estimates of required time and for that reason his input may dominate during construction of the schedule.

Component Design:
Design consideration for each of the above CBT components will be presented individually.

It should be clear from the descriptions below that both the client and the developer must be involved in the design of each component.

I use Toolbook (a computer program from Asymetrix Corp.) to develop Computer Based Training. Toolbook uses a book metaphor to describe the programs and screen images in the CBT. I will use that same metaphor in the following descriptions.

Instructional modules: An instructional module is a book with which the student interacts. It contains the teaching materials such as text, pictures, video, etc.

Design of the instructional modules begins by selecting a CBT style chosen from the following list. The list is arranged from the simplest to the most complex. The order also reflects teaching effectiveness, effective use of the medium, and cost of production. See the article CBT Styles for more detail.

Style 1: A simple "page turning" copy of text.

Style 2: A simple "page turning" copy of text with glossary.

Style 3: A simple "page turning" copy of text with glossary and other hyperlinks.

Style 4: Multimedia CBT.

Style 5: Multimedia CBT implementation of the Mastery Based Learning Model.

After a CBT style has been chosen, it is necessary to design individual pages of a typical instructional module. This involves screen layout, form of student interaction, program reaction to student action, branching opportunities, degree of student control, degree of program control, and a variety of other details.

The final step in the design of instructional modules is the detailed design of the various instructional materials for each learning objective. I am assuming that content design has preceded CBT design and has proceeded to the point that all learning objectives have been identified.

Design of instructional materials must involve design of each individual item (text, graphics, sound, video, etc.) as well as insuring that for each learning objective, the collection of instructional items form a coherent collection which will facilitate the studentís learning.

Formative tests: A formative test is used to provide the student with a measure of his understanding of the subject being presented. In order to provide the student with frequent measures of his understanding, formative tests should occur often and should cover small amounts of material.

Format of the presentation to the student, type of feedback, and timing of feedback are important design considerations. Any combination of the following options are possible. Some of the combinations are not educationally sound and some induce significant development costs.

Presentation options:

  • One question per page
  • All questions on one page with questions grouped according to type (T/F, fillinblank, multiple choice, etc.)
  • All questions on one page with questions not grouped according to type (T/F, fillinblank, multiple choice, etc.)
  • Separate pages for each type of question

Feedback options:

  • Correct or incorrect
  • Provide reward
  • Provide reward with additional challenge
  • Provide correct answer
  • Suggested remedial study
  • Provide remedial instruction
  • Provide score
  • Reasonable combinations of the above

Timing options:

  • After each question
  • After some predefined group of questions
  • After the test has been completed

Design of formative tests also requires a decision about how an individual test is constructed. For any given formative test there are at least four choices for test construction:

  • The same preconstructed test is presented each time
  • The test is selected (randomly or by rotation) from a collection of preconstructed tests
  • The test is constructed by randomly (this may involve weighting) choosing questions from a collection of questions.
  • The test is constructed by assembling computer generated questions

All details discussed in the article Test Construction should be considered when designing and constructing questions for formative tests.

Evaluative tests: Evaluative tests are primarily for reporting purposes. They are designed to evaluate and record the studentís mastery of a subject. Evaluative tests occur less frequently than formative tests and evaluate the studentís mastery of large blocks of the subject matter.

Design of evaluative tests in a CBT system is identical to design of summative tests as described above.

Record management system: A record management system consists of two distinct subsystems.

Information storage and retrieval subsystem: This system is usually a database. For small CBT systems or CBT systems with a small number of students, the database management system (DBMS) may well be custom programmed for the project. For larger CBT systems or CBT systems with a large number of students, the database management system should probably be a stable commercial DBMS such as Access.

Design of the information and retrieval subsystem is no different than the design of any other database. It is important to stipulate what data is to be preserved and to determine all relations between the data items. The developer must also design the data representation inside the database.

Reporting subsystem: This subsystem consists of a report generator which draws its data from the information storage and retrieval subsystem.

Design of the reporting subsystem involves decisions related to each of the following:

  • What reports are required?
  • What is the format of each report?
  • Who generates the reports?
  • How are the reports presented (screen, printed, file, etc.)?
  • To whom are reports distributed?
  • How are reports distributed?

Student management system: The student management system controls the following.

  • Registration, identification, passwords, requirements, curricula, prerequisites, etc.
  • Permits access to the CBT system as a visitor, a reviewer, a student, an administrator, and possibly some other options
  • Who has access to the CBT system
  • What parts of the CBT system are available to a user
  • The order in which modules are available to a user

Design of the student management system requires a complete characterization of all expected types of user, reasons for their use, administrative restrictions on their use, as well as educationally sound decisions related to the proper order of events in the learning process.

After all options have been identified, screen layouts must be designed and the computer programs themselves must finally be designed.

Content management system: The content management system is that part of the total CBT system which controls:

  • Adding new modules
  • Adding new content to existing modules
  • Deleting modules
  • Deleting content from existing modules
  • Editing existing content
  • Adding questions
  • Editing questions
  • Editing answer keys
  • Access to the content management system
  • Creating a record of edits performed
  • Deployment of modifications

The content management system may be as simple as referring all content management items to the developer. A better content management system permits the client to maintain the content without intervention from the developer.

Design of a client operated content management system involves selection of items to be included, screen layout, and decisions related to each item in the above list.

Computer program development:
Developing the computer programs required to operate a CBT system is the responsibility of the developer, but regular reaction and input will be required from the client.

To develop the computer programs the developer should use a main programming tool (Toolbook from Asymetrix Corp.) specifically designed to develop Computer Based Training. In addition the developer may use more general tools such as Access (for DBMS), Visual C++, Visual Basic, or Delphi (for DLL development) as well as special media tools to develop pictures, sound, and video.

Much of this work is independent of course content and in some CBT designs all programming is independent of course content.

Content development:
Developing course content consists of constructing and/or assembling teaching materials for each learning objective as prescribed by the design of instructional materials. Fundamental qualifications of persons performing content development include subject matter expertise and a solid foundation in the principles of teaching and learning. Beyond these two fundamental qualifications, the development team must possess expertise in the various media (writing, graphics, audio, video, animation, etc.) being used as well as the computer tools used to create the media.

Development cost:
Cost of development is most frequently quoted in terms of development time required to develop one hour of CBT time, which itself is impossible to quantify. Estimates I have read in the past three years range from as low as 100 hours of development time per hour of CBT time to a high of 1000 hours of development time per hour of CBT time. In most of these estimates, the cost of actually producing the media was not included in the estimate.

How does one calculate the cost of CBT development? The flippant answer is to try to calculate the cost after the project is complete. For a more acceptable answer, one should adhere to the following general guidelines during the design process and develop a cost estimate at the end of the design process.

Cost control Guidelines:
These guidelines are generalities and it should be expected that each individual situation will impose its own unique modifications on these guidelines.

  • Design the CBT system to meet its intended purpose
  • Adding an option will increase cost
  • Selecting options which the developer has previously implemented will reduce the cost increase
  • Changes late in the development process are extremely expensive
  • Experienced developers may use a higher base rate but total cost is generally less than the total cost resulting from an inexperienced developer with a lower base rate
  • Every dollar spent on design saves three to five dollars in development costs
  • Computer program development is a creative process and not all time (costs) can be accurately predicted
  • Permit at least 10% of the total development cost to be devoted to testing the product
  • Maintenance costs of a poorly designed or poorly implemented system can exceed the cost of initial development

For larger systems it is generally cost effective to initially develop a comprehensive management system (content management, student management) so that no programming costs are involved in the development of the actual instructional modules. In a system consisting of 100 modules (hours of CBT time) or more this approach should reduce total cost per module to the range of 50 hours per hour of CBT time.