Chapter 6: Recommendations for creating tests

What will you find here?

On this page, you’ll find a summary of the most important things teachers should keep in mind when creating electronic tests. These are general principles applicable to tests on any platform and, to some extent, to regular paper-based tests. The recommendations were developed for us by Dr. Libor Juhaňák from the (Institute of Educational Sciences) of the Faculty of Arts. We believe the advice will prove useful not only in the period of remote teaching, but also after returning to in-person teaching.

Recommendations for creating test tasks

The creation of tests and test tasks is a relatively complex activity. Nevertheless, there are some basic recommendations that can help you when creating test tasks:

  • Set out in advance what knowledge or skills you want to test in the upcoming test. Only then set about creating the actual test.
  • The test tasks you create should evenly cover all the knowledge or skills you want to test. At the same time, you should avoid setting problems whose solution would require knowledge or skills that could not be acquired on the course.
  • The assignments must be clear, understandable, and completely unambiguous. Therefore, avoid ambiguous wording and expressions.
  • Be very careful with the use of negation in an assignment (typically in the form of “Which of the choices given is not correct?”). When using it, it’s recommended to highlight the negative wording in bold. Avoid using double negatives altogether.
  • Ideally, the functionality of test tasks should be verified before they are used in live testing. If this is not possible in your case, ask a colleague, for example, to go through the tasks. This enables you partially, at least, to check the clarity and comprehensibility of the tasks.

Closed versus open tasks

Test tasks can be divided into closed and open ones. Whereas in closed tasks the student has to choose the correct answer from several options offered, in open tasks the student has to create the answer themselves. Closed tasks most often include dichotomous tasks (yes/no, true/false), multiple choice tasks, or matching tasks. Open tasks, meanwhile, are differentiated into short-answer tasks (a word or short phrase) and long-answer tasks (sentences and paragraphs).

The use of primarily closed or open tasks depends mainly on your goals and possibilities. Notwithstanding, it’s good to be aware of the basic strengths and weaknesses of both types of tasks.

Closed tasks

Advantage:
Possibility to evaluate automatically.

Disadvantage:
Higher risk that the student will copy, look up, or guess the answers randomly.

Recommendations for creating closed tasks
  • Always make sure that the problem has only one correct answer.
  • Remember that the chance of guessing the correct answer corresponds to the number of options offered. Thus, in the case of a dichotomous task, students have a 50% chance of guessing the correct answer. For multiple choice tasks, we therefore recommend creating 4 to 5 options, as the chances of guessing are then significantly lower (25% and 20%, respectively). For matching exercises it’s likewise recommended to create more options to match up than there are sub-items within the task. Otherwise, students may use a process of elimination to increase the chance of guessing the correct answers.
  • At the same time, care must be taken to ensure that all options are sufficiently “attractive” to students, i.e., that the option cannot be immediately ruled out as the wrong answer.
  • All options should also be of approximately the same length, style, and form of expression. Be especially careful that the correct answer is not always slightly longer than the other options (e.g., for the sake of precision).
  • Ensure that the grammatical form of the task does not suggest the correct answer (e.g. the gender or article for foreign language problems).
  • It’s also worth avoiding tasks that include “all of the above” or “none of the above” as one of the options.

Open tasks

Advantage:
More comprehensive assessment of knowledge and skills.

 

Disadvantage:
More difficult to ensure an objective evaluation.

Recommendations for creating open tasks
  • For short-answer tasks you should always think about all possible ways of writing the correct answer and set the correct answers accordingly (e.g., “organisation” vs. “organization”, upper and lower case, numerical vs. verbal expression). If there are too many ways of formulating an answer, it’s better to rethink the assignment and come up with something new.
  • If you use a short-answer task in the form of a gap-fill, make sure that the meaning of the text is sufficiently clear even with the answers omitted.
  • For long-answer tasks, make sure that the wording of the assignment makes it clear what answer you’re asking students to give and in what form. Specify what kind of answer is required (e.g., a description, explanation, justification, or comparison), in what form, in what scope, etc.
  • To ensure the objective evaluation of long-answer tasks, clearly specify in advance how the problem will be assessed (what criteria will be assessed, on what scale, how partially correct answers will be assessed, etc.). It’s also advisable to prepare a model correct answer against which you can then compare students’ answers.

You are running an old browser version. We recommend updating your browser to its latest version.

More info