Criteria for LanguageQuests - Assessment Tool

• Name assessor:
• E-mail address:
• Title LanguageQuest:
• Website LanguageQuest:

Section A: Critical attributes of a WebQuest

this part of the tool covers formal/organisational matters related to the product features that affect its usability for learners, help to actually do the quest and evaluate it.

• 0
  1
  2
  3
• 1.

  Introduction

  To what extent does the introduction help the student to get an idea of what the task is and its context.

  [i]
  • 0 - Absent, or contains no relevant information
  • 1 - Vague sketch of situation or general description in general terms in some short paragraphs
  • 2 - Fairly realistic description of context with some true-to-life details
  • 3 - The most important contextual elements are described in a realistic, functional, lively and stimulating way
• 2.

  Task

  The description of the task and the specifications of the final product. (NB: these specifications should be in accordance with the criteria mentioned in ‘Evaluation’, part 5 of this section)

  [i]
  • 0 - Absent, or contains no relevant information
  • 1 - Task description in very general terms. No or only vague specifications for the final product
  • 2 - Reasonably concrete description of task and final product in terms of form, size and/or content.
  • 3 - Clear and concrete description of task. Detailed specifications of the final product in terms of content, form and size/length.
• 3.

  Process

  What is the quality of the working procedures to help students do the task? Dependent on the type of task and the target group this process can be more or less structured.

  [i]
  • 0 - Absent, or has adverse effect
  • 1 - Too little: Process described in very general terms. No discriminate steps mentioned. Little or no indication of roles to be assigned, if relevant.
    Too much: The level of detail in the procedure suggested is more likely to cause confusion than offer support.
  • 2 - Reasonably concrete description of steps to be planned for the realisation of the task and and for possible roles to be assigned. Some global suggestions on how to arrive at results.
  • 3 - Clear, concrete and (where functional) detailed description of things to be done and therefore need to to be planned and of the various possible roles and related tasks involved. Clear instructions on how to realise results.
• 4.

  Resources

  Are there sources of information available and to what extent are they functional for the execution of the task. These resources could take the form of documents attached to the WebQuest, WWW links, information to be retrieved by mail or from databases, and, of course, also non-digital content.

  [i]
  • 0 - Absent, or consists of mere search engine(s), or does not contain information relevant for the execution of the task.
  • 1 - Mainly general resources, loosely related to the task at hand, offering insufficient information. Or they offer much information but only a small part is relevant to do the task. To find useful information for task realisation students need to scan texts and search extensively for additional resources.
  • 2 - Resources are fairly specific and provide some useful and relevant information for task execution. To realise a good product hardly any searching for additional information is needed.
  • 3 - Selection of resources provides for an efficient execution of the task. Lots of useful information is available and can be easily located. The available resources are sufficient for task completion.
• 5.

  Evaluation

  The criteria for the final product on the basis of which students will be evaluated. To what extent do they provide clarity for students to plan ahead and realise their work, including their share in the realisation of a collective outcome/result, if applicable.

  [i]
  • 0 - Absent, or contains no relevant information
  • 1 - Criteria are worded in general terms (e.g. “Your presentation and language use will be evaluated”) or with a limited level of specification (e.g. “You will be evaluated on correct language use, functionality of the presentation” etc.)
  • 2 - There is a list of specific aspects that will be evaluated. For each aspect a general indication of the relevant criteria is provided. Examples are: “Adequate content”, “relatively few grammatical mistakes”, “only few spelling mistakes”, “fair contribution to collaborative process and final result’
  • 3 - There is an elaborate “rubric” (as e.g. in this tool) with concrete assessment criteria for a range of aspects such as minimal size, required content elements, accuracy and fluency levels, quality of presentation materials, contribution to teamwork, etc. providing a scoring range per item
• 6.

  Conclusion

  After task completion, to what extent are students invited to reflect on the process, the learning and the usefulness of the knowledge gained?

  [i]
  • 0 - Absent or does not trigger reflection
  • 1 - Learners are invited to report what they themselves think to have learnt, but only in general terms (e.g. ´What do you think you have learnt from this task?´)
  • 2 - Reflection is stimulated by asking students to list specific things that they think to have learned from the task and what they attribute this to. (E.g: Learned: the following words:…especially by making sentences with them and memorising these.) Some structure is offered (e.g. “vocabulary, grammar, idioms, content and cultural knowledge”)
  • 3 - Reflection is stimulated by asking students to list various things they think they have learned from the task. Furthermore they are asked to indicate what has contributed most to their learning, how they account for this and what they intend to do to prevent forgetting what they have learned.
• 7.

  Teacher's page

  How extensive and relevant is the information the teacher is offered on aspects such as the assumed language levels and skills of the target group, preconditions (technology, facilities etc), a profile of the students and the role of the teacher.

  [i]
  • 0 - Absent or contains no relevant information
  • 1 - Information is limited and contains basis and general information with respect to the target group, estimated student language levels and curriculum objectives.
  • 2 - Contains reasonably detailed information with respect to the target group and the required minimum language levels, the curriculum objectives addressed and the expected learning results. Has some information on prerequisites & necessary preparation and hints for the execution.
  • 3 - As in 2 but also includes hints and solutions for possible pitfalls, and suggestions for presentation and classroom management. Has suggestions for adaptations to special target groups (e.g. providing more or less structure) and reports the author’s personal experiences or those of others.

Section B: Criteria concerning the instructional effectiveness of the task

These criteria are based on current insights in the domain of second language acquisition

• 1
  2
  3
  4
• 8.

  Language input (quantitative)

  What is the quantity and variety of the language materials offered?

  [i]
  • 1 - Sources contain little text, hardly any variation in text type, register (e.g formal vs, conversational language, youth language)and modality (spoken, written, sung)
  • 2 - Sources provide a fair amount of text (some web pages or ‘traditional A4 printed pages’), but the text materials are not very varied (mainly short, informative texts or mainly songs or mainly biographical data etc)
  • 3 - Sources provide quite a lot of text (several web pages or ‘traditional A4 printed pages’) with some variety in terms of text types and modality.
  • 4 - Sources provide a substantial quantity of text, both spoken and written in a great variety of text types and lay outs processing of which is functional for task at hand.
• 9.

  Language input (qualitative)

  How suitable and attractive are the task and the language materials involved for the learner target group.

  [i]
  • 1 - Resources consist of search engines only or score low on all of the following aspects:
    - Attractive (“stimulating presentation”)
    - Authentic
    - Within the sphere of interest of the target group
    - Of current interest
  • 2 - Resources score ‘high’ on 1 or ‘average’ on 2 of the following aspects:
    - Attractive (“stimulating presentation”)
    - Authentic
    - Within the sphere of interest of the target group
    - Of current interest
  • 3 - Resources score ‘high’ on 2 or ‘high’ on 1 and ‘average’ on at least 2. Or ‘average’ on 4 of the following aspects:
    - Attractive (“stimulating presentation”)
    - Authentic
    - Within the sphere of interest of the target group
    - Of current interest
  • 4 - Resources score ‘high’ on 3 or ‘high’ on 2 and ‘average’ on the remaining aspects:
    - Attractive (“stimulating presentation”)
    - Authentic
    - Within the sphere of interest of the target group
    - Of current interest
• 10.

  Language input (appropriate level of difficulty, i.e. just above the competence level of the learner (or in SLA terms: Interlanguage + 1)

  Does the difficulty level of the input materials match the target group and/or has the author provided support to adapt the level to the target group

  [i]
  • 1 - Resources are not in the target language or much too difficult or so easy that the learner will hardly learn anything new. No functional support is available that could help the learner to understand enough of the text which in fact is too difficult to be able to do the task.
  • 2 - Resources are too difficult for the target group: the available scaffolding tools such as dictionaries or suggestions to use specific strategies, additional information about the task , graphics, etc. provide only little support to understand so much of the text that the task can be done properly.
  • 3 - Resources are difficult for the target group: the available scaffolding tools such as dictionaries or suggestions to use specific strategies, additional information about the task, provide enough support so that texts can be understood well enough for proper task completion
  • 4 - The difficulty level is well tuned to and challenging for the target group by the choice of the texts and the provision of scaffolding tools aid learners to understand and process the input for task completion.
• 11.

  Process for meaning

  To what extent does the task require understanding of input?

  [i]
  • 1 - Task can be done superficially, with only marginal understanding of the sources materials or even no or hardly any processing of input using cut and paste techniques.
  • 2 - Task can be done properly without some real understanding of the contents in the resources. However, a part of the final product can be realised by copying bits of content.
  • 3 - A fairly thorough understanding of the information in the resources is needed. If only to make an adequate selection of content for the product. Task directs towards paraphrasing and rewording of the information but some instances of ‘borrowing from the orginal’ are feasible.
  • 4 - The task is designed so that learners need to understand and reflect on much of the material provided. Information has to be understood and processed (e.g. selected, evaluated, compared, changed, summarized in order to complete the task. The contribution of cut and paste techniques is very limited for this.
• 12.

  Process with a focus on form

  To what extent does the task stimulate (the further development of) the learner’s language awareness (notice form aspects in the materials and possible regularities in them (such as word order, use of tenses, word endings, pre- and suffixes, plural forms etc.)

  [i]
  • 1 - In no way does the task promote the learner’s attention for aspects of form in the materials offered, such as word order, use of tenses, plural forms etc.
  • 2 - The task does promote the learner’s attention for aspects of form in the resources offered but is not functional: the task can be done without actual further exploration. It is a grammatical exercise in its own right, more or less by way of intermezzo.
  • 3 - The task requires awareness of the formal aspects of the language involved and although this is also somehow related to the content aspects of the task the primary focus is on the grammatical/syntactical phenomenon in itself.
  • 4 - The task requires awareness of the formal aspects of the language involved but only to the extent that this awareness contributes to a better performance in task completion. The focus is on the impact of form on meaning. But learners are stimulated to pay close attention to language AND form.
• 13.

  Language production 1 (presentation)

  To what extent does the task and/or product presentation trigger the use of the target language (L2).

  [i]
  • 1 - Learners are allowed to use their mother tongue (L1) in all phases of doing the task.
  • 2 - Task requires some use of L2 for production and/or presentation but materials are highly structured and predefined also allowing for memorisation of content. Occasional use of L1 is no problem.
  • 3 - L2 use is mandatory for the presentation of the final product and sometimes also during preparation. (quality indication: understandable for a native speaker). Thorough preparation for this is OK.
  • 4 - Task requires a substantial amount of L2 use in the preparation and presentation of the final product. Some prepared notes can be used but –depending on the task and/or product features - presentation requires ‘impromptu’ L2 use.
• 14.

  Language production 2 (interaction)

  To what extent does the task trigger communication in L2 between various people?

  [i]
  • 1 - Learners are allowed to use L1 in all phases of doing the task. Task description is also in L1.
  • 2 - L2 is used for task description and additional information but there is no functional reason for learners to use L2 when doing or collaborating for task related activities.
  • 3 - Task contains stimuli for learners to use L2 while working on the task (in groups). The Teacher’s Page has suggestions for appropriate moments for L2 use during lessons.
  • 4 - Learners are stimulated to use L2 in every possible way when working on tasks and during group work. The task description and all task related information and resources are in L2. Task is designed in such a way that oral or written communication beyond the classroom (e.g. with e-Twinning partner school, foreign informant/expert etc) contributes to the quality of the final product.
• 15.

  Use of strategies (receptive and productive)

  To what extent does the task provide the learner reasons to compensate for his lack of knowledge by using compensatory strategies (such as productive communication strategieën strategies and/or receptive guess- and prediction strategies)?

  [i]
  • 1 - Due to detailed structuring and the prefab nature of the task no demands are made on strategic activities.
  • 2 - Considering the level of difficulty of the source materials and the criteiria for the presentation of the final product it can be assumed that learners will need to make use of compensatory strategies. There are, however, no explicit references to this issue.
  • 3 - The task cannot be done well without making use of compensatory strategies. Possibilities in the receptive and/or the productive domain are mentioned. There are references to exercises and training opportunities.
  • 4 - The task cannot be done well without making use of compensatory strategies. Possibilities in both the receptive and the productive domain are mentioned. Actual training is embedded.

Section C: Assessing to what extent the task is task-oriented rather than subject matter-oriented (i.e. focussed on the target language as an object of study

These criteria have been derived from the field of ‘task-based learning’, ‘co-operative learning’ and ‘competence-oriented learning’.

• 1
  2
  3
  4
• 16.

  Focus on subject matter or on task-product?

  [i]
  • 1 - The task appears to have been designed to accommodate the wish to teach and practice a certain subject-item (‘the past tense’, ‘introducing oneself’). A substantial part of the time is devoted to this (not seldom without direct and necessary link to the task-product). The product acts as as kind of encore, and is not always convincingly functional.
  • 2 - It is clear that the designers had distinct subject-items in mind. They are e.g. clearly emphasized in the production process. But the designers also tried to place the subject-items in a functional context. The subject-items clearly do not form the core of the student-activity. Nevertheless the mastery of such subject-items plays an important role in the assessment of the products.
  • 3 - The fact that the designers had certain subject-items in mind and would like to stress them, can still be recognized, but working on the product and the requirements for content and form are paramount. The achievements are mainly assessed on criteria concerning content.
  • 4 - Appears to be designed starting almost exclusively from the final product (a video, a website, a report, an exhibition) and the requirements for information used and skills acquired. Subject-items, if at all present, are there to support learners in realising their goals. What is learned and possibly practiced is always functional for the content and the product.
• 17.

  Concern of the teacher or concern of ‘imaginary client, person, or agency for whom the product is intended’ (degree of reality)?

  [i]
  • 1 - The final product has little reality value. In fact presentation is only necessary to allow marking. Not being functional the assessment could also be based on a test or paper.
  • 2 - The task is the primary, functional framework, but the degree of reality is substantially flawed to increase the possibilities for teachers to assess and mark the result. Presentation: in the form of a paper or orally in front of the class.
  • 3 - Is basically designed starting from an imaginary, more or less realistic request from a ‘client’. Full authenticity however is not realised e.g. because presentation of the product is in the form of a report to the class instead of a functional procedure or artefact such as a film-performance, a course, a brochure, a manifestation.
  • 4 - Is completely designed starting from a request or need of an (imaginary) client. The product specifications and the assessment criteria are directly related to this. The presentation of the product should suit the situation of the ‘client’. The assessment is mainly focussed on the estimated degree of client satisfaction.
• 18.

  Closed or open tasks?

  [i]
  • 1 - The task and/or parts of the task ask(s) for one concrete result (Type: “In which of the following restaurants can you get a hamburger?”) There is only one correct solution, which was pre-determined. Resources are supplied and mandatory.
  • 2 - There is some variation in outcome or product, but the requirements for product and presentation are fairly rigid. There is little variation in the final products to be expected, nor is it wished for. Other resources than the mandatory ones may be used, but the latter should at least be consulted, often with set assignments.
  • 3 - Many different results are possible, but there is so much control that the various product do not differ very much. The use of the supplied resources is recommended. The use of these (or additional) resources is free.
  • 4 - Requests a product or parts of the product using specifications (Type: “Agree on a menu. You can spend € 10 each.”) Essentially there is an infinite number of different “correct solutions”. The way the students approach the problem is free. The supplied resources are optional. Originality is stimulated.
• 19.

  Everyone does the same, or teamwork with distribution of roles & tasks?

  [i]
  • 1 - Task guides to “All students do the same”. Can also be done individually.
  • 2 - Is presented as “teamwork”, but there is no distribution of roles and tasks. Tends more to ‘1’ than to ‘4‘.
  • 3 - There is a distinct distribution of roles, but there is a clear overlap in the various team members’work. Tends more to ‘4’ than to ‘1‘.
  • 4 - Task guides to a distribution of tasks in accordance with interests and talents. It pre-supposes a team and allows negotiation on distribution of roles and work involved in the related subtasks. The roles may be interdependent or supplement each other but there is little overlap in content.

• Total section A:
• Total section B:
• Total section C:

• = An acceptable LanguageQuest scores:
  13 points in section A (items 1 – 7) among which no more than 2 ones
  18 points in section B (items 8 – 15) among which no more than 2 ones
  ?11 points in section C (items 16 – 19)
• = A good LanguageQuest scores:
  15 points in section A (items 1 – 7) among which no more than 1 one
  21 points in section B (items 8 – 15) among which no more than 1 one
  12 points or more in section C (items 16 – 19)
• = A great LanguageQuest scores:
  17 points in section A (items 1 – 7) where all scores are twos or higher
  24 points in section B (items 8 – 15) where all scores are twos or higher
  13 points or more in section C (items 16 – 19)

NB - The model for calculating the total score is partly determined by a slight preference for task or problem based education and the insight that language competence develops through content-based, meaningful activities rather than because of the systematic practice of discrete subject matter items. You may have different ideas about this. In that case, with a view to the characteristics of your learners (e.g. beginners!) and/or your personal preferences, you may prefer LanguageQuests that score only moderately on the criteria 8, 13, 14 and 16-19.

Suggestions for improvement