Home     Try it out     Download     Products     Case studies     Communities     Support     
Wednesday, February 26, 2003
List shortcuts

Question:    

What is the best practice for multiple authors working together?

Answer:

Introduction


Aims

The aim of this article is to serve as a guide that provides information to help the reader setup Perception for Web successfully within a multiple author environment.

A multiple author environment is defined as a Perception Server for which more than one author create questions, assessments, templates and other resources. These authors may all be local or remote, or there maybe a mixture of local and remote authors.


Readership

This document is aimed at the following people in organisations deploying Perception in a multiple author environment:

  • Project Leader
    The Perception Project Leader in an organisation is the person responsible for ensuring the successful deployment and use of Perception. They will want to read this document to ensure that they are aware of the best practice recommendations and can (along with the Gatekeeper) design and monitor procedures in their organisations that implement the recommendations in this best practice guide.

  • Perception Gatekeeper
    The Perception Gatekeeper is generally responsible for the Perception server(s) that an organisation has. They will have a good working knowledge of the whole of the Perception suite of programs and will be responsible on a day-to-day basis for implementing the agreed procedures. Although the Gatekeeper is spoken of, in this document, as if they were one person, organisations will of course want to ensure that they have a back-up Perception Gatekeeper in the event of leave or sickness.

  • Perception Authors
    A Perception author, is anyone in the organisation who has been tasked with providing content for inclusion in a Perception assessment. The term generally applies to people who create questions and assessments, however, some authors may specialise in providing graphical, multimedia, Java or Flash content for inclusion in questions and assessments. Authors will want to read this best practice guide to understand the issues that arise in a multiple author environment.



Assumptions

This document assumes that you have successfully installed your Perception Server and Perception authoring software. Full installation instructions are given in the Perception Server Readme file and the Perception author Readme file.


Perception authoring tools

Perceptionís authoring tools are used to create questions, assessments and templates. There are two types of tool Ė those that are Windows-based applications (WBA) that run on the authorsí desk-tops and those that are browser-based applications (BBA) that run on your Perception webserver.


Desk-top authoring tools

The desktop authoring tools are Windows applications which run on the authorsí PCs and connect to the chosen authoring databases. There are three desktop authoring tools:

  • Question Manager
    Question Manager is the tool you use to manage your Perception question database. With it you can create hierarchical trees of Perception topics, within which you store your Perception questions.

    With Question Manager you can add questions to the topics within your question database by either writing your questions using a simple wizard, or using the flexible question editor, or by writing your questions in QML. In addition Question Manager allows you to import the contents of QML or QTI XML files into a topic.

    Question Manager connects to the question database using ODBC. It can open an MS Access format question database to which there is file path and can open SQL Server or Oracle question databases via system DSNs.

  • Assessment Manager
    Assessment Manager is the tool that authors use to manage your Perception assessment database. With it authors can create Perception assessments by drawing upon questions from a question database to add to an assessment.

    Assessment Manager connects to the assessment and question databases using ODBC. It can open an MS Access format databases to which there is file path and can open SQL Server or Oracle databases via system DSNs.

  • Template Editor
    Template Editor is the tool that you use to create and edit Perception templates. Templates are text files that act as style-sheets for Perception assessments, controlling many formatting features.

    Template Editor connects to the assessment and question databases using ODBC in order to allow you to preview your assessment blocks whilst you edit the assessment template. Template Editor connects solely to MS Access format databases and does not connect to SQL Server or Oracle databases.



Browser-based authoring tools

The browser-based authoring tools are part of the Perception Server installation. They are server applications which run on Perception Server and to which authors navigate with their browsers. They are accessed from the home page of Perception Enterprise Manager. They connect to the question and assessment databases via the ODBC system DSNs that the Perception Server is configured to use.

The browser-based tools generally offer a reduced subset of question and assessment creation features compared to their desk-top counterparts. However, the browser-based tools have the added feature of enforcing a rich security model which controls exactly what an author can and cannot view, create, edit and delete in the question and assessment databases. The permissions afforded to an author depend on the administrator account they use to access Enterprise Manager.

There are two browser-based authoring tools:

  • Question Manager
    The browser-based version of Question Manager offers similar functionality to its desk-top equivalent, in as much as it allows you to manage your question database, adding, editing and deleting topics and questions. The browser-based Question Manager, however, does not allow authors to create Drag and Drop, Explanation, Fill in Blanks, Hotspot, Macromedia Flash, Matching, Matrix, Pull-down list, Ranking, or Select a Blank questions. Nor can authors create and use tags, or topic outcomes in the browser-based tool.

  • Assessment Manager
    The browser-based version of Assessment Manager offers similar functionality to its desk-top equivalent, in as much as it allows you to manage your assessment database, adding, editing and deleting assessments. The browser-based Assessment Manager, however, can only create assessments with a single question block and with a maximum of two assessment outcomes. Nor can authors use tag equations or control topic reporting from the browser-based tool. Unlike the WBA Assessment Manager using BBA Assessment Manager authors cannot publish from one database to another.



Topic tree structure best practice

Perception topics allow you to group the questions in a question database into a logical hierarchy. Each topic has a name and optionally authors can give topics descriptions and outcomes. Topic outcomes allow authors to create assessments that provide specific feedback to participants based on the percentage score that they have achieved for each of the topics in an assessment. Topic outcomes can also be reported on in some reports in the Perception reporting tool Enterprise Reporter.

Topics and topic outcomes make it simple to create assessments and reports based on learning objectives. To utilise these features you should observe the following best practice recommendations:

  1. Ensure that you have defined your learning objectives before you create your question database topic structure.

  2. Create a topic for each learning objective.

    Where one learning objective is logically a part of a more general learning objective then use the hierarchical nature of Perception topics to create a sub-topic of the general topic. For example, if you wanted to create assessments that tested participantsí knowledge of Microsoft Office then you might have the following questions:

    1. What is Microsoft Office?

    2. What is Microsoft Excel?

    3. Which Microsoft Excel menu item do you use to open the Row Height dialog box?

    4. What is Microsoft Word?

    5. Which Microsoft Word menu item do you use to open the Font dialog box?


    You might decide that these five questions test knowledge in five different learning objectives:

    1. Microsoft Office

    2. Microsoft Excel

    3. Sheet formatting in Microsoft Excel

    4. Microsoft Word

    5. Document formatting in Microsoft Word


    The logical hierarchy of these topics would be:

    Topic Hierarchy


    A question database that echoed this topic hierarchy would look like the following in desk-top Question Manager and Browser-based Question Manager:

    Topic hierarchy in Question Manager


  3. Ensure that each question is placed in the lowest possible level of the topic tree to ensure that the topic in which questions are created echoes most precisely the learning objective tested by each question.

    For example, the question "Which Microsoft Word menu item do you use to open the Font dialog box?" could logically be placed in the topic Microsoft Office, or the topic Microsoft Office \ Word or the topic Microsoft Office \ Word \ Document formatting. However, the lowest level in the topic tree is the best place for this question as it most precisely describes the learning objective tested by the question.

  4. If you wish to report on specific learning objectives in Enterprise Reporter, ensure that any topics that contain questions that are used within an assessment, are set to be reported upon in that assessment. (Reported topics are set in the Control Block of the assessment.)

  5. If you wish to provide feedback on specific learning objectives to your participants during the assessment, then ensure that all topics from which questions are drawn have appropriate topic outcomes set and that the assessments using these topics have assessment outcomes set to display topic feedback.


It is important to ensure that all authors, both local and remote work with the same, up to date topic strucutre when authoring their questions. If the Perception Gatekeeper enforces the good practice points detailed in this document then this will be the case.


Best practice for organisations with local authors

This section outlines the best practice for an organisation where all the authors creating content for Perception assessments work locally on the same LAN. The recommendations are:

  1. Use a development database, a staging database and a production database

    Setup three SQL Server or Oracle databases:

    1. Development database
      This contains just the question and assessment tables. All authors should use their Windows-based authoring tools to connect to these tables and should author directly in these databases.

    2. Staging database
      This which contains all the Perception Server tables should be setup in conjunction with a staging Perception Server. All authors can publish their assessments from the development database to the staging server. The staging server can be used to beta test assessments to ensure that they behave as required and that the required reports can be produced from the questions in the assessments.

    3. Production database
      This contains all the Perception Server tables should be setup in conjunction with a production Perception Server. Only the Perception Gatekeeper should be able to publish to this server. This server should be the live server from which participants take assessments.


    The setup of the various databases and servers will look a little like the diagram below.

    Databaese and Servers.


    Local authors work on a development SQL or Oracle database, then publish to a staging server. The Gatekeeper publishes from the staging server to the production database

    Note:  
    The development database and the staging database are most logically hosted on the same database server.


  2. Centrally implement the question database topic structure in the development database

    It is best practice to implement the question database topic structure centrally. This ensures central control of the learning objectives that can be reported upon in Enterprise Reporter and also that can be part of the feedback given to your participants.

    If you were to allow your authors to create their own topics then several authors might independently create topics that contained questions that tested knowledge of the same learning objective. When these questions are then published to the staging server your staging question database would have more than one topic that tested for the same learning objective. This would disrupt your attempts to select assessment questions, produce reports and provide feedback to participants all based on defined learning objectives.

    For information on recommended best practice on the design of topic trees please see the section Topic tree structure best practice above.

  3. Centrally implement question Tags

    Question Tags are also a method of classifying questions. Unlike topics, the utility of a tag is not to classify the question according to a learning objective, but to classify a question according to other criteria (for example difficulty). When creating an assessment in Assessment Manager tags can be used to specify which questions should be included in the assessment.

    It is best to agree the range of tags, and also the possible values for each tag, centrally and to create these tags in the development database. This way all authors will have access to, and use, the centrally agreed tag set.

  4. Set up your authors so that they are all using the desk-top Windows-based authoring (WBA) products connected to the development database

    Each of your authors should have a copy of the desk-top Windows-based authoring software. This will allow them to create any of the Perception question types, to set the reported topics in their assessments and to publish their question, assessments and resource files (for example graphics, templates) to the staging server.

    All authors should connect to the development question and assessment database. This will give them access to the centrally defined topic structure into which they can create their questions.

  5. Ensure that authors use the question status flag

    The question status flag applies to each question individually and can be set to Incomplete, Normal and Retired. Only questions with a Normal status are published.

    If you have some authors creating assessments based on questions that have been created by other authors, then the use of the questions status flag will ensure that assessments that draw on these questions will not include questions that are still in development.

  6. Design, implement and police a resource naming policy

    In a multiple author environment, without a resource naming policy, there is a high probability that the file names of resources that authors include in their questions and assessments will conflict. For example it is conceivable that two authors working independently might both include a graphic called question1.jpg in their assessments.

    A resource naming policy will overcome these conflicts. For information on best practice for resource naming please see the section Resource naming best practice below.

  7. Create templates centrally and disseminate these to your authors

    Perception templates are used to control the look and feel of Perception assessments and to control other aspects of assessment delivery, such as whether the assessment is delivered in standard or question-by-question delivery format. Templates are a good tool to create enterprise-wide look & feel designs for your participantsí Perception experience.

    These templates are best created centrally and then disseminated to authors so that they can use them when trying out their content in Question Manager and Assessment Manager. Generally you will not want authors to spend a great deal of time customising or creating their own templates as this will dilute the enterprise-wide look & feel.

    Further information on templates can be found in the knowledgebase question:

      How do I use templates in Perception assessments?


  8. Try out your assessments and the reports that they generate on your staging server

    The key resource which is available to organisations once assessments have been deployed and taken by participants is the answer database. This contains the results of the assessments that have been taken by your participants and is queried by the reporting tool Enterprise Reporter. The quality of the reports that you can extract from the answer database depends to a large degree on the design of the questions that make up your assessments.

    Once an assessment has been published by an author from the development database to the staging server it is important to run the assessment several times and then to use Enterprise Reporter to pull off sample reports. This will show up any inadequacies in question design that can be rectified in order to improve the quality and utility of reports that can be generated by Enterprise Reporter.

  9. Setup the Perception Gatekeeper as the only user with rights to publish to the production server

    The Perception Gatekeeper is the keystone to the processes that you setup to control your Perception Server. He/she should be given DSN access and the login password to the production Perception Server database so he/she can publish assessments that have been tested on the staging server and approved for release. He/she will also need file access to the Perception Server directory tree on the production server so that when publishing he/she can include templates and resource files as necessary.

    Restricting rights to publish to the production server to the Perception Gatekeeper will ensure that authors cannot independently decide to make assessments live. The Gatekeeper will be able to enforce the policy that only assessments that have been tested on the staging server can go live.

    You may wish to setup procedures through which the Gatekeeper needs to operate when making an assessment live. For example, you may want to require that the Gatekeeper:

    • Consults the assessment author to confirm that the copy on the staging server is correct and complete

    • Only publishes at a particular time of day (or night)

    • Ensures that the Perception Server is momentarily taken offline when publishing occurs.



Best practice for organisations with remote authors

There are three ways in which remote authors can work. They can:

  • Connect as a local author using terminal services

    Or:

  • Use BBA to author directly to the staging server

    Or:

  • Work locally offline and send the Gatekeeper Qpacks of their assessments.


Which way it is best to set up remote authors depends on two factors. First, whether they have a reliable Internet connection. Secondly, whether it is more important that the author has access to the full functionality of the Windows-based authoring software, or whether it is more important that the Perception Gatekeeper can enforce security on topic and assessment access, by using the Enterprise Manager security model.

The graphic below shows how the three different types of remote authoring interact with the development, staging and production Perception databases:

Types of Remote Authoring


There are three ways in which remote authors can work. They can connect as a local author using terminal services, or they can use BBA to author directly to the staging server, or they can work locally offline and send the Gatekeeper Qpacks of their assessments.


Remote authors with reliable Internet connections

Remote authors who have reliable Internet connections can be setup to author using Terminal Server and Windows-based authoring, or using Browser-based authoring.


Using Terminal Server to login as a local author

Remote authors who have a login to your network and who can use a Terminal Server client to access that network can login and use the Windows-based authoring software as if they were local authors. Best practice for such authors is much the same as best practice for truly local authors as detailed in the section Best practice for organisations with local authors.


Using Browser-based Authoring

Remote authors can login to Browser-based authoring by navigating to Enterise Manager in their browsers. Remote authors can be given login IDs and passwords that will restrict the actions that they can perform in Enterprise Manager. The following recommendations are specifically applicable to remote authors using BBA:

  1. Remote BBA authors should author directly to the staging server.

    BBA authors need to browse to an installation of Perception Server to run their authoring tools. In addition BBA authors cannot publish from BBA. As a result they should author directly to the staging database and will not have access to the development database.

  2. The Perception Gatekeeper should use the Enterprise Manager security model

    The Perception Gatekeeper will need to setup the authors with login IDs and passwords prior to the authors commencing work. The Gatekeeper will want to take advantage of the group-based security implemented by Enterprise Manager to restrict the topics and assessments that the author can view. The Perception Gatekeeper should ensure that he/she gives remote authors access to the topics within which they will be creating questions, by setting the security permissions to those topics such that the remote authorsí groups have full access to those topics.

  3. BBA authors should send any new graphical content to the Gatekeeper

    BBA authors can use the graphical resources already located on the staging server. If BBA authors wish to add graphical content that is not already on the staging server they will need to send this content to the Gatekeeper so that this can be placed on the server. BBA authors should adhere to the orgaisations Resource naming best practice policy.

  4. Try out your assessments and the reports that they generate on your staging server

    It is important that remote authors test their assessments and the reports that result from taking these assessments. The best practice advice here is the same as that outlined for local authors.

  5. Setup the Perception Gatekeeper as the only user with rights to publish to the production server

    To ensure that only tested and approved assessments and questions are published to the production server, rights to publish to the production server should be limited to the Gatekeeper. The best practice advice here is the same as that outlined for local authors.



Remote authors without reliable Internet connections

Remote authors who do not have reliable Internet connections will not want to use BBA or Terminal Server as there would be a possibility of them losing the connection in the middle of authoring a question or assessment. Instead they will want to author all their content locally on their remote PC using remote copies of the Windows-based authoring software and local copies of the question and assessment databases in MS Access format. The best practice specifically for such authors is:

  1. Ensure that the remote author has an up to date copy of the authoring topic structure

    The key issue for remote authors who use their own local copies of the authoring databases is keeping the topic structure in the remote question database synchronised with the central development database. This is important to ensure that the remote author places questions that address particular learning objectives into the appropriate topic created for that learning objective.

    The Perception Gatekeeper plays the key role in keeping remote authorsí remote databases up to date. The procedure that should be implemented is as follows:

    1. When the development question database topic tree is first created, the Gatekeeper should use Assessment Manager to connect to the development assessment and question database. He/she should create a dummy assessment that includes all questions from all topics and sub-topics in the development question database and then publish this assessment to new MS Access assessment and question databases.

      This will create an MS Access format question database with all the topics and questions from the development database. This MS Access format database should be emailed to all those remote authors using remote copies of the WBA software, to use as the question database into which they place their questions and from which they draw questions for their assessments.

    2. Once a remote author using remote MS Access databases has authored an assessment this needs to be added to the development database for publising to the staging server, testing, and eventual publishing to the production server. To do this the remote author should publish the assessment to a Qpack file and then mail this file to the Gatekeeper. The Gatekeeper can then import the Qpack file to the development database. This assessment should then be published (by the Gatekeeper) to the staging server and beta tested to ensure that it produces the reports required.

    3. It is likely that periodically new topics will be added to the development database. The Gatekeeper needs to ensure that these topics are disseminated to those remote authors using remote MS Access databases. To do this the Gatekeeper should:

      • Nominate a date and time when he/she will repeat step i. above

      • Email all authors informing of the impending update

      • Make a dummy assessment that chooses all questions from all topics in the development database

      • Publish this dummy assessment to a Qpack and send the Qpack file to all remote authors who use remote MS Access authoring databases

      • The remote authors can then import this QPack to update their remote questions databases, and then delete the dummy assessment from their assessment database.


      This procedure will update the remote authors with the new topics and questions that have been added to the development database, whilst leaving untouched the questions and topics that were already in the remote authorsí remote MS Acess databases.


  2. Try out your assessments and the reports that they generate on your staging server

    It is important that remote authors test their assessments and the reports that result from taking these assessments. The best practice advice here is the same as that outlined for local authors.

  3. Setup the Perception Gatekeeper as the only user with rights to publish to the production server

    To ensure that only tested and approved assessments and questions are published to the production server, rights to publish to the production server should be limited to the Gatekeeper. The best practice advice here is the same as that outlined for local authors.



Resource naming best practice

Perception resources are ancilliary files used in questions and assessments, such as graphics and multimedia files. When an assessment is published from one database to another the publishing routine offers the option to copy the resource files from the authorís resource directories to the serverís resource directories. If a resource naming policy is not developed, then it is likely that two authors will create or use different resources that have the same file name. For example two authors might both use the file name question1.jpg.

If two authors do use the same file name and both publish their assessments to the staging server then only one of the resource files will exist on the staging server. Which resource is on the staging server depends on whether the second author to publish used the Overwrite existing files option when publishing. This situation would be clearly unsatisfactory as assessments that reference graphics that have been overwritten or not published will display the incorrect graphic to the participant.

To overcome this issue it is best to develop and enforce a resource naming policy. Such policies are designed to specify how all authors are to name their resource files and to ensure that no two resource files have the same file name. As long as a resource naming policy implements these two criteria then it should prevent problems. Each organisation may wish to devise their own reource naming policy, but an example is given below by way of illustration.


Example resource naming policy

The following is an example resource naming policy that might be appropriate for a university where all authors have a login ID for the university network.

Resource Naming Policy
Introduction
This policy is aimed at all authors creating content for Perception assessments. It is designed to ensure that no two authors create resources with the same file name. Doing this would result in a conflict on the Perception Server and incorrect content being presented to participants.
Scope
Resources for the purpose of this document are defined as any of the following files:

  • Assessment Qpacks sent by remote authors to the Perception Gatekeeper for importing into the development database
  • Graphic, multimedia, Flash and Java files created by authors and included in their question content
  • Template files created centrally or created by authors
Naming convention
All resource file names should

  1. begin with the authorís login ID for the university network
  2. followed by an underscore
  3. followed by the module code for the module for which the resource was created
  4. followed by an underscore
  5. followed by a short descriptive name
  6. followed by a dot and the appropriate resource extension.

Resource file names should contain spaces and should not be more than 32 characters long to ensure that the resource can be burned on to any CD file systems should it be used in a Perception for Windows assessment.

Here is an example of a resource that adheres to this convention. The resource has been created by John Smith who has a network login of smithj. The resource was developed for the Computer Studies course CS001. The resource is a JPG graphic of a computer terminal. The resource name is:

smithj_CS001_terminal.jpg


It is likely that organisations will need to modify this example resource naming convention to suit their own setup. However, whatever modifications are made it is important that all authors are aware of this policy and that the Perception Gatekeeper ensures that it is observed by all authors.


Best practice for retired questions

Some questions may have a useful life-span, beyond which it no longer is desirable to include them in assessments. When re-publishing an assessment to a Perception Server questions that are no longer referenced by the assessment do not automatically get deleted. This can create a situation where retired questions are still presented to participants.

To avoid this happening the following good practice stesp should be observed:

  1. When an author decides that a question is to be retired they should inform the Perception Gatekeeper.

  2. When the Gatekeeper performs an update of the development database (just prior to sending a Qpack to remote authors) he/she should set the status of the obsolete question to retired in the development database, and manually delete the question from the staging and production database.


These steps will ensure that the question will no longer be presented in assessments.



Document ID: misc013
This question applies to the following: Question Mark Perception (version 3.1+)
Last revised on: 30 August 2002
 
   
Home     Try it out     Download     Products     Case Studies     Learning cafť     Support   
Copyright © 1995 2002 Question Mark Corporation and/or Question Mark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Question Mark Computing Ltd. All other trademarks are acknowledged. Legal information