Question:    

Answer:

Introduction

Aims

The aim of this article is to serve as a guide that provides information to help the reader setup Perception for Web successfully within a multiple author environment.

A multiple author environment is defined as a Perception Server for which more than one author create questions, assessments, templates and other resources. These authors may all be local or remote, or there maybe a mixture of local and remote authors.

Readership

This document is aimed at the following people in organisations deploying Perception in a multiple author environment:

• Project Leader
  The Perception Project Leader in an organisation is the person responsible for ensuring the successful deployment and use of Perception. They will want to read this document to ensure that they are aware of the best practice recommendations and can (along with the Gatekeeper) design and monitor procedures in their organisations that implement the recommendations in this best practice guide.

• Perception Gatekeeper
  The Perception Gatekeeper is generally responsible for the Perception server(s) that an organisation has. They will have a good working knowledge of the whole of the Perception suite of programs and will be responsible on a day-to-day basis for implementing the agreed procedures. Although the Gatekeeper is spoken of, in this document, as if they were one person, organisations will of course want to ensure that they have a back-up Perception Gatekeeper in the event of leave or sickness.

• Perception Authors
  A Perception author, is anyone in the organisation who has been tasked with providing content for inclusion in a Perception assessment. The term generally applies to people who create questions and assessments, however, some authors may specialise in providing graphical, multimedia, Java or Flash content for inclusion in questions and assessments. Authors will want to read this best practice guide to understand the issues that arise in a multiple author environment.

Assumptions

This document assumes that you have successfully installed your Perception Server and Perception authoring software. Full installation instructions are given in the Perception Server Readme file and the Perception author Readme file.

Perception authoring tools

Desk-top authoring tools

The desktop authoring tools are Windows applications which run on the authors’ PCs and connect to the chosen authoring databases. There are three desktop authoring tools:

• Question Manager
  Question Manager is the tool you use to manage your Perception question database. With it you can create hierarchical trees of Perception topics, within which you store your Perception questions.

  With Question Manager you can add questions to the topics within your question database by either writing your questions using a simple wizard, or using the flexible question editor, or by writing your questions in QML. In addition Question Manager allows you to import the contents of QML or QTI XML files into a topic.

  Question Manager connects to the question database using ODBC. It can open an MS Access format question database to which there is file path and can open SQL Server or Oracle question databases via system DSNs.

• Assessment Manager
  Assessment Manager is the tool that authors use to manage your Perception assessment database. With it authors can create Perception assessments by drawing upon questions from a question database to add to an assessment.

  Assessment Manager connects to the assessment and question databases using ODBC. It can open an MS Access format databases to which there is file path and can open SQL Server or Oracle databases via system DSNs.

• Template Editor
  Template Editor is the tool that you use to create and edit Perception templates. Templates are text files that act as style-sheets for Perception assessments, controlling many formatting features.

  Template Editor connects to the assessment and question databases using ODBC in order to allow you to preview your assessment blocks whilst you edit the assessment template. Template Editor connects solely to MS Access format databases and does not connect to SQL Server or Oracle databases.

Browser-based authoring tools

The browser-based authoring tools are part of the Perception Server installation. They are server applications which run on Perception Server and to which authors navigate with their browsers. They are accessed from the home page of Perception Enterprise Manager. They connect to the question and assessment databases via the ODBC system DSNs that the Perception Server is configured to use.

The browser-based tools generally offer a reduced subset of question and assessment creation features compared to their desk-top counterparts. However, the browser-based tools have the added feature of enforcing a rich security model which controls exactly what an author can and cannot view, create, edit and delete in the question and assessment databases. The permissions afforded to an author depend on the administrator account they use to access Enterprise Manager.

There are two browser-based authoring tools:

• Question Manager
  The browser-based version of Question Manager offers similar functionality to its desk-top equivalent, in as much as it allows you to manage your question database, adding, editing and deleting topics and questions. The browser-based Question Manager, however, does not allow authors to create Drag and Drop, Explanation, Fill in Blanks, Hotspot, Macromedia Flash, Matching, Matrix, Pull-down list, Ranking, or Select a Blank questions. Nor can authors create and use tags, or topic outcomes in the browser-based tool.

• Assessment Manager
  The browser-based version of Assessment Manager offers similar functionality to its desk-top equivalent, in as much as it allows you to manage your assessment database, adding, editing and deleting assessments. The browser-based Assessment Manager, however, can only create assessments with a single question block and with a maximum of two assessment outcomes. Nor can authors use tag equations or control topic reporting from the browser-based tool. Unlike the WBA Assessment Manager using BBA Assessment Manager authors cannot publish from one database to another.

Topic tree structure best practice

Best practice for organisations with local authors

1. Use a development database, a staging database and a production database

   Setup three SQL Server or Oracle databases:

   1. Development database
      This contains just the question and assessment tables. All authors should use their Windows-based authoring tools to connect to these tables and should author directly in these databases.

   2. Staging database
      This which contains all the Perception Server tables should be setup in conjunction with a staging Perception Server. All authors can publish their assessments from the development database to the staging server. The staging server can be used to beta test assessments to ensure that they behave as required and that the required reports can be produced from the questions in the assessments.

   3. Production database
      This contains all the Perception Server tables should be setup in conjunction with a production Perception Server. Only the Perception Gatekeeper should be able to publish to this server. This server should be the live server from which participants take assessments.

   
   The setup of the various databases and servers will look a little like the diagram below.

   

   
   Local authors work on a development SQL or Oracle database, then publish to a staging server. The Gatekeeper publishes from the staging server to the production database

   Note:   The development database and the staging database are most logically hosted on the same database server.

   
   

2. Centrally implement the question database topic structure in the development database

   It is best practice to implement the question database topic structure centrally. This ensures central control of the learning objectives that can be reported upon in Enterprise Reporter and also that can be part of the feedback given to your participants.

   If you were to allow your authors to create their own topics then several authors might independently create topics that contained questions that tested knowledge of the same learning objective. When these questions are then published to the staging server your staging question database would have more than one topic that tested for the same learning objective. This would disrupt your attempts to select assessment questions, produce reports and provide feedback to participants all based on defined learning objectives.

   For information on recommended best practice on the design of topic trees please see the section Topic tree structure best practice above.

3. Centrally implement question Tags

   Question Tags are also a method of classifying questions. Unlike topics, the utility of a tag is not to classify the question according to a learning objective, but to classify a question according to other criteria (for example difficulty). When creating an assessment in Assessment Manager tags can be used to specify which questions should be included in the assessment.

   It is best to agree the range of tags, and also the possible values for each tag, centrally and to create these tags in the development database. This way all authors will have access to, and use, the centrally agreed tag set.

4. Set up your authors so that they are all using the desk-top Windows-based authoring (WBA) products connected to the development database

   Each of your authors should have a copy of the desk-top Windows-based authoring software. This will allow them to create any of the Perception question types, to set the reported topics in their assessments and to publish their question, assessments and resource files (for example graphics, templates) to the staging server.

   All authors should connect to the development question and assessment database. This will give them access to the centrally defined topic structure into which they can create their questions.

5. Ensure that authors use the question status flag

   The question status flag applies to each question individually and can be set to Incomplete, Normal and Retired. Only questions with a Normal status are published.

   If you have some authors creating assessments based on questions that have been created by other authors, then the use of the questions status flag will ensure that assessments that draw on these questions will not include questions that are still in development.

6. Design, implement and police a resource naming policy

   In a multiple author environment, without a resource naming policy, there is a high probability that the file names of resources that authors include in their questions and assessments will conflict. For example it is conceivable that two authors working independently might both include a graphic called question1.jpg in their assessments.

   A resource naming policy will overcome these conflicts. For information on best practice for resource naming please see the section Resource naming best practice below.

7. Create templates centrally and disseminate these to your authors

   Perception templates are used to control the look and feel of Perception assessments and to control other aspects of assessment delivery, such as whether the assessment is delivered in standard or question-by-question delivery format. Templates are a good tool to create enterprise-wide look & feel designs for your participants’ Perception experience.

   These templates are best created centrally and then disseminated to authors so that they can use them when trying out their content in Question Manager and Assessment Manager. Generally you will not want authors to spend a great deal of time customising or creating their own templates as this will dilute the enterprise-wide look & feel.

   Further information on templates can be found in the knowledgebase question:

   How do I use templates in Perception assessments?

   
   

8. Try out your assessments and the reports that they generate on your staging server

   The key resource which is available to organisations once assessments have been deployed and taken by participants is the answer database. This contains the results of the assessments that have been taken by your participants and is queried by the reporting tool Enterprise Reporter. The quality of the reports that you can extract from the answer database depends to a large degree on the design of the questions that make up your assessments.

   Once an assessment has been published by an author from the development database to the staging server it is important to run the assessment several times and then to use Enterprise Reporter to pull off sample reports. This will show up any inadequacies in question design that can be rectified in order to improve the quality and utility of reports that can be generated by Enterprise Reporter.

9. Setup the Perception Gatekeeper as the only user with rights to publish to the production server

   The Perception Gatekeeper is the keystone to the processes that you setup to control your Perception Server. He/she should be given DSN access and the login password to the production Perception Server database so he/she can publish assessments that have been tested on the staging server and approved for release. He/she will also need file access to the Perception Server directory tree on the production server so that when publishing he/she can include templates and resource files as necessary.

   Restricting rights to publish to the production server to the Perception Gatekeeper will ensure that authors cannot independently decide to make assessments live. The Gatekeeper will be able to enforce the policy that only assessments that have been tested on the staging server can go live.

   You may wish to setup procedures through which the Gatekeeper needs to operate when making an assessment live. For example, you may want to require that the Gatekeeper:

   • Consults the assessment author to confirm that the copy on the staging server is correct and complete

   • Only publishes at a particular time of day (or night)

   • Ensures that the Perception Server is momentarily taken offline when publishing occurs.

Best practice for organisations with remote authors

Remote authors who have a login to your network and who can use a Terminal Server client to access that network can login and use the Windows-based authoring software as if they were local authors. Best practice for such authors is much the same as best practice for truly local authors as detailed in the section Best practice for organisations with local authors.

Using Browser-based Authoring

Remote authors can login to Browser-based authoring by navigating to Enterise Manager in their browsers. Remote authors can be given login IDs and passwords that will restrict the actions that they can perform in Enterprise Manager. The following recommendations are specifically applicable to remote authors using BBA:

1. Remote BBA authors should author directly to the staging server.

   BBA authors need to browse to an installation of Perception Server to run their authoring tools. In addition BBA authors cannot publish from BBA. As a result they should author directly to the staging database and will not have access to the development database.

2. The Perception Gatekeeper should use the Enterprise Manager security model

   The Perception Gatekeeper will need to setup the authors with login IDs and passwords prior to the authors commencing work. The Gatekeeper will want to take advantage of the group-based security implemented by Enterprise Manager to restrict the topics and assessments that the author can view. The Perception Gatekeeper should ensure that he/she gives remote authors access to the topics within which they will be creating questions, by setting the security permissions to those topics such that the remote authors’ groups have full access to those topics.

3. BBA authors should send any new graphical content to the Gatekeeper

   BBA authors can use the graphical resources already located on the staging server. If BBA authors wish to add graphical content that is not already on the staging server they will need to send this content to the Gatekeeper so that this can be placed on the server. BBA authors should adhere to the orgaisations Resource naming best practice policy.

4. Try out your assessments and the reports that they generate on your staging server

   It is important that remote authors test their assessments and the reports that result from taking these assessments. The best practice advice here is the same as that outlined for local authors.

5. Setup the Perception Gatekeeper as the only user with rights to publish to the production server

   To ensure that only tested and approved assessments and questions are published to the production server, rights to publish to the production server should be limited to the Gatekeeper. The best practice advice here is the same as that outlined for local authors.

Remote authors without reliable Internet connections

Remote authors who do not have reliable Internet connections will not want to use BBA or Terminal Server as there would be a possibility of them losing the connection in the middle of authoring a question or assessment. Instead they will want to author all their content locally on their remote PC using remote copies of the Windows-based authoring software and local copies of the question and assessment databases in MS Access format. The best practice specifically for such authors is:

1. Ensure that the remote author has an up to date copy of the authoring topic structure

   The key issue for remote authors who use their own local copies of the authoring databases is keeping the topic structure in the remote question database synchronised with the central development database. This is important to ensure that the remote author places questions that address particular learning objectives into the appropriate topic created for that learning objective.

   The Perception Gatekeeper plays the key role in keeping remote authors’ remote databases up to date. The procedure that should be implemented is as follows:

   1. When the development question database topic tree is first created, the Gatekeeper should use Assessment Manager to connect to the development assessment and question database. He/she should create a dummy assessment that includes all questions from all topics and sub-topics in the development question database and then publish this assessment to new MS Access assessment and question databases.

      This will create an MS Access format question database with all the topics and questions from the development database. This MS Access format database should be emailed to all those remote authors using remote copies of the WBA software, to use as the question database into which they place their questions and from which they draw questions for their assessments.

   2. Once a remote author using remote MS Access databases has authored an assessment this needs to be added to the development database for publising to the staging server, testing, and eventual publishing to the production server. To do this the remote author should publish the assessment to a Qpack file and then mail this file to the Gatekeeper. The Gatekeeper can then import the Qpack file to the development database. This assessment should then be published (by the Gatekeeper) to the staging server and beta tested to ensure that it produces the reports required.

   3. It is likely that periodically new topics will be added to the development database. The Gatekeeper needs to ensure that these topics are disseminated to those remote authors using remote MS Access databases. To do this the Gatekeeper should:
      • Nominate a date and time when he/she will repeat step i. above

      • Email all authors informing of the impending update

      • Make a dummy assessment that chooses all questions from all topics in the development database

      • Publish this dummy assessment to a Qpack and send the Qpack file to all remote authors who use remote MS Access authoring databases

      • The remote authors can then import this QPack to update their remote questions databases, and then delete the dummy assessment from their assessment database.

      
      This procedure will update the remote authors with the new topics and questions that have been added to the development database, whilst leaving untouched the questions and topics that were already in the remote authors’ remote MS Acess databases.

   
   

2. Try out your assessments and the reports that they generate on your staging server

   It is important that remote authors test their assessments and the reports that result from taking these assessments. The best practice advice here is the same as that outlined for local authors.

3. Setup the Perception Gatekeeper as the only user with rights to publish to the production server

   To ensure that only tested and approved assessments and questions are published to the production server, rights to publish to the production server should be limited to the Gatekeeper. The best practice advice here is the same as that outlined for local authors.

Resource naming best practice