IT Services

XMarks blog


Wednesday, November 07, 2007

 

Choosing the right one - open source licences

From the start in the XMarks project, we set up a page within our site to be our project's rights register, where we could keep track of the external libraries we were using in our XMarks development and thus ensure we were able to select the appropriate OSI approved open source licence for our project. This is something that it was made clear we needed to do from the very first project inception meeting for our programme. We had a meeting early on with our University's Research Contracts Officer to alert him and his colleagues to the fact that we'd be needing to select an open source licence for the XMarks project.

All very good.

But you can't really decide what licence to use in advance, as the options open to you depend to some extent on the use the project makes on other people's work and how that work is licenced. Obviously the use of libraries etc will change and develop over the life of the project, so you can't necessarily agree in advance what licence will best meet your project's needs.

And then what we didn't do was keep absolute track of every library dependency. Towards the end of the project, we also discovered that we weren't really clear exactly how licensing issues applied to our project, and what was really meant by 'distributing' someone else's source code. Some long and agonising conversations ensued. Were we distributing this work? Did that throw up any licence incompatibilities?

Ross Gardler from Oss Watch spoke to us on the phone, and sent some very helpful emails
with advice and explanations to us. In the end, Rowan Wilson, also from Oss Watch, was able to come to Sussex and meet with members of the XMarks team and with Tim Harman, Research Contracts Officer at the University of Sussex. This face-to-face meeting was very useful, and enabled us to clear up a lot of our questions.

The web services component of our project required PHP to be compiled with a number of external libraries, so that we could call these libraries via PHP. However, we weren't planning to distribute these libraries ourselves and nor did our code directly invoke the library functions.

In the end, with advice from Oss Watch, we decided that we probably weren't distributing these libraries via the XMarks project. This meant that we could choose to licence the web services component pretty much as we wanted to.

We therefore decided that our XMarks web services component could be licenced with the MIT Licence. This is a permissive licence that enables many kinds of uses of the software, and we hoped that using a permissive licence would facilitate engagement with the project by external groups.

We decided to release our XMarks plugin for Moodle with the GNU General Public Licence 2, as this is what Moodle is licenced under and it was likely that our plugin could be viewed as a derivative work of Moodle.

So the "lessons learned" that we intend to take forward to future projects are:
  • get as clear as possible about the issues around open source licensing, as early as possible,
  • and seek advice from the experts when things start looking complicated!
Many thanks to Ross and Rowan from Oss Watch for all the help and advice.

Monday, September 10, 2007

 

MINTED and XMarks at Alt-C conference

We gave a presentation at ALT-C 2007 (Association of Learning Technologist's Conference) on 5th September. This presentation covered the general background to our interest in integrating Moodle with other enterprise systems, and described the work we've done with MINTED (a previous JISC funded project using the IMS-Enterprise data standard to integrate data between a student records system and Moodle) as well as explaining what the XMarks project is all about.

Thursday, August 16, 2007

 

Configuring your web server

XMarks implements mutual authentication of the client and server through the use of certificates. To perform peer validation your web server must be set up so that the directory that contains the xmarks API is set to require the client to present its certificate.


Here are some example settings for Apache 2.xx

SSLVerifyClient none

SSLVerifyClient require
SSLVerifyDepth 5
SSLCACertificateFile /System/Library/OpenSSL/certs/yourCertCA.pem
SSLCACertificatePath /System/Library/OpenSSL/certs
SSLOptions +FakeBasicAuth
SSLRequireSSL
SSLRequire %{SSL_CLIENT_S_DN_O} eq "My University" \
and %{SSL_CLIENT_S_DN_CN} eq "server1.example.co.uk"


Or For multiple servers

...
and %{SSL_CLIENT_S_DN_CN} in {"server1.example.co.uk", "server2.example.co.uk", "server3.example.co.uk"}


Thursday, August 02, 2007

 

XMarks XML schemas

In order to pass assessment and marks information around, the XMarks project has defined a lightweight information model which is expressed using XML Schema. The XSDs for XMarks have been given the namespace:

http://www.sussex.ac.uk/its/xmarks/XSDSchemas

and we have made the XSD files available there.

The overall schema is called xmData and it includes four XMarks schemas:
  • xm_id_schema.xsd - defines how the identifiers are passed around the system
  • xm_assessment_schema.xsd - defines how information about assessments is represented
  • xm_mark_schema.xsd - defines how information about marks are represented
  • xm_error_schema.xsd - defines how errors are represented and made available to calling code
These schemas are documented here, with a diagram for each of the schemas:


http://www.sussex.ac.uk/its/xmarks/XSDDocHtml/index.html


(To view the documentation at its most useful, make sure you use the "Expand All" option for the Schema Component Representation).

We have also produced a diagrammatic representation of the entire XMarks schema:


XMarks XML schema diagram


click here to view the entire schema (this opens in another window). It's a large, detailed diagram, and it may be easier to view the diagrams for each individual schema in the documentation above.

Wednesday, July 25, 2007

 

XMarks design overview part (b)

In the previous post, I talked about the design pattern that we'd suggested for integration projects, and sketched out briefly how the various layers of this pattern were implemented in terms of the XMarks project.

This post is concernd with the design of XMarks in terms of the various PHP interfaces and classes that we have created. These rely on the object model of PHP version 5.


XMarks class diagram



Click on the diagram to open a larger version of it in a new window.

The "data access layer" of the design pattern has been developed into a simple set of core functionality in the xm_methods_interface. This object interface is implemented:
  • for the assessment tool "local" service - in the diagram above, this class is named xm_methods_moodle
  • for the student records system "central" service - in the diagram, this class is named xm_methods_SRS
The web services layer of the design pattern has been developed using encapsulation too, so that the details of the transport layer implementation can be hidden from the data access layer. This makes it easy to alter how the transport layer is being managed, for example by switching from REST to SOAP.

In the diagram above, the interface xm_transport_layer_interface is implemented by xm_transport_layer, which is an abstract superclass inherited by the class that actually does most of the work, xm_transport_rest.

Finally, these two layers interact through an association between xm_methods_interface and xm_transport_layer_interface.

The code to set these objects up would look like this:
if we are doing a rest call{
// New Transport object
$xmTransLayer = new xm_transport_rest();
// New Methods object
$crsMethods = new xm_methods_SRS($xmTransLayer);

Friday, July 20, 2007

 

XMarks design overview (part a)

There's been a bit of a gap without any posts here, so I'm going to try to write some posts that clarify what we're doing and where we're going with the project.

In our original design for XMarks, we talked about building an architecture that would make it easy to create a web service for the exchange of assessment and marks data, and we identified a kind of "design pattern" for this kind of web service:



XMarks Design Pattern


Click on the thumbnail to view a larger size image.

The scenario we're envisaging is that an institution is using one or more assessment tools to support teaching and learning, and that they need to transfer marks back from the assessment tool into a central student records system.

So there would effectively be a pair of web services:
  • a student records system/central records system service (which could be described variously as the SRS, the CRS or the "central" service),
  • an assessment tool service (which we'll describe as the "local" service)
We have designed XMarks as a symmetrical architecture, with the same classes and interfaces for both the "local" and the "central" service.

We are implementing XMarks for Moodle as an assessment tool, and for our own bespoke student records system at the other end.

And building a web services architecture which we hope will encapsulate away a lot of the detail and enable relatively painless deployment.

More on the design in a second posting.

Wednesday, June 06, 2007

 

Managing association conflicts between central records system and Moodle

The XMarks Moodle module is designed to associate assessment records on your Central Records System (CRS) with activities on your Moodle installation. It does this by allowing a tutor to match records from the two systems. But obviously there can sometimes be conflicts. Minor conflicts are where the submission deadlines do not match on the two systems. An option to resolve the conflicts is displayed. Major conflicts are caused by the data being changed subsequent to a match being established and can only be resolved by dis-associating the records and starting again.

Here is a list of major conflicts:

* CRS course module id (cmid) for the CRS id (crsid) does not match Moodle cmid for the crsid
* Moodle crsid for the cmid does not match CRS crsid for the cmid

For minor conflicts, the system offers a "Resolve conflicts" button:

* The CRS hand in date for the crsid does not match the Moodle hand in date for the cmid
- The date of the CRS hand in date is later, in which case resolve conflicts is possible
- The date of the CRS hand in date is earlier, in which case resolve conflicts displays warning due to implications to the student experience, and audit trails.

Tuesday, May 22, 2007

 

XMarks team meeting, Tuesday 22nd May

At the XMarks meeting today, we talked about what remains to be completed
on the synchronisation of assessments.

We have designed this activity to be carried out by a tutor using an assessment tool of some sort, in our case our Moodle installation.

Moodle

Paolo demo-ed the current XMarks Moodle plugin. We agreed that it was now
ready enough to be trialed with faculty. Carol and Paolo to take forward.

The plugin currently allows a tutor to associate a Moodle activity with an
assessment on the student records system

It also allows a tutor to dis-associate the activity.

In our sequence diagram, after a sync or desync, the enterprise system
returns the assessment object and Moodle uses this to re-set itself. Paolo
to update code to carry out this additional step.


Web service/persistence framework

We have not yet designed a de-sync method to implement the dis-association
from Moodle.

WSDL still under construction. A SOAP service will need to run on the
enterprise database. Arran to report back on progress at next meeting,
hopefully with a running service and ideas about how to make it easy for
other institutions to implement.


Enterprise level

When an assessment is synched, the moodle activity id needs to be passed
and stored so there needs to be a new column created to manage this. Carol
to request this col.

We talked quite a lot about ways of ensuring XMarks is easy for other
institutions to use.

Where John has used Sussex Direct functions, he'll build these into an
XMarks library so that the code can be used by other institutions. We will
build functions to make XMarks as easy to use as possible - John discussed
functions that would handle translations between the required XSD XML
formats and data arrays, working on the assumption that the array indexes
matched XSD element names. John to develop library

We also talked about ways of designing a generic layer and an
institution-specific layer, possibly using the object interface features of
PHP5 - John and Arran are going to explore this further and report back at
next meeting


Tuesday, May 08, 2007

 

Integrating with Moodle - how XMarks can help

The xmarks plugin enables Moodle to be integrated with a student grades database.

The Moodle-specific plugin provides a Moodle block that allows the quiz, lesson and assignment activities to be synchronised to assessments on a grades database. It flags any conflicts, such as hand-in dates and hand-in methods, and on the click of a button resolves them. An overview of the original assessment syncronisation process is available here

The awarded and updated grades of a synchronised assessment are passed to the grades database using the standard Moodle cron job (which is recommended by the Moodle community to be fired every five minutes).

The Moodle xmarks plugin was initially designed for version 1.8, but has been modified to run for version 1.6. I don't expect there to be major developments needed to change the version for which it will run until Moodle 1.9, where they have changed the grading database tables significantly.

Labels: ,


Saturday, April 28, 2007

 

JISC Peer Review day

The Toolkits and Demonstrators' Peer Review meeting up in Bolton on 24th April was interesting and useful.

We noted down the following issues to keep an eye on:
  • ensure that we don't build in to XMarks conventions of assessment management that only apply to Sussex
  • sort out a unit testing framework
  • test on a wide range of browsers
  • expose our Subversion server for access to appropriate XMarks code and artefacts
  • make it easier for XMarks to be evaluated by developing a simple download kit
  • provide more resources at an intermediate level - somewhere between very simple and technical
  • use more images on the web site, e.g. screen shots
  • FAQ/Glossary/acronym
And we also looked at the Minted project web site, and realised it would be very helpful if we provided some simple samples of data in the IMS Enterprise XML format.

Friday, April 20, 2007

 

Dest_id & Source_id

Learned from last time, post will now be run through htmlspecialchars...

Why it's not already i don't know *glares at blogspot* , If i wanted to post html i'd use the edit html button ?!

Anyway.

In each of the XSDs defining the structure of the communiques sent between the Central Records System and Moodle there is an ID element.

XSD
-----

<xs:element name="id">
<xs:complexType>
<xs:sequence>
<xs:element name="source_id" minOccurs="0" maxOccurs="1">
<xs:complexType>
<xs:attribute name="service_name" type="xs:string" use="required"/>
<xs:attribute name="period" type="xs:string"/>
<xs:attribute name="course" type="xs:string"/>
<xs:attribute name="group" type="xs:string" />
<xs:attribute name="assessment" type="xs:string"/>
</xs:complexType>
</xs:element>
<xs:element name="dest_id" minOccurs="1" maxOccurs="1">
<xs:complexType>
<xs:attribute name="service_name" type="xs:string" use="optional"/>
<xs:attribute name="protocol" type="xs:string" default="soap" use="optional"/>
<xs:attribute name="uri" type="xs:anyURI" />
<xs:attribute name="period" type="xs:string"/>
<xs:attribute name="course" type="xs:string"/>
<xs:attribute name="group" type="xs:string" />
<xs:attribute name="assessment" type="xs:string"/>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>


<id xmlns="http://www.w3schools.com"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://direct.sussex.ac.uk id_element.xsd">
<source_id service_name="moodle"/>
<dest_id service_name="sussex_direct" protocol="soap" uri="https://direct.sussex.ac.uk/soap/api.php" period="AUT06/07" group="GroupA" course="G5050"/>
</id>


Consisting of a source_id element and a destination_id elements, each consisting of multiple attributes.

The ID element is quite generic, in that it will be used in *all* communications, and therefore must be pretty flexible.

It also provides the basis for protocol selection by the transport layer, along with resource locators.

The idea behind this was to have a multi language transport class, where we ink out a basic set of functions, and implement them in soap, and rest. Then users could pick which transport language they wanted to use.

All calls are made to that one class which then extracts the protocol and uri to use and delegates it to a sub set of functions for that protocol.. so you'd have a basic soap_getassesment function which dealt with establishing the soap client, sending a document type procedure call to a soap server and returning an XML object.

so Moodle Logic -> Xmarks Buisiness logic -> Language specific transport functions ---|

|--- Language specific transport functions -> Xmarks Buisiness logic -> SRS Specific logic

Wednesday, April 18, 2007

 

Open ended methods.

It was eventually decided the the immaturity of sca in regards to php,
was going to be quite a large issue. Many sites wishing to impliment
any of the Xmarks Code would have had to upgrade to php 5.21 which in some cases
may not have been an option.

There are also issues with the weird doc man style of defining sca enabled functions, and issues with includes, which may have hindered development.

We have therefore started considering the original proposal, which was to support
a number of languages to do data transport.

The logical way to do this would be to remove as much complexity as possible
out of the actual function calls, and instead put the complexity in the 'objects' being passed between the two parties.

This could mean for our soap implementation that we would only ever call one function on the remote server.... But that, that function would have the intelligence necessary to figure out what to do with the data is just recieved. This seems very much like the SOAP document style way of doing things ...

It also means multiple types of data could be passed in one called, or multiple instances of the same data... As there'd be no special logic at the point where moodle with the central records system.

For example:


<container>
<assessment_data assessment_type="quiz" contributory="true" conflation_rule="confAB" weighting="100" by="ac221" date="18/04/07">
<id source_id="soap://moodledev.sussex.ac.uk/xmarks/sync.php,course1,quiz1" dest_id="soap://direct.sussex.ac.uk/xmarks/sync.php,course1,quiz1">
</id>
<assessment_handin date="28/04/07" time="15:05">
<location>Xmarks development office</location>
<location>etc...</location>
</assessment_handin>
<assessment_data assessment_type="quiz" contributory="true" conflation_rule="confAB" weighting="100" by="ac221" date="18/04/07">
<id source_id="soap://moodledev.sussex.ac.uk/xmarks/sync.php,course1,quiz2" dest_id="soap://direct.sussex.ac.uk/xmarks/sync.php,apples_course,quiz1">
<assessment_handin date="28/04/07" time="15:05">
<location>Xmarks development office</location>
<location>some other location</location>
</assessment_handin>
</id>
<marks_data......>
</marks_data......>
</assessment_data>
</container>


The generic parser method would walk through the children of the root node,
and would find child 0, it would look at the node name for child 0 and determine that this was
an assessment object, it would then validate it using the appropriate schema for an assessment object and pass it on to the function that specialises in assessment objects.

This function would look at the destination ID, and check to see if this assessment that, that ID pointed at had been linked with the sourceID.. it it hadn't then the linking process would start, if it had , it would compare values within the assessment object and update where necessary.

Friday, March 30, 2007

 

SCA (Service component architecture) Installation and SOAP issues

Got our proposed transport/rpc method SCA working finally !!

Turns out that SCA is only supported on php versions 5.20 or greater, which accounts for the SOAP errors we were getting on our test server.

It also turns out that SOAP has a caching service, and seeing as SCA uses php's built in SOAP functions, php was caching all the SCA generated WSDL documents . This lead to great confusion whilst testing, as new SCA functions exposed in the remote SCA component were not being recognised in the local SCA component.

Once the directives in php.ini were altered to:
soap.wsdl_cache = 0
soap.wsdl_cache_enabled = 0

Everything was fine and dandy.

Next job is to test that SDO (service data objects) can be sent using SCA, then we can actually get down to implementing the products of the last four meetings :)

Wednesday, March 21, 2007

 

XMarks Technical Team meeting 20th March 2007

Here are brief decision notes from our Technical Team meeting on 20th March
2007

1. Arran to take forward getting the Service Component Architecture
installed on our servers

2. In terms of how things would look on Moodle, we felt that the point at
which the student clicked the "Submit All and Finish" button would be the
point at which marks were transferred over from Moodle to the
Administration database

3. We generated a list of rules around managing submission:

a) Student can make multiple attempts up to deadline (if the assessment
permits it)

b) Implication of (a) is that we need to prevent staff from editing the
record on the SRS side of things (Sussex Direct) until the deadline has
elapsed

c) If complicated mark rules have been set up in Moodle, then it's Moodle's
job to follow those rules and present the appropriate mark for transferring

4. A number of issues emerged that we would need to think through further,
e.g. discuss with the Academic Office:

a) submissions might not be transferred if students didn't press the
"Submit All and Finish" button

b) how would we be able to allow late submissions in exceptional
circumstances?

c) are there any current encodings of marks where mark is not an integer
(e.g. using an "A", "B" scale)

d) Some Quizzes might have a combination of automatically marked and
tutor-marked questions. We need to manage this too.


5. We agreed that for the peer review on 25th April, we would like to
demonstrate a working version of Assessment set up. We would probably aim
to use Service Component Architecture to do this.

Next meeting 30th March to discuss Service Component
Architecture


Friday, March 09, 2007

 

Draft DTDs and example XML

We've posted up DTDs for the draft Assessment and Marks data Information Model along with a couple of example XML files, onto the XMarks Project Activities page.

 

XMarks technical team meeting 9th March 07

Action points from today's meeting:


1. Arran to take a more detailed look at Service Component Architecture
library:

http://www.osoa.org/display/PHP/SCA+with+PHP

2. Arran and John to look into the RCP/Document style web services issue,
starting off with this discussion posted by Warwick - linked to from here:

http://www.sussex.ac.uk/its/xmarks/2007/02/rpc-style-and-document-style-web.html

3. Arran to scan in his UML sequence diagrams and him and Carol to put them
onto the Project web site

4. Arran and John to look at the XML listing coming out of Sussex Direct
for the "get_assessments" listing

5. Paolo and Carol to talk over the Moodle end of things

We agreed that next week's discussion would be more focused on Moodle.

We also agreed that we'd aim to implement the Use Case "Tutor creates an
assignment on the learning management system" for the Peer Review meeting
on 25th April.


Tuesday, March 06, 2007

 

Presentation to Assessment SIG, 22nd Feb 2007

These are the slides from the presentation to the Assessment SIG :

assessment_sig_22_feb_07.ppt

This is the information model as discussed by the SIG during the presentation:

XMarks draft info model v0-2.pdf
(As the information model changes, we will post information about it on the Project web site, in the "Activities and Deliverables" section)

Wednesday, February 28, 2007

 

XMarks technical team meeting 28th Feb 07

Action notes:


1. Arran to work on hand drawn UML sequence diagrams for these use cases we
discussed:
* tutor uses assessment tool to create an assessment that MATCHES an
existing record on the admin database
* tutor uses assessment tool to create an assessment that then CREATES a
new record on the admin database

Also to draft out a sequence diagram for sending marks, for discussion at
next meeting.

I've found this article on UML extremely helpful:

http://www.agilemodeling.com/artifacts/sequenceDiagram.htm


2. Arran and Carol to go through the DTD and create some example XML
documents for various stages of the UML sequence diagrams

3. John and Arran to look at getting XML output from Sussex Direct into the
XML format

4. Paolo to identify the various Moodle activities that might end up
generating marks data and to think about the steps we need to model

5. Carol and Paolo to take forward engaging with the Moodle community [not
sure we talked about this, but we should do it anyway :-) ]

I've set the next meeting up for this friday 9th March at 11 am - in a tea
bar if people fancy an XMarks cuppa.


Tuesday, February 27, 2007

 

Data formats/approaches to consider - OKI OSID

The Open Knowledge Initiative maintain a number of data models aimed at supporting a service orientated approach - these are the OSID models - Open Service Interface Definitions

I am posting while I have the links to hand and will return to discuss the OSID assessment and grading definitions when I've had a chance to read them.

 

Data formats/approaches to consider - the Blackboard API

Analysis of the Blackboard API may give us some ideas about sensible ways of managing transactions with assessments and marks.

This is a quote from the java docs of the API on the gradebook:

Provides the public implementation classes for Gradebook subsystem. The Gradebook is basically a 3-D container for "attempts". The "columns" are represented as Lineitem objects. The "rows" are Users in the course. Each cell displays a single value that may be calculated from one or more Scores. (The third dimension exists by allowing multiple attempts for a given line item).

A list of basic actions that can be carried out with the gradebook:

* Create a Lineitem and set appropriate properties
* Persist Lineitem
* Store Lineitem Id reference for later use
* When student interacts with "gradeable" resource, create a Score object and set appropriate properties, including Lineitem Id stored earlier
*Persist Score

The description and the list of actions are quoted directly from the javadocs documentation, which is viewable here:

http://www.utm.utoronto.ca/~ajwang/bb/bb/
blackboard/data/gradebook/package-summary.html


So far, in our thinking about scope we have stated that multiple attempts will not be tracked by our XMarks data format.

The "lineitem" is basically an assessment/assignment, and the operations on it are defined here:

http://www.utm.utoronto.ca/~ajwang/bb/bb/blackboard/data/gradebook/Lineitem.html

Thursday, February 22, 2007

 

An information model for XMarks

XMarks needs an information model for data transfer, and we are still evaluating possible existing formats and getting advice and support from the community (see some recent postings on this blog).

But we are also looking at what we think the information model needs to contain.

At the Assessment SIG today, I presented the background to the XMarks project, and then circulated a very early draft information model for comment. There was a good response from the group, who made a number of useful suggestions. It sparked off quite a lot of discussion about exactly what the scope of the information model was.

For example, some of the issues raised and the discussion on scope were:
  • managing penalty points for late submission - my feeling is that the management of penalty points, mitigating evidence etc should all be kept out of scope for this model, and that all the model needs to do is to collect the raw data from which deductions can be made if necessary. not everyone shared this view. optional attributes could be provided to support the deduction of penalties.
  • tracking which items of data might be displayed to potential employers etc - an interesting issue, and one we need to keep aware of but which probably falls outside the scope of XMarks
  • supporting multiple marks for a given assessment (for example, if a peer review exercise results in several marks) - our feeling is that the aggregation of marks is Someone Else's Problem, and that XMarks should just model a single mark for a given assessment. But I know Sam had some thoughts about that, and I wonder what the Peer Pigeon team think?
As well as this useful discussion on scope, people suggested a number of useful additional data items that we would probably want to track.

Assessments
  • assessment authored by (a person id)
  • assessment authored date
  • assessment moderated by (a person id)
  • assessment moderated date
  • date after which no submission is possible
Marks - at overall mark-list object level
  • sourcedid of the assessment in the system where the assessment took place
  • invigilated by (a person id)
  • moderated by (a person id)

Marks - at a given student's mark object level

  • marked by could be a machine process, in which case the system should indicate that, but also track who wrote the marking schema
  • duration of actual test time

 

FREMA and XMarks

At today's Assessment SIG (special interest group) meeting in Southampton, it was good to hear Yvonne Howard talking about the semantic wiki that the FREMA project have developed. The FREMA project was funded by JISC to develop a reference model for e-assessment. The semantic wiki is a record of the careful modelling work that they did, but is also designed to grow and change along with the community.

I first came across the FREMA wiki a couple of months ago and have been finding it a very stimulating place to visit and browse. I've spent some time looking at the Summative Assessment Service Usage Model and trying to see where our XMarks endeavour fits in. There's a use case here for Tracking which defines it as:

Players: SIS
  1. SIS logs that assessment has been assigned to candidate
  2. SIS logs marks achieved by candidate
  3. SIS logs assessment has been moderated
  4. SIS logs assessment grades have been awarded
(quoted from http://frema.ecs.soton.ac.uk/wiki/index.php?title=Tracking)

This is pretty much XMarks territory, although in XMarks we're aware that we need to do quite a lot of matching between assessment records on different systems, which isn't covered in this use case.

So is XMarks basically Track Service v1.0?

I asked this as a question in the FREMA wiki:
http://frema.ecs.soton.ac.uk/wiki/index.php?title=Talk:Summative_Assessment_%28
Artefact_Assessment%29

Wednesday, February 21, 2007

 

RPC style and document style web services

At our XMarks technical team meeting yesterday we discussed the issue of RPC vs document style web services. I think as a team we are more used to RPC style services. So I emailed Warwick, as I thought I could remember him mentioning the issue at the JISC Toolkit and Demonstrator projects start up day in January, to ask him if he had any good references on the subject.

He has set up a thread on the Icodeon site to discuss this more:

http://icodeon.wiki-neon.adaptavist.com/display/soa/2007/02/21/
Document+Style+Web+Services?focusedCommentId=924#comment-924


This has got lots of interesting stuff and some useful references, and can give us a place to discuss our questions about document style web services too.

Thanks Warwick!

Sunday, February 18, 2007

 

Potential data format: IMS-Enterprise

Could the "membership" data object of the IMS-Enterprise (IMS-E) data format be appropriate for the XMarks data exchange format? It contains attributes for interimresult and finalresult.

In a typical use of the IMS-E format, the "group" data object will be mapped to programme, course/module or teaching group within a given degree programme.

The "membership" object will allow the association of a given person with such a group.

The interimresult structure can be used to hold multiple interim results, for example 'Mid-Term', 'Part 1' etc:
  • resulttype (the type of interim result, e.g. 'Mid-term')
  • mode (a descriptive name for the grading mode, e.g. 'Pass/Fail', 'Percentage' etc)
  • values (a structure that contains one or more valid values for the result)
  • result (the actual result that the member was assigned, ideally a member of the values list)
  • comments (comments about the interim result)
The finalresult structure is similar, except that it does not contain the resulttype attribute. This isn't needed, as the result type is already known - it is defined as being final.

So the membership data object can allow the transfer of interim and final marks data.

Why this approach is not sufficient for XMarks

XMarks is about the exchange of both marks and assessment information. A typical use case for exchanging assessment information would be that:
  • The course definition contains a set of pre-defined assessments that will be used to determine the final result of a given course. This information needs to be passed from the course definition database into the learning management system, particularly where the learning management system is going to deliver the method of assessment.
But the IMS-E interimresult and finalresult data structures don'tt enable the assessment structure of the course to be passed separately from the results data.

Additionally, the IMS-E specification version 1.1 states that both interimresult and finalresult data objects are deprecated and are going to be removed from IMS-E version 2.0. So it might not make sense to develop a service that relies on them.

What we can learn from IMS-E

If we view the summative assessment methods as a core part of a course definition, then it might make sense to pass them as attributes of the "course" data object.
Perhaps we could take the same view of formative assignments.

However, it also makes sense to view the actual modes of assessment and results as being an attribute of the student's membership of a course, as the IMS-E specification models them.

This is powerful because it allows different students to be assessed with different modes - which happens for example when a student has disabilities which mean that the standard mode is inappropriate for them.

The IMS-E structure also allows for the possibility that different students taking the same assesment mode will have their grades represented differently. This can occur when MA and MSc students are taking the same course, and the two degree programmes are using different notations for marking.

Thursday, February 15, 2007

 

Potential data format: ANSI TS 130

ANSI TS 130 describes a student transcript for use in Electronic Document Interchange, typically between schools or other educational institutions or school districts, to transmit current and historical data.

It contains structures within it that allow an exchange of courses taken and grades attained.

Structure TST (highly simplified)
  • test name
  • date test administered
  • version of test
  • level of test
  • level of test/individual/course (e.g. 1st grade, 2nd grade, Post-secondary 1st year etc.)
  • how test norming was carried out
  • other details, e.g. language of test
For each TST structure, there must be at least one SBT (sub-test) containing information about the components of the test.

For each SBT record, there must be at least one corresponding SRE (test results) score.

Could this be the data exchange format used by XMarks?

A key part of the XMarks project is that assessment data needs to be exchanged prior to marks data being available. The TS 130 structure is much more geared to exchanging data on results. It doesn't contain all the information that would be needed for the exchange of assessment data, such as the conflation rules and weighting, location and mode of submission etc.

The emphasis for marks data is also not quite right for the XMarks project. In order for a learning management system to pass raw marks data back to a student record system, then it must be possible to encode information relating to non-submission, late submission etc.

The emphasis in TS 130 is on exchanging final results where issues of conflation, lateness, penalties etc have all been already taken into account.

Thus our view is that this format wouldn't be appropriate for XMarks.

Saturday, February 10, 2007

 

Notes from the Enterprise SIG 6th Feb 2007

Paolo and I attended the Enterprise SIG on 6th Feb 2007, where we fed back on work we'd done in the MINTED project and talked about plans for XMarks.

There was then a discussion on some of the issues that the SIG members thought would be relevant for XMarks to consider.

Permissions and security

Discussion of XACML (eXtensible access control markup language), an access control policy language ratified by the OASIS consortium (Organization for the Advancement of Structured Information Standards)

Traceability

Where did this message come from?
Ensuring it is possible to determine how a mark gets modified
At least one HE institution has failed to get its final degree classifications ratified due to poor paper logging.
Is there a policy around the logging of changes?

Reliability of messaging

OASIS WSRM (Web Services Reliable Messaging)


Interoperability - learning from other projects

The SHELL Project - information passed between institutions using IMS LIP format. The Shell project worked in a similar space to the XMarks project in that it was looking at data exchange and looking at the way that data exchange would impact on business processes.

Monday, January 29, 2007

 

Thinking ahead - producing an installer

One of the things that we're keen to do with XMarks is to make it as easy as possible for people to try out our system.

One way of doing this would be to find a way of distributing an easy installation kit that would get a working system pretty much up and running.

At the recent JISC Toolkits and Demonstrators Startup meeting, we found out about the NSIS (Nullsoft Scriptable Installation System) project, an Open source project hosted on SourceForge.

Looks like cool software. It's windows-based, which might be a limitation for us.

I'm mostly blogging about it as a kind of a book-mark; even if we don't use NSIS it's important we think about how to package our system to make it as easy as possible to try out.

Friday, January 19, 2007

 

Defining the web services

We had our first technical team meeting yesterday, where we talked about the web services for XMarks, and what kind of underlying data representation we'd need

First of all, we thought about how assessment/assignment ids would need to be tracked around the system.

Currently, our administration database (aka our student records system SRS) contains a record for all contributory (summative) assessments and all non-contributory (formative) assignments - except for those currently existing only in the learning management system!

And Moodle - our learning management system - currently has its own records for quizzes etc.

So we could see assessment data flowing in two directions.

a) Our administration database could assert the existence of a given assessment/assignment - in which case it would be able to pass the Admin id into Moodle

or

b) Moodle would be able to assert the existence of a given quiz, in which case it would be able to pass the Moodle id out to the admin database.

See illustrations below:















Thumbnail of XMarks handshake (flipchart sketch)




Thumbnail of more refined diagram of XMarks handshake


Flip chart sketch of the "handshake" or id exchange Diagram of the "handshake" or id exchange

We could potential have a synchronous handshake style exchange where the target system in each case passed back the id that it was going to use.

From there, we thought through in a bit more detail exactly what the web service might look like:















Thumbnail of flipchart sketch of web service highlevel overview




Thumbnail of web service highlevel overview


Flip chart sketch of web service overview Diagram of the web service overview

Tuesday, January 16, 2007

 

Welcome to Xmarks!

Welcome to the XMarks project!

We'll be exploring ways of exchanging marks and assessment data between our university Student Records System and our Learning Management System.

The overall approach will be to use a web service, and this is our current thinking about a flexible, generic web services architecture:


A flexible web services architecture

Archives

January 2007   February 2007   March 2007   April 2007   May 2007   June 2007   July 2007   August 2007   September 2007   November 2007  

This page is powered by Blogger. Isn't yours?