RSS

Category Archives: Tin Can

Image

Tracking User Clicks Using the xAPI Part 2

We have a working LRS! It has been a couple of months since the first post on this that you can see here. Due to other projects progress has been a little slow. Back in May we got a couple solid weeks of time to work on the next step of being able to see what users are doing in the software and comparing that against what an expert would have done. When we left off we were making statements, but writing them to a text file. We now have a working LRS powered by a SQL database that can store the statements as they are produced by the system. So lets walk through a few topics here to get you up to speed on what we have done.

The Statements

Below is an image of what the statements looked like when we were first producing them.

statement_old

You can see we are missing a couple of things, namely for us the URI and the time stamp. So the first thing we did was create the simple web page here that has our URIs for one of the verbs we are using: created.

created

Interacted, the other verb, was available through the ADL so we set the URL to the ADL repository.
Now that we have that set, here is what the statements looked like:

statement_new

You can see we are displaying the URL for the verb URI. Finally we needed to add the timestamp. This will help us gauge how much time it takes the user to perform a task. So in the TCL file we were able to pull the time from the computer and display it in the statement. That got us to what we think are good statements for our system. Next we needed to store them.

The LRS

I am very fortunate to have a very good database guy that works with me. He went through the xAPI spec and made tables in the database for each item that can be written in a statement. So if we ever want to expand to store more items from the statements we already have the infrastructure built. As I stated in the first post, we are starting small and will work our way up to more complex statements as necessary. Now that we are storing the statements we needed to display them. We built an HTML page that will display the statements in real-time. Now as we are writing to the database, we are displaying them in the web page as you can see in the image.

tincanscreenshot

We are super excited to be at this stage where we are seeing the statements being generated real-time.

The Next Step

We are now at the point that we have distributed the TCL file to many people to start using in their day-to-day activities in the software. We feel that this will help us find the patterns that we need to build higher level activity statements. What I mean by that is that although all the clicks are great, as we compare them to one another, we get a good idea of what the user is doing. On a higher level, we can tie these click patterns into context-style activities which really starts to give us a feel for what the user is trying to do. Our first goal is certification testing using the xAPI. Comparing clicks to clicks to see if the user did the right thing. We are going to accomplish this be creating a Gant chart of the clicks which will allow for an easy visual comparison. Our ultimate goal to be realized in the next 4-6 weeks is to provide reccomendations to the user as they use the software. We think using the activity statements is the way to do that. Build the clicks into a pattern and use those patterns to recommend pertinent help when people need it. We would love to hear your thoughts on the path we are taking and where we are going. Stay tuned as the next month is going to be exciting!

Advertisements
 
Leave a comment

Posted by on August 5, 2013 in eLearning, Tin Can, Uncategorized

 

Tags: ,

Tracking Software Clicks with Tin Can

Background

I work for a software manufacturer who is looking for ways to certify our users.  Not just on written tests, but actual use of the software.  If we are going to certify the users we want to know they are following the correct process and doing it in a timely manner.  Proctored exams are a difficult thing from the man power perspective and leave a lot of room for error.  When we heard about the Tin Can API and started to talk about it we thought it had potential.

The Plan

We started looking at having our software generate Tin Can statements as the user worked their way through the software.  We could also timestamp those clicks so we could tell how long the user was taking to perform these actions.  Now, let’s take a step back for one minute, our software has thousands of fields, checkboxes, and radio buttons.  It would be a LOT of data to get all of those clicks.  In our mock up and test system we have limited the amount of data collected to entering the panel and creating an entity.  This also allowed us to simplify the verbs we would use in the beginning.  For the panel, a panel is where most of the work in the software is performed, we use the verb interacted.

panel

Sample of a panel

So the statement is “User” interacted with “panel name”.  Then when the user clicks the create button (or the mesh button in the example above) to create an entity the verb used is “created”.  So the statement is “User” created “entity”.  We also get a command that runs when the panel is exited.  It runs with the verb created right now, which is not ideal, but for initial testing it is OK.  We could also timestamp the commands to see how long it takes the user to complete the action.  The other thing this gives us is the ability to see if the user runs the same command back to back without exiting.  Seeing this would show us user struggle in a particular panel.  With that plan, we could give the user the requirements for the test and the expected result and then let them go.  We would have an expert create a baseline set of statements that we could then compare the users test against to ultimately decide if they were certified.  If not certified we could make suggestions on content that might be useful lto help them improve their skills.

Where are we now? 

We have the software creating the statements and writing them to a text file.

statement1

Interacted With Panel Statement

statement2 

Created Entity Statement

We have the database for the LRS created and will start working on the front end GUI very soon.  Once we have that we will try it out in our training room during a live class.  This will give us a good test of the LRS and the statements that are generated.  We will also have a sample “expert” run created to compare results.  This will give as a great test of all the systems and allow us to tweak before a deployment

The future….

We would like to take the data that this can generate and build a system to help users get immediate and productive help when needed.  We will be able to tell what panels a user has gone through in a session.  Using this data we can suggest help topics, videos, blog posts, etc. to help them when they ask for help.  We are looking at an icon the user would press to activate this function.  We could also use this data to help figure out how the interface can be improved and what panels our users frequent.

We still have a lot of work to do and I will continue to blog our progress as we go forward.

 
3 Comments

Posted by on March 20, 2013 in Tin Can, Uncategorized

 

Tags: ,