Tracking User Clicks Using the xAPI Part 2

We have a working LRS! It has been a couple of months since the first post on this that you can see here. Due to other projects progress has been a little slow. Back in May we got a couple solid weeks of time to work on the next step of being able to see what users are doing in the software and comparing that against what an expert would have done. When we left off we were making statements, but writing them to a text file. We now have a working LRS powered by a SQL database that can store the statements as they are produced by the system. So lets walk through a few topics here to get you up to speed on what we have done.

The Statements

Below is an image of what the statements looked like when we were first producing them.


You can see we are missing a couple of things, namely for us the URI and the time stamp. So the first thing we did was create the simple web page here that has our URIs for one of the verbs we are using: created.


Interacted, the other verb, was available through the ADL so we set the URL to the ADL repository.
Now that we have that set, here is what the statements looked like:


You can see we are displaying the URL for the verb URI. Finally we needed to add the timestamp. This will help us gauge how much time it takes the user to perform a task. So in the TCL file we were able to pull the time from the computer and display it in the statement. That got us to what we think are good statements for our system. Next we needed to store them.


I am very fortunate to have a very good database guy that works with me. He went through the xAPI spec and made tables in the database for each item that can be written in a statement. So if we ever want to expand to store more items from the statements we already have the infrastructure built. As I stated in the first post, we are starting small and will work our way up to more complex statements as necessary. Now that we are storing the statements we needed to display them. We built an HTML page that will display the statements in real-time. Now as we are writing to the database, we are displaying them in the web page as you can see in the image.


We are super excited to be at this stage where we are seeing the statements being generated real-time.

The Next Step

We are now at the point that we have distributed the TCL file to many people to start using in their day-to-day activities in the software. We feel that this will help us find the patterns that we need to build higher level activity statements. What I mean by that is that although all the clicks are great, as we compare them to one another, we get a good idea of what the user is doing. On a higher level, we can tie these click patterns into context-style activities which really starts to give us a feel for what the user is trying to do. Our first goal is certification testing using the xAPI. Comparing clicks to clicks to see if the user did the right thing. We are going to accomplish this be creating a Gant chart of the clicks which will allow for an easy visual comparison. Our ultimate goal to be realized in the next 4-6 weeks is to provide reccomendations to the user as they use the software. We think using the activity statements is the way to do that. Build the clicks into a pattern and use those patterns to recommend pertinent help when people need it. We would love to hear your thoughts on the path we are taking and where we are going. Stay tuned as the next month is going to be exciting!

Leave a comment

Posted by on August 5, 2013 in eLearning, Tin Can, Uncategorized


Tags: ,

Behavior Leads to Behavior

I am currently reading “Last Child in the Woods” by Richard Louv and as I am reading a couple of lines jumped out at me.  “Behavior leads to behavior”, a maxim of Gilbert Lieberman he also says “For a long time we talked about knowledge leading to behavior; instead, we believe that behavior leads to behavior.”  To put this in context they are talking about teaching our children about nature and bringing nature into the classroom.  I think these two quotes have more meaning beyond this book and context.  It seems that as an industry, L&D is very focused on simply getting knowledge to people, which in some cases is needed.  In the other cases is passing knowledge going to make people exhibit behavior change?  Not necessarily, especially if there is no context relating the knowledge being passed to them to what they do or need to do.  If we want people to exhibit behavior change I think we need to immerse them in stories relating what we are trying to teach them to what they are doing.  There are many ways that we can do that, through scenarios, practice exercises, graphic novels, etc.  If you haven’t taken a look at what Kevin Thorn is doing with a serious comic, you really should here.  I think it is a perfect example of showing behavior in a story to try and change behavior.  Not only does it show behavior, but it shows the consequences of behavior.  Ultimately we hope this leads to behavior change, which takes us back to behavior leading to behavior.  The more we can get people to engross themselves in the behavior, the more likely we are to aid them in changing the behavior.  Off base?  Thoughts?  Let me know in the comments.

Leave a comment

Posted by on April 22, 2013 in Uncategorized



Fear can be a powerful thing.  I had been thinking about this post for a few days when this blog post appeared in my email from Marc and Angel Hack Life about overcoming fear.  I don’t believe in new years resolutions, but this year I said I was going to write more.  I will be honest, fear has driven me to not write more.  What if I say the wrong thing, what if it doesn’t come across correctly.  These are just a few of the things that run through my head constantly.  Every morning I wake up to a JFDI (Just F-ing Do It) sticker staring me in the face. I also hear the chorus of “Do It” from the Up to All of Us crew as I contemplate topics to write on. Well I think its time to Just F-ing Do It.  If I write it in public, it is going to give me motivation to get moving.  I am going to write one blog post per week to make myself better.  Am I going to write something stupid?  Probably.  Are some things going to sound a little different than I want them to?  Yup.  As the  blog post linked above said, just keep moving forward.  So here we go forward, the first step in writing more.  While every post is not going to be perfect, it will be a step forward.  Are you afraid of something?  Meet it head on and go for it!

1 Comment

Posted by on April 10, 2013 in Uncategorized



Tracking Software Clicks with Tin Can


I work for a software manufacturer who is looking for ways to certify our users.  Not just on written tests, but actual use of the software.  If we are going to certify the users we want to know they are following the correct process and doing it in a timely manner.  Proctored exams are a difficult thing from the man power perspective and leave a lot of room for error.  When we heard about the Tin Can API and started to talk about it we thought it had potential.

The Plan

We started looking at having our software generate Tin Can statements as the user worked their way through the software.  We could also timestamp those clicks so we could tell how long the user was taking to perform these actions.  Now, let’s take a step back for one minute, our software has thousands of fields, checkboxes, and radio buttons.  It would be a LOT of data to get all of those clicks.  In our mock up and test system we have limited the amount of data collected to entering the panel and creating an entity.  This also allowed us to simplify the verbs we would use in the beginning.  For the panel, a panel is where most of the work in the software is performed, we use the verb interacted.


Sample of a panel

So the statement is “User” interacted with “panel name”.  Then when the user clicks the create button (or the mesh button in the example above) to create an entity the verb used is “created”.  So the statement is “User” created “entity”.  We also get a command that runs when the panel is exited.  It runs with the verb created right now, which is not ideal, but for initial testing it is OK.  We could also timestamp the commands to see how long it takes the user to complete the action.  The other thing this gives us is the ability to see if the user runs the same command back to back without exiting.  Seeing this would show us user struggle in a particular panel.  With that plan, we could give the user the requirements for the test and the expected result and then let them go.  We would have an expert create a baseline set of statements that we could then compare the users test against to ultimately decide if they were certified.  If not certified we could make suggestions on content that might be useful lto help them improve their skills.

Where are we now? 

We have the software creating the statements and writing them to a text file.


Interacted With Panel Statement


Created Entity Statement

We have the database for the LRS created and will start working on the front end GUI very soon.  Once we have that we will try it out in our training room during a live class.  This will give us a good test of the LRS and the statements that are generated.  We will also have a sample “expert” run created to compare results.  This will give as a great test of all the systems and allow us to tweak before a deployment

The future….

We would like to take the data that this can generate and build a system to help users get immediate and productive help when needed.  We will be able to tell what panels a user has gone through in a session.  Using this data we can suggest help topics, videos, blog posts, etc. to help them when they ask for help.  We are looking at an icon the user would press to activate this function.  We could also use this data to help figure out how the interface can be improved and what panels our users frequent.

We still have a lot of work to do and I will continue to blog our progress as we go forward.


Posted by on March 20, 2013 in Tin Can, Uncategorized


Tags: ,

DevLearn 12 Recap

After attending my first DevLearn last year, I was very excited to participate in DevLearn 2012.  I signed up for the DevLearn Junto which I have to admit I was a bit nervous to attend.  Many of the names I was seeing are people I really look up to in this industry.  The week started out great with a very cool keynote from Jon Landau.  Watching the behind the scenes footage from Avatar made me think how cool it would be to be able to use the cameras they had on the actors faces for elearning.  Imagine realistic avatars that could be put into safety situations or more readily convey emotion to the student.  I have to imagine we are not far away from this being a realistic situation.  I spent most of the week in the “backstage” area of the Junto.  I have to say for me personally, this was a great experience.  The opportunity to talk directly with people in unstructured conversations and listen in on other conversations was truly a great learning experience.  A big thanks to Aaron Silvers and Megan Bowe for leading the Junto.  All in all it was a great week and more than I could have expected.  One thing that hit me during Dayna Steele’s closing keynote, she made the point to “Just Ask”.  It hit home after the whirlwind week I had just gone through and the people I had met.  A lot of the people that I had spent time with at the Junto or at dinners in the evenings are people that I look up to in this industry.  People that, quite frankly, I would have been too nervous to talk to in normal situations.  As conversations took place there were things that I asked some people to explain or take a step back so I could understand something.  Anyone I asked this of seemed more than happy to to do that.  I guess the point is, if you are new to the industry or just new to attending a conference (as I was at DevLearn 11) take the time to ask if you don’t understand.  I regret not doing that enough at DevLearn 11 and am very happy to have the opportunities that I did at DevLearn 12.  Use the social media outlets ask questions, chances are you are going to learn a lot in the process.  Don’t be afraid to ask the questions, everyone I have met so far in this industry is more than happy to answer them for you.

Leave a comment

Posted by on November 17, 2012 in Uncategorized


Advanced Actions in Captivate to Grade a Workflow

After presenting our self paced course at DemoFest in November, it became apparent that we needed to add someway to track if people were actually learning the workflow we are trying to teach.  So we set forth to come up with a way that we could reward those that are able to get the process down within a minimum number of clicks.  Here is the scenario we came up with; at the end of the chapter we would give a quiz that has the students make the clicks as if they are in the software.  We give no direction at first for them to find the click.  If they get the right click in the first 3 clicks they get 10 points.  If they do not get it in the first 3 clicks, a hint caption appears.  Then when the find the right menu option, a hint caption also appears telling them why it is the right click.  When they get the correct click here they get 5 points.  We require a passing grade of 80% of the total points available.  This allows them to “miss” 3-4 clicks in the quiz.

Here is how we did it in Captivate:

First I set up some variables to record the points.  Here is a shot of the user defined variable dialog:

  There are 5 variables in the file:

addtototalpoints – this adds the score of 5 for an answer that comes after the first 3 clicks

correctpoints – this adds the score of 10 points for the correct answer in the first 3 clicks

percentage – this calculates the percentage for the pass/fail of the quiz, this is the percentage number multiplied by 100

percentagenumber – is the decimal number calculated by dividing the totalpoints variable by the captivate variable cpQuizInfoTotalProjectPoints

totalpoints – this is the total points scored in the quiz

With the variables set we now moved creating the items needed on the stage.  First we created two click boxes on each slide.  One for the correct answer and one for the answer after 3 clicks.  The second click box was created with a hint caption to tell the student why they should be clicking the option.  We also created a caption to guide the user that will appear after missing the correct click 3 times.  Once everything was created on the stage the advanced actions were created.  Here are the actions for each slide:

First on entry the slide_enter action is created:

  Each slide has a slide enter action as seen here. The enter action shows the first click box and hides all other items on the stage. Note:  I have also played with setting clickbox1 for 3 seconds and upon a “wrong” answer, having the slide Continue.  Then have the second checkbox start a run for 3 seconds.  It accomplishes the same goal without having to put the enter action.

Here is an image of the advanced action.

Next we created another advanced action if the person did not get the correct clicks in the tries.

This action hides the first click box worth 10 points, shows the caption and shows the second click box worth 5 points.  This action is set on the click box to run after the student has click incorrectly 3 times.

One thing to make sure of when creating the click boxes is the only have the 10 point click box by checking the Add to Total checkbox in the Reporting section of the properties.  If the 5 point click box has the Add to Total checkbox clicked, the points will be added to the total number available for the quiz.  This is not what we want, if we have 8 side we want 80 points to be the best score (10 points for each slide).

See below for the actions:

Now that each slide is set with the slide entities, advanced actions are used to add the score.

  Here you can see the two actions that were added to the clickbox.  The Correctpoints action simply takes the correctpoints variable and adds 10 points to it.  So the variable is simply correctpoints = correctpoints + 10.  The second action Incorrect_slide# simply hides the correct click box and shows the incorrect text box an the caption.

The incorrect clickbox uses the following actions:

  The AddPointstoTotal action simply adds 5 points to the total by using the variable.  So the action is AddPointstoTotal = AddPointstoTotal + 5 and Go to next Slide to advance to the next slide.

Once all the slides are set, we get to scoring the quiz.  On the last slide there is the action set below to total the score:

Here you can see the core being set using the variables from earlier in the post.  This is set on slide entry for the last slide which is the scoring slide.

The final action is used to set the pass fail option.  It is a conditional action to set a pass/fail message.  The action is shown here:

If the percentage is greater than 80 the Pass caption is shown.  If the percentage is less than 80 the Fail caption is shown in the Else condition.

Hopefully this makes sense.  If you have any questions please let me know.  Also any suggestions on how I might be able to this in a more streamlined or better fashion please let me know.  I will try to get a file uploaded in the next few days.


Posted by on January 22, 2012 in Captivate


Tags: , , ,

The “Duh” Moment

So while at DevLearn I heard a lot of talk about personalized learning experiences and reuse of content.  It didn’t hit me until I got back and processed what it all meant.  I spent some time researching different ways to focus the content rather than the tools to make deliver the content.  I deal with software training so I do a lot of screen capturing.  I use Captivate to do that and will continue to use it for the foreseeable future.  However, we are providing more than just screen capture demonstrations to our clients.  We are not just trying to show them button clicks, but why they do the things they do in the software.  So we are trying to teach them concepts as well as button clicks.  Currently we are using HTML and some embedded interactivity from Flash to deliver this content.  Again I am focusing on the delivery method and its limitations rather than the content itself.  SO in my research I started to look for blogs by some of the people I had seen speak at the conference.  That gets me to the “duh” moment.  I think we have all had that moment, when that new concept hits us and we get it.  That moment happened reading this blog post from Reuben Tozman.  After reading this article I had that moment of duh, that makes complete sense and why weren’t we doing this?  In my day to day job I am also responsible for our product documentation which can be the basis for our content.  Using the methods talked about in Reuben’s post and some other research I have been doing with XML it would seem to allow us to share that content in a better way.  We can have one source of content that can be used for both our eLearning courses and our product documentation.  I am very excited about the prospect of this and can’t wait to dive into how we are going to do that.  Stay tuned to for future posts as we move forward and make this happen.

Leave a comment

Posted by on December 6, 2011 in eLearning


Thoughts from DevLearn 2011

It was 3 weeks ago today that DevLearn wrapped up and I left Vegas.  As this was my first DevLearn and actually my first experience at any eLearning conference I wasn’t sure what to expect.  I was nervous and excited to arrive as I was speaking and presenting at the DemoFest session.  Each day started the Morning buzz which is a discussion group broken by topic.  These discussion were extremely informative and just provided a great forum for talking with other attendees.  The keynote speaker Michio Kaku started day one.  His session was absolutely stunning, his combination of wit and amazing predictions was something to see.  Even if you didn’t agree with what he was saying, there was something to learn just from watching his presentation style and delivery.  Following his session the Expo opened and I was completely overwhelmed by the size and scope of the vendors that were there.  I think I made at least 3 laps just taking it in before stopping and talking to one of the vendors.  I spent a lot of time in the Expo the rest of the week talking to vendors and learning everything I could about their products or services.  The concurrent sessions were great throughout the rest of the week.  Since the focus of my work is software simulation I focused my session attendance on sessions focused around those topics.  In hindsight I wish I had branched out a bit of for nothing else than to pick up ideas from other areas.  Day one ended with Tom Koulopoulos talking about the Cloud.  It was another exhilarating session by a very dynamic speaker.   Day 2 started with the final keynote Steven Rosenbaum talking about curation and the context of learning.  Again, just fantastic and very interesting topic.  In the evening of day two I had the honor of presenting one of our projects at DemoFest.  I cannot begin to express the amount of fun and how awesome this experience was for me.  The people that stopped by were so great and gained so much knowledge just from talking about what we were doing in comparison to them.  I am hoping to be accepted to the DemoFest again at a future DevLearn.  Finally on Day 3 it was my turn to speak.  I wasn’t sure what to expect from this session.  Looking back I am so grateful for the opportunity to present.  I have presented many times internally to my company about our training plans and I always get the blank stares.  There was actual dialog in this session!  People seemed excited to be there and the discussion was great.  I was so excited by the session and getting actual feedback and discussion as the session went along.  Finally the conference ended with the Ignite session.  The session was a chance for 6 people to present a new idea in learning.  The session was really inspiring and well done.  Great job by the folks from the eLearning Guild!  All in all the conference experience could not have been better.  I spent the plane ride home processing what I learned at the conference.  In fact I probably spent the next 2 weeks processing what I learned and reading blogs and following Twitter for the people I had met and seen at the conference.  I have really started to rethink our approach to learning as we create material for our customers.  We have already started to implement some new thinking focused more on content instead of the technology.  I think anyone wondering if they should attend, you definately should.  The experience cannot be matched.  In fact I told my wife when I arrived home (at 4:30am) that I didn’t care if I paid for it myself, I would be attending next year.


Posted by on November 25, 2011 in eLearning