RSS

Category Archives: Uncategorized

Image

Tracking User Clicks Using the xAPI Part 2

We have a working LRS! It has been a couple of months since the first post on this that you can see here. Due to other projects progress has been a little slow. Back in May we got a couple solid weeks of time to work on the next step of being able to see what users are doing in the software and comparing that against what an expert would have done. When we left off we were making statements, but writing them to a text file. We now have a working LRS powered by a SQL database that can store the statements as they are produced by the system. So lets walk through a few topics here to get you up to speed on what we have done.

The Statements

Below is an image of what the statements looked like when we were first producing them.

statement_old

You can see we are missing a couple of things, namely for us the URI and the time stamp. So the first thing we did was create the simple web page here that has our URIs for one of the verbs we are using: created.

created

Interacted, the other verb, was available through the ADL so we set the URL to the ADL repository.
Now that we have that set, here is what the statements looked like:

statement_new

You can see we are displaying the URL for the verb URI. Finally we needed to add the timestamp. This will help us gauge how much time it takes the user to perform a task. So in the TCL file we were able to pull the time from the computer and display it in the statement. That got us to what we think are good statements for our system. Next we needed to store them.

The LRS

I am very fortunate to have a very good database guy that works with me. He went through the xAPI spec and made tables in the database for each item that can be written in a statement. So if we ever want to expand to store more items from the statements we already have the infrastructure built. As I stated in the first post, we are starting small and will work our way up to more complex statements as necessary. Now that we are storing the statements we needed to display them. We built an HTML page that will display the statements in real-time. Now as we are writing to the database, we are displaying them in the web page as you can see in the image.

tincanscreenshot

We are super excited to be at this stage where we are seeing the statements being generated real-time.

The Next Step

We are now at the point that we have distributed the TCL file to many people to start using in their day-to-day activities in the software. We feel that this will help us find the patterns that we need to build higher level activity statements. What I mean by that is that although all the clicks are great, as we compare them to one another, we get a good idea of what the user is doing. On a higher level, we can tie these click patterns into context-style activities which really starts to give us a feel for what the user is trying to do. Our first goal is certification testing using the xAPI. Comparing clicks to clicks to see if the user did the right thing. We are going to accomplish this be creating a Gant chart of the clicks which will allow for an easy visual comparison. Our ultimate goal to be realized in the next 4-6 weeks is to provide reccomendations to the user as they use the software. We think using the activity statements is the way to do that. Build the clicks into a pattern and use those patterns to recommend pertinent help when people need it. We would love to hear your thoughts on the path we are taking and where we are going. Stay tuned as the next month is going to be exciting!

Advertisements
 
Leave a comment

Posted by on August 5, 2013 in eLearning, Tin Can, Uncategorized

 

Tags: ,

Behavior Leads to Behavior

I am currently reading “Last Child in the Woods” by Richard Louv and as I am reading a couple of lines jumped out at me.  “Behavior leads to behavior”, a maxim of Gilbert Lieberman he also says “For a long time we talked about knowledge leading to behavior; instead, we believe that behavior leads to behavior.”  To put this in context they are talking about teaching our children about nature and bringing nature into the classroom.  I think these two quotes have more meaning beyond this book and context.  It seems that as an industry, L&D is very focused on simply getting knowledge to people, which in some cases is needed.  In the other cases is passing knowledge going to make people exhibit behavior change?  Not necessarily, especially if there is no context relating the knowledge being passed to them to what they do or need to do.  If we want people to exhibit behavior change I think we need to immerse them in stories relating what we are trying to teach them to what they are doing.  There are many ways that we can do that, through scenarios, practice exercises, graphic novels, etc.  If you haven’t taken a look at what Kevin Thorn is doing with a serious comic, you really should here.  I think it is a perfect example of showing behavior in a story to try and change behavior.  Not only does it show behavior, but it shows the consequences of behavior.  Ultimately we hope this leads to behavior change, which takes us back to behavior leading to behavior.  The more we can get people to engross themselves in the behavior, the more likely we are to aid them in changing the behavior.  Off base?  Thoughts?  Let me know in the comments.

 
Leave a comment

Posted by on April 22, 2013 in Uncategorized

 

Fear

Fear can be a powerful thing.  I had been thinking about this post for a few days when this blog post appeared in my email from Marc and Angel Hack Life about overcoming fear.  I don’t believe in new years resolutions, but this year I said I was going to write more.  I will be honest, fear has driven me to not write more.  What if I say the wrong thing, what if it doesn’t come across correctly.  These are just a few of the things that run through my head constantly.  Every morning I wake up to a JFDI (Just F-ing Do It) sticker staring me in the face. I also hear the chorus of “Do It” from the Up to All of Us crew as I contemplate topics to write on. Well I think its time to Just F-ing Do It.  If I write it in public, it is going to give me motivation to get moving.  I am going to write one blog post per week to make myself better.  Am I going to write something stupid?  Probably.  Are some things going to sound a little different than I want them to?  Yup.  As the  blog post linked above said, just keep moving forward.  So here we go forward, the first step in writing more.  While every post is not going to be perfect, it will be a step forward.  Are you afraid of something?  Meet it head on and go for it!

 
1 Comment

Posted by on April 10, 2013 in Uncategorized

 

Tags:

Tracking Software Clicks with Tin Can

Background

I work for a software manufacturer who is looking for ways to certify our users.  Not just on written tests, but actual use of the software.  If we are going to certify the users we want to know they are following the correct process and doing it in a timely manner.  Proctored exams are a difficult thing from the man power perspective and leave a lot of room for error.  When we heard about the Tin Can API and started to talk about it we thought it had potential.

The Plan

We started looking at having our software generate Tin Can statements as the user worked their way through the software.  We could also timestamp those clicks so we could tell how long the user was taking to perform these actions.  Now, let’s take a step back for one minute, our software has thousands of fields, checkboxes, and radio buttons.  It would be a LOT of data to get all of those clicks.  In our mock up and test system we have limited the amount of data collected to entering the panel and creating an entity.  This also allowed us to simplify the verbs we would use in the beginning.  For the panel, a panel is where most of the work in the software is performed, we use the verb interacted.

panel

Sample of a panel

So the statement is “User” interacted with “panel name”.  Then when the user clicks the create button (or the mesh button in the example above) to create an entity the verb used is “created”.  So the statement is “User” created “entity”.  We also get a command that runs when the panel is exited.  It runs with the verb created right now, which is not ideal, but for initial testing it is OK.  We could also timestamp the commands to see how long it takes the user to complete the action.  The other thing this gives us is the ability to see if the user runs the same command back to back without exiting.  Seeing this would show us user struggle in a particular panel.  With that plan, we could give the user the requirements for the test and the expected result and then let them go.  We would have an expert create a baseline set of statements that we could then compare the users test against to ultimately decide if they were certified.  If not certified we could make suggestions on content that might be useful lto help them improve their skills.

Where are we now? 

We have the software creating the statements and writing them to a text file.

statement1

Interacted With Panel Statement

statement2 

Created Entity Statement

We have the database for the LRS created and will start working on the front end GUI very soon.  Once we have that we will try it out in our training room during a live class.  This will give us a good test of the LRS and the statements that are generated.  We will also have a sample “expert” run created to compare results.  This will give as a great test of all the systems and allow us to tweak before a deployment

The future….

We would like to take the data that this can generate and build a system to help users get immediate and productive help when needed.  We will be able to tell what panels a user has gone through in a session.  Using this data we can suggest help topics, videos, blog posts, etc. to help them when they ask for help.  We are looking at an icon the user would press to activate this function.  We could also use this data to help figure out how the interface can be improved and what panels our users frequent.

We still have a lot of work to do and I will continue to blog our progress as we go forward.

 
3 Comments

Posted by on March 20, 2013 in Tin Can, Uncategorized

 

Tags: ,

DevLearn 12 Recap

After attending my first DevLearn last year, I was very excited to participate in DevLearn 2012.  I signed up for the DevLearn Junto which I have to admit I was a bit nervous to attend.  Many of the names I was seeing are people I really look up to in this industry.  The week started out great with a very cool keynote from Jon Landau.  Watching the behind the scenes footage from Avatar made me think how cool it would be to be able to use the cameras they had on the actors faces for elearning.  Imagine realistic avatars that could be put into safety situations or more readily convey emotion to the student.  I have to imagine we are not far away from this being a realistic situation.  I spent most of the week in the “backstage” area of the Junto.  I have to say for me personally, this was a great experience.  The opportunity to talk directly with people in unstructured conversations and listen in on other conversations was truly a great learning experience.  A big thanks to Aaron Silvers and Megan Bowe for leading the Junto.  All in all it was a great week and more than I could have expected.  One thing that hit me during Dayna Steele’s closing keynote, she made the point to “Just Ask”.  It hit home after the whirlwind week I had just gone through and the people I had met.  A lot of the people that I had spent time with at the Junto or at dinners in the evenings are people that I look up to in this industry.  People that, quite frankly, I would have been too nervous to talk to in normal situations.  As conversations took place there were things that I asked some people to explain or take a step back so I could understand something.  Anyone I asked this of seemed more than happy to to do that.  I guess the point is, if you are new to the industry or just new to attending a conference (as I was at DevLearn 11) take the time to ask if you don’t understand.  I regret not doing that enough at DevLearn 11 and am very happy to have the opportunities that I did at DevLearn 12.  Use the social media outlets ask questions, chances are you are going to learn a lot in the process.  Don’t be afraid to ask the questions, everyone I have met so far in this industry is more than happy to answer them for you.

 
Leave a comment

Posted by on November 17, 2012 in Uncategorized