analytics, data, data analytics, edtech, education, eLearning, Moodle, online education, online learning, Student Success, Uncategorized

Using the Engagement Analytics and the Activity Completion Report to Monitor Student Progress

38c3bdf0021b71d1065571f53fbf5f01

Being a fan of the Ironman and Marvel Avengers movies, I have long wanted a way to quickly scan online classes to get a basic read of how students are performing in real-time, and for years now it has bothered me that the data for real-time analysis of basic student success metrics has been all around us in the Moodle LMS, but too difficult to capture in any meaningful way.   Finally, a year ago though I began to pilot the use of a combination of the Moodle Engagement Analytics Block (https://moodle.org/plugins/report_engagement) to track the Log-in Activity of students coupled with the use of the Moodle Activity Completion Report to track academic progress within the course.  I feel that the pilot has been fairly successful for me personally, so I thought I would share a quick how-to on this approach.

Before this approach would work with Completion Tracking I had to adopt a basic design principle.  I have been using the Completion Tracking feature in Moodle (must be enabled in Course Settings) exclusively for module assignments involving actionable items, things that a student is expected “to do”.  This has helped to not only clarify expectations for students, but to also enables the effective and efficient use of the Activity Completion Report to monitor classes for student progress in ways that hopefully work to improve student success.

EnableCompletionTracking

The Completion Tracking component of assignments is indicated to students in Moodle by the presence of distinctive check boxes in the right margin parallel to the assignments for which this feature has been enabled.  There are a number of different criteria these check boxes can be set to in order to assure individual students that they are completing all of the required items for a given module.  These check boxes also contribute the data needed to track the relative progress of that student against the aggregate module/course completion of the class.  Check boxes can be manually checked by students or set to automatically fill based on:

  • A preset score
  • Submitting an assignment
  • Posting a set number of posts and replies to a forum
  • Clicking/viewing a link
  • Receiving a grade

Completion TrackingActivityCompletionReportThe effective use of this tool can help assure students that they are completing all activities for a given module and that each activity is completed in the way intended by the instructor.  For instructors, students in the Activity Completion report are listed along with their peers which as I have said, I have find to be a more useful metric than simply monitoring late assignments.  Assignments that are past due are coded in pink and the process of identifying students that are lagging behind their peers can be checked quickly and easily, particularly if the report is bookmarked and incorporated into a browser menu.

Activity CompletionThe key to effectively using the second element of this approach, the Engagement Analytics block, is to calibrate it to only measure weekly Login Activity.  This can be accomplished by setting the block to weight Login Activity for 100% of the aggregated score while setting Assessment Activity and Forum Activity each to 0%.

Login100I have found the Engagement Analytics block to be unsatisfactory when tracking Assessment Activity and Forum Activity largely due to both technical errors as well as practical limitations where the tool tends to be too inflexible for widespread use when due dates are extended or modified in any way.  Listed below are the options available for tracking Log-in Activity, and as you can see even with just using this one metric, the options are fairly robust:

  • Expected Logins in the Past Week
  • Expected Logins per Week
  • Expected Average Session Length (in minutes)
  • Expected Time Since Last Login (days)
  • Session Length (minutes)

Unnamed imageLogin Activity is now easily monitored through the dashboard provided with the Engagement Analytic block.  This block sorts students based on risk % as well as color coding students on a Green, Yellow, Red scale for easy monitoring.  The names of students with the highest risk % appear at the top of the list with a corresponding color gradually moving from green to red.  The student’s name is linked to more detailed analysis of why they are being flagged by the block.

The end result of this dual combination is a system offering instructors the ability to easily track and monitor student Login Activity and relative Activity Completion in a way that has scalablility without losing effectiveness.

 

 

 

Standard
2017 Presentation of Best Visual Effects
edtech, education, eLearning, Moodle, online education, online learning, Uncategorized

What we can learn about eLearning from the Oscars

2017 Presentation of Best Visual EffectsAfter watching all that was the 2017 Academy Awards last night, I couldn’t help but particularly notice the awards for special effects.  The Jungle Book was nominated based on how truly realistic the CGI generated animals in this latest rendition of the story of Mowgli appeared.  Another nominee, Rogue One: A Star Wars Story was astounding because of their application of these same advances in CGI to convincingly render human beings, some long since passed away, within the same space as live actors.  The Jungle Book ended up taking home top honors last night for Best Visual Effects, but I was surprised at how many additional awards had been presented at an earlier ceremony for Scientific and Technical Achievements.

66cbc1eb8c0d8c0aa01702d33534243c814636c3e995b930b26d66b220bf8edc-770x443The Scientific and Technical Achievements ceremony awarded 18 scientific and technical achievements that were recognized for various technological achievements including expression-based facial performance-capture technology, wireless microphones, and high resolution cameras.  As I listened to the presenting actors in last night’s Oscar ceremony discuss this particular aspect of movie making I began to think the whole conversation had a familiar feel.  This familiarity may have been best expressed by Rogue One’s Riz Ahmed and Felicity Jones when presenting the award for Best Visual Effects.  They noted that the award had changed names a number of times over the years including best engineering effect, computer generated effect and concluded, “…as I like to call it, we don’t need actors anymore we have computers effect” (emphasis mine).  This comment really brought it home for me, it is the same struggle that we have in online and blended education: how to incorporate emerging technologies within the discipline of education without compromising and/or displacing role of the instructor and along with it the quality and effectiveness of the F2F experience in our pursuit of the limitless potentials of the digital world.

635960002564646182-the-jungle-book-016-wc-0250-comp-v0439-right-86405I think it is telling how many of these Academy Awards went to technologies focused on moving the digital space towards a singularity of experience where the worlds represented digitally are so incredibly close to reality that our experience is almost seamless.  There seems to be a shared recognition by Hollywood and Higher Ed of the value of only entering into these digital worlds with our full humanity.  Immersing ourselves in a structure that is “real”, is moving not just for the students and the audience, but for the actors and instructors as well. The desired result for both industries is a shared experience that is meaningful, where the lesson clicks, the movie works, the narrative moves forward and hopefully, we leave the experience in some way changed.

As I watched the clips for the nominated films for Best Visual Effects, it occurred to me that the element most in need of further training and development for seated/online/blended instructors in this new digital landscape may in fact actually be “acting” lessons.  This need is most evident when I work with instructors teaching in a traditional classroom, but trying to adapt to some sort of technological innovation to their instruction.   Largely the mode of technology is inconsequential compared to the way they have constructed their learning space within their own mind and those of their students.  I have seen technologies miss the mark many times over the years, whether it is a web-cast course where the instructor begins with a distant wave and a far-off voice greeting the remote students like they are literally only within shouting distance of the room, or the use of magnification and projection technology where the instructor gets pulled into their own presentation rather than focusing on speaking to their students as if they are looking over their shoulder.  Technologies are finding their way into our learning spaces, but many times they are not being conceptualized and implemented in a way that sells us on the effect that the technology is meant to create.  When we as instructional designers and instructors are not specific about the problems that we are attempting to address with digital technologies, then the execution in our delivery can fall flat and prove to be more of a distraction than an advantage. 

king-the-tiger-as-richard-010After the Oscars, I thought about this as I was listening to an earlier discussion by a previous Visual Effects award winner, Life of Pi, and it was mentioned that at one point the movie was not going to be allowed to be filmed in India.  The Indian government believed that a tiger had been starved and abused based on some scenes toward the end of the movie.  It was only when they demonstrated that the tiger was nothing more than a blue sock that the filming was allowed to move forward.  Here technology served a need that could not otherwise have been met.  It enhanced the narrative and both the actor and audience benefited.  It is this type of problem-based approach that we need to develop for the responsible integration of technologies into our classrooms as well.  When all parties, the instructor, the student and the instructional designer approach a digital environment from the same intentional point of view, the potential of a digital technology is maximized as the desired outcome shares a common end point that everyone can work toward and a common metric with which to evaluate the relative success of the technology.

lightsabergreenscreenWhat does it look like when these interests are not aligned?  I inevitably get a bad feeling I when watching Star Wars Episode III because something just doesn’t feel right.  Something doesn’t work in the much maligned Star Wars Prequels, but particularly the third one for me despite demonstrating the full power of all of the technological wonder that was LucasArts at the time.  I have a hard time putting my finger on what is missing in the classroom sometimes, but I think that for me identifying what is wrong with Episode III is more readily understood and agreed upon and might be instructive for comparative purposes.  Episode III definitely mixes in its share of bad acting but there is an additional feeling of discombobulation that accompanies the characters.    You have a milestone shift in Episode III when almost all of the movie is CGI, not one single clone trooper is real, they are all 100% CGI.  In this last installment, you have great actors like Samuel L. Jackson, Ewen McGregor and Natalie Portman who just can’t quite pull off acting in an environment where literally everything doesn’t exist, except in the computer.  They seem to have lost their muse, and I would say here is where a wider lesson on the dangers of too much digital technology without an accompanying strategic visualization exists for online learning as well.

article-2143371-130dc4ec000005dc-143_964x641You see the original Star Wars movies weren’t shot with CGI and green screens, they were shot on honest-to-goodness physical movie sets, with actors in costumes, but once the allures of technology became available we began to move from movie props and scenes to green screens and CGI.  Just to prove that I am not totally a Luddite, I can vividly remember first seeing Jurassic Park in the theaters in 1993 and being on the edge of my seat for the entire movie.  I even remember midway through the movie realizing that what I was seeing wasn’t real at all, but that somehow I had totally found myself pulled into this digital world.  It wasn’t until almost 20 years later when I first watched the Marvel Avengers movie, and more recently Star Wars The Force Awakens that I had similar experiences.  I wouldn’t necessarily expect the actors in these types of movies to win Academy Awards, but honestly, based on their abilities to overcome the detached, impersonal, hollow qualities of some of their ill fated prequels maybe it isn’t a surprise that there is a separate award ceremony for the digital artists working to bridge these gaps.  To paraphrase Princess Leia from Rogue One, this gives us hope.

jurassic-park-t-rexWhat we need in online learning is a move away from the zero sum game.  What we need is a partnership between the physical and the virtual.  We need to incorporate both elements into a learning environment in which both the actors and audience, instructors and students, are equally immersed.  We have long meditated on what it means to adopt technologies in education that increase learning and that act as force multipliers to enhance our abilities to move students in ways that transport them into  learning spaces that are crafted to facilitate learning in ways that cannot be replicated in the F2F classroom.  In these spaces the sum of the parts is greater, not less, than the whole.  In order to do this we need a collaborative, unified vision of what we are trying to teach.  We need objectives, and we need to envision how these individual components of technology, course design, and instructor expertise will work together synergistically to create a digital reality that maintains a sense of “presence”.

The introduction of new technologies to this space have the potential to either augment instruction to levels not previously possible, or to detach the instructor and students from the learning experience in ways that are devoid of interaction and engagement.  When this happens, the technology becomes a distraction rather than an enhancement.  I guess that is one of my secret fears of technology, and this stems all the way back from high school shop class.  Technology, as I have said before, is a force multiplier.  When that force is multiplied in constructive ways, then everyone benefits and the discipline of education moves forward, but when applied in destructive ways it has the ability to do tremendous harm, to maim and crush.  This might be physically when it is a band saw in shop class, but emotionally do damage to self-confidence when unleashed in the classroom.  Many times the students we most want to benefit from the advantages of online education, like flexible time and locations, are also our most vulnerable.  When technology is poorly executed, they are the least able to absorb the lapse.

hpwrwbvfqdeytr5pnlvaIn the end the challenges being addressed and rewarded by the Motion Picture Academy are not a surprise to us in eLearning.  The formula and respective technologies that are currently moving things forward recognize the need for an underlying framework.  I have written in previous posts of our acute ability to sense whether we are in a machine environment alone or where there is a person in there with us.  The sense of presence, as it turns out, is not as easy to fabricate as we might think.  I think Jurassic Park worked for me largely because so much of the set wasn’t CGI, and the actors were immersed in an environment that convinced me because I felt the gravity of their presence.  Because of this, I ascribed the same reality to the velociraptors they were all actually just pretending to see.  For much of the same reason, Star Wars Episode III failed for me, but Rogue One and The Force Awakens seem to work.  Technologies that enhance us as actors and instructors, that capture us as humans “being” seem to be advancing in a direction that we feel a need to reward, technologies that don’t are replaced or panned.  This question of whether there is a person “under” the technology is one that is even more urgent to address in online education where we are not merely viewers of a digital experience, but participants, and the changes we seek in our audience need to last longer than the credits.  

 

 

Standard
Hand drawn sketch of Hogwarts
ADA, edtech, education, eLearning, Harry Potter, Moodle, online education, online learning, Uncategorized

Bringing the Magic of Hogwarts to your Online Class

We’re currently in that magical period of possibility and optimism know as “graduation season” where all of us involved in the world of academia either professionally or by proxy through someone graduating are feeling the pressure of the school year wind down and the dreams of many culminate into just so much pomp and circumstance.  For the first time in a while I found myself in a space to reflect on the past year and think a little about the future.

Hand drawn sketch of Hogwarts

Working in the field of eLearning, more and more ADA compliance in online classes is coming to dominate the agenda of the local, regional and national conferences I attend.  Sitting in these sessions has provided me a great opportunity to rethink the traditional ways that I view learning.  Recently I was thinking about the ways that I interact with and interpret the world around me, and I had a sudden insight into Harry Potter.  I think that the magic, the REAL magic of Hogwarts may not in the invisible barrier that keeps death eaters from entering (even thought that is admittedly pretty cool), or the power of the Whomping Willow or even the ever-changing staircases and perpetual feasts in the great hall.  The real magic for me about Hogwarts is that from the time Harry Potter and the Sorcerer’s Stone, the first Harry Potter book, was published in 1997 to the date the first movie was released in 2010; 13 years had passed.  So when fans of the book finally got to see the movie, there was more than a little bit of pressure on Hollywood to “get it right”.  And in my opinion they did, in remarkable fashion they got it right.  As a matter of fact the movies really held their own cinematically as you can learn more about here.

But perhaps MORE importantly even than the cinematic techniques that made the films so captivating, is that in fact, for millions of fans all over the world the images and sets depicted in the movie didn’t just look like Hogwarts or favor Hogwarts, they WERE Hogwarts.  We all recognized it and accepted it for being what it was.  And this is significant because none of us had ever actually SEEN Hogwarts.  It was just an incredibly detailed and complex place that had been constructed in our minds eye.  We immediately recognized it though and loved it.

This has important ramifications for ADA.  If we’re not careful, much of the content in our online courses can tend to be overly visual.  Pictures, video, fonts, and even audio description of the processes that we are trying to teach presuppose that we can see the content that is being referred to.  Working with ADA modifications, I have been in courses before where the instructor describes everything students are supposed to do by using phrases like “take THIS, and move it HERE and watch out for THIS because THIS is what can happen as you can plainly see…” This type of ambiguous direction given while teaching neglects learners with visual disabilities and more broadly is largely a squandered opportunity to interject meaningful descriptions that will enable instruction to begin to take on a richer, more multi-layered quality for all students.

The underlying assumption when we rely too heavily on visual content in teaching is that we are accessing the most expedient method of conveying meaning and  concepts by somehow burning the image of what we are trying to teach into the brains of our students.  But this pedagogy is flawed, we all know that visual recollection is notoriously inaccurate, eyewitness testimony is barely even admissible as evidence in court.  We forget way more than we ever remember of what we see, often almost immediately.  This is probably out of necessity, think about what happens to our computers when we never edit and never delete images, before long the hard drive fills up and they crash.  Out of necessity I think we dump most of what we see, (which is why I can’t ever find my keys).  Don’t believe me, just try and follow the simple trick that Keith Barry demonstrates at the beginning of this TED talk:


Even the stuff we really want to keep in our memories can be difficult, the faces of loved ones can be frighteningly hard to recall over time for instance, but our other senses don’t seem to have this problem.  People and places we know best we “know” by more than their appearance.  We know these things by a compilation of sounds, scents and textures which go to compose a composite identity in our mind’s eye.  It is this level of familiarity where our students really become comfortable and begin to understand the content of a given course.  It isn’t merely the face of George Washington crossing the Delaware for instance, but the feel of where this event fits within the timeline of American history and the sounds and smells of the American revolution.  This is why we went on field trips as kids, it is why we have museums and it is why we must do better in online instruction.  Real understanding is not burned in through the retinas, it is composed within the mind, and to gain the depth required for this, we need more than one dimension of instruction.
Percy Jackson The Lightening Thief Movie poster with Red Circle and SlashNot only do we know when Hollywood gets it right, we know when it gets it wrong as well.   For my kids and I, in addition to Harry Potter, we were fully on board for the Lord of the Rings movies and enjoyed the Hunger Games movies where the pieces clicked seamlessly within the worlds that we had constructed within our minds.  Something about the Narnia movies however, just didn’t feel right.  Something was missing and the series lagged.  Additionally, I was amazed at the level of disappointment in my oldest daughter in 2010 when her 9-year-old-self displayed a degree of outrage that I would have only thought possible for an adult when she watched the Percy Jackson the Lightening Thief movie.  Quite simply they had had gotten it all wrong.  Despite never having actually seen, heard or touched any part of Camp Half-Blood, she know immediately that what she was seeing on the screen was not it, she along with millions of other had constructed an intricate world in their minds that did not appear for them on the screen.

There may be hope in these successes and failures of Hollywood for us as online educators.  The ability to command the “theater of the mind” as Orson Welles described it or Dan Carlin crafts it in his Hardcore History Podcast, means that our content online can be deeper, richer and more engaging now than ever before when we were limited by time and location in the face-to-face environment.  We have be creative in new and different ways, we need to construct content that really conveys what we are trying impart to our students in multiple ways.  This is not just a concern for meeting ADA compliance, but it is a level of understanding that infuses the construction of good quality content with multiple modes of learning and provides context for this content within a wider world of knowledge.  When we work to increase the level of quality in our online classes for all students, the gaps to meet ADA compliance begin to shrink.  By engaging students in a multi-sensory way we are allowing them to plot the three dimensional points within the theater of their mind, slowly rendering a thorough working model  of a mathematical concept, historical event or biological cycle.

This is really where we are trying to get in the end anyway isn’t it?  A picture that is not only visual, but mental.  One we can turn and move in our mind, change, adapt, and apply to other content that we’ve learned.  These  conceptions that we help our students construct empower them with a sense of ownership over the content, and this ownership is central to their learning.

In speaking of the discipline of mathematics, Richard Feynman said:

quote-to-those-who-do-not-know-mathematics-it-is-difficult-to-get-across-a-real-feeling-as-richard-p-feynman-51-33-49

“To those who do not know mathematics it is difficult to get across a real feeling as to the beauty, the deepest beauty, of nature…If you want to learn about nature, to appreciate nature, it is necessary to understand the language that she speaks in.” — Richard P. Feynman

Understanding content from multiple perspectives is one of the distinctions that defines the role of the teacher from that of simply a content expert.  It is not incumbent upon the the content expert to understand how to teach another person a given subject; but to deconstruct content or a lesson into its essential parts, to work through not just how something works, but how to EXPLAIN how something works, this falls squarely into the arena of the instructor. To build this understanding, we often try to relate the new to the old, to connect what is already understood with what is being introduced.  If we are not making these connections in a multi-sensory way, then by definition, our lesson and content is shallow, often ineffective and certainly not ADA compliant.

A simple example of this multi-dimensional view of instruction is evident in the Youtube videos I routinely watch on how to repair my car.  Rarely do I watch the video and then go out an make the repair without coming back several times during the process to rewatch sections of the video.  The first viewing is just that, a viewing.  I have some prior knowledge to relate to, but typically the majority of the content is new to me.  As I begin to make the repair though, I am applying a 2 dimensional video in my mind to a 3 dimensional vehicle, and with this process I am taking in the metallic smell of the oil and gas from the engine, the feel of the wires, bolts, parts and filters and I apply these pieces of information to my viewing of the video the successive times that I view it.  I am building the car engine in my mind, I am beginning to be able to see more in the video than what is there on the 2 dimensional screen.  I am taking the image from the video and turning it in my mind searching for answers to my questions, listening for descriptions of parts of the process that I can’t see from the angle of the camera and thinking critically about the content that is being presented.  I am learning.

Screen Shot 2016-05-19 at 4.45.53 PM

Screenshot of my latest project, replacing the radiator in my car

Similarly, when I was in high school I had a very unique experience to have a teammate on our wrestling team who was blind.  As you can imagine wrestling is a challenging sport with or without disabilities, but the amount of knowledge that I was able to acquire about wrestling by having the perspective of a blind teammate was invaluable.  Frequently our coach would pull out the blindfolds an we would all wrestle “blind” in practice.  As you can imagine, you quickly gain a sense of how reliant we are on visual cues in wrestling when you put on the blindfold and step on the mat, and  you quickly realize just how limiting these cues can be.  It turns out that wrestling has as much if not more to do with what you can’t see as with what you can.  Body position, hand-control, breathing these are all things you become much more aware of when you can’t see.  As you work through the uncomfortableness of not having sight, you begin to become much more aware of your other senses, your surroundings, the position of your opponent, the position of your own body, your balance.  You begin to construct the match within your mind, which is a much more malleable format for creativity, problem solving and understanding.  In short you stop reacting to scenarios and you begin to learn to wrestle.

When we begin to make these deeper types of associations, we are linking to prior knowledge that has already made it past many of the filters to learning that we have in place.  We have on some level already burned the neural pathways and are working to connect the new knowledge into a larger matrix of understanding which we use to move in and interpret the world.  When we access this material and build upon it we are not only engaging the visual cortex but the smells, textures and “feel” of the material.  This I believe is the first step for a lot of online instructors in the journey toward ADA compliance.  Adding depth to online content creates space for accessibility.  Adding descriptions to actions, captions to video, sound to readings are all ways of working to access the brains of our students in complex ways to create recognition, understanding and comprehension.  Well designed curriculum doesn’t just help students with disabilities, it helps everyone.

 

 

 

Standard
analytics, data, data analytics, edtech, education, eLearning, Moodle, online education, online learning, Uncategorized

Using Excel to Analyze Data in Moodle to Better Understand our Students

If you know the enemy and know yourself, you need not fear the result of a hundred battles. If you know yourself but not the enemy, for every victory gained you will also suffer a defeat. If you know neither the enemy nor yourself, you will succumb in every battle.— Sun Tzu

The intrinsic value of understanding is something that has been perceived since ancient times; understanding of our enemies, understandings of ourselves and in this context understanding our students (who I would like to categorically emphasize are not our enemies, but who I do think I could benefit from understanding better in my online classes).  If we understand our students, if we can locate them, if we know “where they are” within the flat 2 dimensional space of our online courses, then we can begin to intelligently engage in a way that creates the sense of “presence” that can be so elusive to capture in an online courses.

avatar_landscapeI believe that this unquantifiable sense of “presence” is something worth pursuing.  As a species we anthropomorphize everything, we name hurricanes, we see a face in the moon, we put sweaters on our pets.  We look for connections, for faces everywhere, what if the same is true for the students in our online classes?  What if the same level of connectedness that gave audiences the “Avatar blues” back in 2010 from watching James Cameron’s “Avatar” movie where the vibrant reality of the Pandora moon so pulled them in that their actual reality seemed to pale in comparison could be harnessed to produce a more dynamic, vibrant, responsive learning environment where students experience the person within the machine?  What impact might this have on their success and ability to persist in a difficult course?

CooperEvery semester we have countless online students come to our physical campus to ask for help with some part of their classes.  Often we get the comment that ultimately the problem wasn’t one that couldn’t be solved remotely, but more that they felt that they couldn’t do it “alone”.  To examine this concept of being “alone” I want to quickly indulge in one additional movie reference, for any who have seen the movie “Interstellar” you might recall the climax of the movie involves Cooper the main character of the movie played by Matthew McConaughey trapped 5th Dimensionally in the finite space of his daughter’s bedroom but with infinite time.  The beings who placed him in this space were trying to reach out and make contact, but even with the technology to manipulate infinite time they lacked the ability to make a single real connection within any single unit of time.  They brought Cooper to this tesseract, this 5th Dimensional space because they believed that his personal connection to his daughter would help guide him to the “present” moment where the changes necessary to save the human race could be made.

With online classes in some ways we have created a similar tesseract of 4 Dimensional space.  Our students are all present within the space of a single “shell” of an online course, but log-in and interact with an infinite possibilities of time, some first thing in the morning, some at lunch, some after they put the kids to bed and some burn the midnight oil, some Mondays and Wednesdays, some on Tuesday and Thursdays and some only on the weekends.  Analytics are just now beginning to give us the tools as instructors to make the connections within this space to use synchronous technologies including anything from the simple rapid response email or forum activity that you get when someone else is online the same time as you are to Instant Messaging, web conferencing and other Web 2.0 technologies that are bringing the world closer together every day to connect in real-time to our online students and I believe real-time interventions can impact the overall success, completion and persistence of our students.

On of the benefits of moving to Moodle 2.7 has been the availability of the new improved logging features.  The question remains though on how best to make use of this available data.  In order to ensure student success interventions need not only be accurate in their assessments, but also timely.  In a separate post I will expand more on our recent use of the Moodle Engagement Analytic to better track student success within a course and to provide real-time interventions to improve success and completion.  For this post however I want to focus on the use of student log-in data to target optimum times to allocate availability for online office hours, tutoring etc…  Additionally I would like to propose some rudimentary methods for identifying clustering and wave patterns to better understand the nuanced engagement patterns that students have within specific courses, and propose some potential drivers for student engagement.

First the process for extracting data:

  1.  1.  Within Moodle navigate within the Administration block to Logs under Reports:Logs

2.  This will gain you access to the logging data for your individual course within Moodle.  This is the first stage where you will need to consider exactly what it is that you are searching for in order to best select your data.  Since the option to select a relevant date range is not provided, I generally stick with the defaults and go ahead and pull all users for all dates:

 

Relevant Range for Student Data3.  Depending on when within the semester you pull your data, this could end up being a pretty large file.  The download options (located at the bottom of the first page of data)  include .csv, .tsv, Excel and Open formats.

Moodle Data Download Options

 

 

4.  Once your file is downloaded (remember .csv can handle more files than Excel) you will need to pull the data into a spreadsheet package that supports Pivot Tables.  For this article I will be using Excel.  The first column will be labeled Time, and we will need to break the data apart in order to filter for activity by day of the week, period the semester as well as time of day.  Below is an example of the default Time Column as well as the columns used to extract the data.

You will want to Insert a blank column beside the column that you are splitting, in this case, you would right-click on Column to the right of the column you ares preparing to expand.  For instance when splitting the “Time” column below, you would first right click on Column “B” and select “Insert” from the drop down menu.

Insert column ExcelExcel Time

Text to Column Button

 

 

 

 

 

 

As you progress through the “Text to Columns” wizard, Select “Delimited” and “Next” and then choose your delimiting factor, in this case it would be the comma separating the Date from the Time, but in other instances you might need to enter a custom delimiter, such as a colon when you want to separate the hour from the minutes if you wanted to know what hours of the day your students are most active.

DelimitedComma

Date

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Once you have completed the “Text to Columns” and your columns are set correctly you will want to click on the “Insert” tab and select “Pivot Chart”

pivot-chart

In the data selection window, select all data you might want to analyze(you can do this by selecting the complete column), remember to give your new columns a heading or they will invalid for use in a Pivot Chart.  Go ahead and select the option to open your chart in a new sheet, this keeps things more organized in the event that you want to create a second pivot chart.

You are now ready to do some basic analysis of the data.

By dragging a column header into the Axis Category as well as the Values category.  This will automatically generate a “count” which can be analyzed.  Below is an example of a course where the “Text to Columns” feature mentioned above was used to identify the hours of the day that students accessed the course over the semester.  As you can see from the wave pattern within an average day there on average is a “lunch time” and “evening” spike in activity.

Pivot table axis and values

Here you can see where the the activity is being analyzed for all dates of the semester and increased activity in this instance can be correlated with the Due Dates of assignments.

Spikes correspond with assignment due dates

In this case the pattern was less predicable, but it turns out a correlation was able to be drawn not only between increases in student activity around assignment due dates, but also clustering of student activity seemed to mirror instructor activity within the forums:

Mirror Effect

In this example student activity has been filtered to identify the most active days of the week:

Pivot Chart filter

Finally the activity patterns and behavior of specific students can be analyzed.  Below is an aggregate view of an entire class, but as you can see in the example above certain students can be filtered out and compared based on grade performance, competency scores etc…

Students sorted by contact with course

As you can see these are just a few examples of some fairly simple but potentially powerful analytics to help discover a more informed and nuanced understanding of how your students are interacting and performing within your online course.  This type of analysis can be used to confirm assumptions, dispel incorrect biases and potentially lead to changes in design and delivery of online content to help achieve higher rates of student success and completion within a course.

Why Does it Matter?

The core question of “”why does any of this matter?” is broad and I have a few specific scenarios of where this type of analysis might immediately be useful, but at a more meta level to paraphrase Socrates,  the unexamined course may not be worth teaching.

A specific example of
where this type of analysis might matter would be the direct comparison of the levels of activity over time and behavioral patterns of students who earn an “A” in a course with the activity and behavioral patterns of students who ultimately withdraw or fail the course.  Questions about a correlation between cumulative time spent in a course, or frequency of interactions within a week, or high levels of activity early within a course and success could be investigated.  If correlations can be detected we can investigate if they appear to be persistent  among courses, schools etc…

If we can find a correlation between frequency of contact with a course throughout a week or the semester and success, then an additional analysis of our courses might be to search for a sort of “mirror” effect where our general activity within a course, or specific types of activity as instructors might in fact have some effect in real-time or near real-time in influences the activity of our students.  If I believe that increasing the frequency of interaction within a forum throughout a given week helps students remain more engaged and produces better quality discussions and there is a correlation between their activity and the relative levels of my participation in a forum as the instructor this knowledge could empower an instructor to make adjustments in their own interaction and activity to achieving the results in student activity that that they are after.

Without this type of analysis it is left to the instructor’s judgement to determine the levels of student interaction, if we hold that it is possible for the instructor to participate too much and shut down conversation, or participate too little where students feel that no one is really there and they disengage, then objective quantifiable analysis of student engagement over time becomes a very important metric that has largely been relegated to a very subjective tool.  Perhaps even more importantly, questions about what types of activity cause students to perceive that we are present and engaged in the course could be very valuable and powerful information in helping faculty that are many times pulled in too many directions already to be more effective in their efforts and more efficient as well.  In short this type of analysis might could give us a real picture of how our interaction is in fact affecting our students.

Besides the influence of our perceived participation within a course, the basic instructional design of a course may also have an enormous affect on the activity and engagement of students within a course.  If in fact the setting of Due Dates has a quantifiable influence over the activity of students within a course, then by changing the scheduling of assignments each week and over the course of the semester we  might be affecting the rates of success and completion within our courses.  For instance I found clusters of activity where students were logging-in and interacting within the course doubled once I moved from a single due date on discussions forums on Sunday nights to a discussion format where students needed to post their initial post was due by Thursdays and all posts and responses were due by Sunday.  I don’t know if yet if there is a measurable affect of this type of doubling of scheduled activity within a course on success and completion, but with this type of comparative analysis this might can now be measured.

A final example of potential uses for this type of analysis would might would result from detecting a reliable widespread pattern on logging in around lunchtime each day and again at night.  Instructionally these might be times that we would want to make ourselves available for online office hours, tutoring, meeting in small groups etc…  Additionally we might consider chunking our instructional material and assignments to explicitly accommodate this pattern of interaction.  I believe that it is probable that students are wasting time and energy in replicated efforts, beginning assignments they are unable to finish during lunch only to do the same assignment again at night, choosing to tackle complicated material at lunch only to find that they have left the more simple content to work on at night and so forth.  It would be interesting to attempt to match the chunking of material into a smaller more manageable piece of instruction designed to be done during a lunch break and a larger piece of content designed to be done in the evenings.  Work done during lunch would need to be easily saved for when students inevitably get interrupted so that they can come back to where they left off later in the evening and content could be stacked with intro level content to be interacted with during the day when time is a more of a premium and distractions are more present and the more complex content reserved for evenings when it is easier to concentrate for longer periods of time uninterrupted.

These are just a few examples, in the end these correlations might prove to be false, but I believe that ultimately this type of analysis helps us to understand our students, and that in and of itself holds considerable value.  I believe we are entering into a new era of online learning where we can now not only construct a classroom that allows students to escape some of the space-time limitations that we have historically placed on higher education (you have to be in the same physical space as the instructor at the same time) but with increasing analytic tools, the proliferation of real-time communications and 24/7 access to the Internet we are beginning to reliably have the ability to connect with students within this new space in new and meaningful ways.

Standard
Uncategorized

Hacking at the Scroll of Death in Moodle

Moodle Scroll of Death

As an instructor for online courses in Moodle, the scroll of death has long been an annoyance.  More recently as an administrator addressing the needs ADA students using screen readers the scroll of death has begun to chafe even more as it forces students to navigate all their previous content in order to access their current assignments and material.  Additionally it has always just seemed like poor design to not present the most recent content in the most easily accessed area of the course at the top of the page. visibility hidden

In an effort to alleviate this pain, and to help ensure that all students are aware when new content is released each by allowing the content to appear in the primary display when students access the course, I began to move the new topic week to the top of the course as it was released from week to week.  This process is a little bit of a pain but it does spare students from navigating the scroll of death every time they work in the class.  Still I have found that it can cause confusion for some students (particularly those who fall behind), and the process ends up leaving me with a course where all of my content is in exactly the wrong order to import to the next semester.

So in this past week I began to think about a student that I worked with who required the use of a screen reader.  They had installed the Moodle App (which I am becoming a fan of using) on their Ipad  and they were using the app interface rather than the web interface for their class in Moodle because the app was concise and easy to navigate.  I also began to realize that everything on the ipad is concise and easy to navigate, specifically with the use of labeled pictorial icons.  I started to think about what Moodle would be like if instead of scrolling to navigate from topic week to topic week, we could navigate Moodle in the same way.

A quick Google search revealed that this functionality does exist with the Grid Format which you can check out here if you’re interested: https://moodle.org/plugins/view/format_grid  The problem I have with installing this option on our Moodle server is that as a self-hosted institution we try to keep custom modifications to a minimum and from what I can tell of the plug-in, the interface involves a pop-up type of interface, which is nice but can give problems for students and at this point seems like it would add unnecessary complications.

As an alternative I started working with the options under:

Edit settings  Administration –> Edit Settings

Under Course format select the dropdown menu for Course layout and select Show one section per page.

This will set your course to only display one Topic at a time.  One Topic per PageThe benefits of this approach are that all of your content can fit on one page.  Theoretically your students would never have to scroll, but the trick has always been how to navigate from section to section, and how to identify which content is current.

Jump toThe default navigation for this view is via the Jump to… dropbox at the bottom of the page and with tab navigation (think Moodle Books) on the left and right of the screen.  The Jump to… tool while effective is almost too subtle to be reliably located and used and it can’t be configured to indicate which material is currently assigned or graphically give any context as to what the unit might be about.  The tab on the other hand is pretty fabulous.  The great part about both navigation tools is that they will not link to Topic Weeks that you have not set to be visible under course settings.  This is important to remember depending on how you control the release of your content.

Tab Navigation

The great benefit of the Jump to… tool is the ability to pull the direct link to view each section individually.  Essentially the URL looks like this:   https://moodle.yourserver.edu/course/view.php?id=course#&section=section# 

This code is important because it basically allows you to create icons that will allow students to navigate to the section or topic week of their choice.  It is then pretty simple to insert a Label resource in the header of your Moodle course and then within the WYSIWYG editor select the insert a table icon:

Insert a table

Under the General Properties tab I set my table to 8 Columns (I have 7 Topic Weeks and a section for Getting Started material) 2 Rows (the second row I use for labeling) I align the table to the center and I set the cell padding and spacing at 5 just so things don’t look too crowdeTable General Propertiesd.Within each cell I insert a picture that relates to each particular topic week set to a standard size, I usually go for 100×100 and be sure to label the ALT image for screenreaders.  I then link the image to it’s appropriate section URL.  On the second row I include labels for each image you will also want to link these as they are easier for screenreaders to navigate.Populate Table

A final challenge is devising a method to only link to active sessions.  Remember the native navigation tools won’t link to sections that you have not released in your course settings, but your shortcuts will which can present a problem.  There are two methods that I have come up with so far:

The first one is a little tricky.  Within the HTML for the table you can insert the code for visibility:hidden within the style for each cell, this will remove the cell completely from view until you want to release the content.  It is a little bit of a pain, but honestly not much different than constantly moving new topic weeks to the top of the page in my opinion.  You can see in the example that Section 7 and 8 have been hidden.

visibility hiddenThe second option is to go ahead display all icons in the table and control the release of content by simply hiding it in weeks that you have not released yet.  This is much easier and probably the approach I will go with in the future.

With either method, you still need to indicate the current week’s material, and if you are going with the second option on controlling content you will need a visible way to differentiate content that is available from content that has not yet been released.  This can be controlled pretty easily within the WYSIWYG editor with controlling the coloring and highlight of the current topic.  For ADA compliance additional labeling indicating “current week” and “not available” should also be used.Course layout

I am still piloting this look, but the end result should be a much simpler and aesthetically pleasing interface allowing all students particularly those using assistive devices to navigate their course in Moodle without having to endlessly scroll to find their content.

Standard
edtech, education, eLearning, Grading, Marking Guide, Moodle, online education, online learning, Rubrics, Uncategorized

Grading Smarter not Harder with Rubrics and Marking Guides in Moodle

This month I have had the privilege to work with a group of new instructors as well as our adjunct instructors in an Online Instructor Certification course based in Moodle.  One particular area that has surfaced in our training that I want to highlight in this article deals with delivering informative, targeted feedback on assignments. Within Moodle there are a number of ways to go above and beyond the Moodle default of “simple direct grading” and incorporate Rubrics and/or Marking Guides.

In the image below, within the “Grade” category there is a drop down menu. The default value is “Simple direct grading”, but by clicking on the drop down menu you will also see options for “Marking guide” and “Rubric”.

select rubric

With either option, you are prompted to choose either to build your “Marking guide” or “Rubric” from scratch, or create a new form from a template:

Rubric build from scratch

Assuming you don’t have a template (I didn’t) you will want to choose the first option. At this point the layout changes slightly depending on whether you have selected a “Marking guide” or “Rubric”.

Rubrics:

If you are familiar with Rubrics, or even have one that you are currently using offline, then the online rubric builder is fairly self explanatory with the familiar grid layout, and a default of three criteria. In the image below I have selected as example criteria: Grammar, Spelling and Content. You can easily edit these to fit your purposes, as well as edit the levels of proficiency and the value of points for each grading criteria.

Rubric setup

Once the Rubric is calibrated to your liking, then grading of student work consists of simply “clicking” on the appropriate box for each criteria (further “Rubric Options” are pictured above as well, these are pretty self-explanatory, but probably something you should take a moment to think through, particularly if you are setting this course up for another instructor.) Additional feedback comments can be added as well in the column on the far right when grading an assignment as pictured below:

rubric interface

Marking Guide:

I personally feel a Marking Guide is the Minimum Effective Dose of a grading rubric. Much like the rubric it provides grade categorization to add meaning and context to a graded assignment for students. Similar to a Rubric, maximum values can be set for each grading criteria to help ensure that the amount deducted for a particular criteria doesn’t go beyond a preset ceiling allowing a single criteria to overrun the overall grading scheme. What is not present are the varied levels of assessment, this makes the Marking Guide easier to build and in my opinion more than adequate for most assignments, particularly where the instructor building the course is also the one teaching the course and assessing work submitted.

In addition to the targeted criteria and scoring, an additional space allotted to provide comments and additional feedback. With the Marking Guide, customized comments can be pre-programmed to easily provide your most frequent feedback.

Unnamed image (1)

Benefits and Upcoming Features:

Currently the advanced grading functionality is not available for forums, but by disabling grading in forums and creating an “offline assignment” in the gradebook where a rubric can be set-up,forums can be graded in separate tabs or screens by reading forums on one and grading the offline assignment with the rubric on the other. There is some indication that Moodle will bring Advanced Grading to discussion forums in the near future, but in the meantime I believe this would be preferable to scoring rubrics by hand on either a separate sheet of paper or a Word document to be later be uploaded.

The benefit of these approaches is a more informed and empowered student, particularly if the assignment is one in which corrections can be made and the assignment resubmitted.  Additionally, these LMS-based tools also help faculty to avoid downloading documents and/or keeping duplicate records on laptops or PCs and the use of the “save and show next” button replaces the need for scoring on a separate document to upload for each student resulting in a more efficient workflow for the instructor.

Standard
ADA, edtech, education, eLearning, modification, Moodle, online education, online learning

Using User Overrides in Moodle to provide Testing Modifications for Quizzes and Assignments

As a new semester begins, I thought I would post on some great features in Moodle for providing individualized modifications for a few of the most common accommodations that we encounter in higher education.

Not so many years ago providing an accommodation in an online course such as extended time, multiple attempts etc…involved a complex process of either generating multiple copies of a single assessment or releasing the assessment with full modifications enabled and then limiting the the terms of access of all students except the one with the modification.  Ultimately this approach was limited in scale and efficiency.

For Quizzes:

For the past couple of years since migrating to Moodle 2 we have had access to the “User Override” option, but I find that frequently instructors are unaware of these options.

To access the User Override click on a quiz you have created and then select User Overrides

Unnamed image (1)

Within the Override window you can select individual students and assign unique values for open and close dates, the amount of time allowed and the number of attempts allowed.  These changes only affect the selected student and all grades both for the student with accommodations as well as those without appear within the gradebook under the same entry.

Unnamed image (2)

For Assignments:

Within assignments a similar functionality is found within the “grant extension” functionality.  To locate this option, click on the assignment you wish to grant an extension and then click on “View/grade all submissions” for the particular student who requires the modification.

Unnamed image (3)

Within this drop down menu you will see the options for individual modifications including both the option to grant an extension as well as allow another attempt.

Unnamed image (4)

When granting an extension you will see the option to set an individually modified due date, be sure to check the “enable” box before attempting to set an alternate due date.

Unnamed image (5)

Unnamed image (6)To Allow another attempt, simply click on this option from the dropdown menu.  You will see confirmation of both modifications appear within the “Status” column within the “View/Grade” screen.  “Reopened” is confirmation of another attempt being allowed and “Extension granted” message is confirmation that accompanies the newly effective due date for that particular student.

At present these are the only two activities within Moodle that I am aware of with individual modification functionality, but I would love to learn of more, please feel free to leave comments below.  I am glad to see options for individualizing learning expanding both in modifications as well as adaptive release functionalities which I hope to post about in the near future.

Standard