“If you know the enemy and know yourself, you need not fear the result of a hundred battles. If you know yourself but not the enemy, for every victory gained you will also suffer a defeat. If you know neither the enemy nor yourself, you will succumb in every battle.” — Sun Tzu
The intrinsic value of understanding is something that has been perceived since ancient times; understanding of our enemies, understandings of ourselves and in this context understanding our students (who I would like to categorically emphasize are not our enemies, but who I do think I could benefit from understanding better in my online classes). If we understand our students, if we can locate them, if we know “where they are” within the flat 2 dimensional space of our online courses, then we can begin to intelligently engage in a way that creates the sense of “presence” that can be so elusive to capture in an online courses.
I believe that this unquantifiable sense of “presence” is something worth pursuing. As a species we anthropomorphize everything, we name hurricanes, we see a face in the moon, we put sweaters on our pets. We look for connections, for faces everywhere, what if the same is true for the students in our online classes? What if the same level of connectedness that gave audiences the “Avatar blues” back in 2010 from watching James Cameron’s “Avatar” movie where the vibrant reality of the Pandora moon so pulled them in that their actual reality seemed to pale in comparison could be harnessed to produce a more dynamic, vibrant, responsive learning environment where students experience the person within the machine? What impact might this have on their success and ability to persist in a difficult course?
Every semester we have countless online students come to our physical campus to ask for help with some part of their classes. Often we get the comment that ultimately the problem wasn’t one that couldn’t be solved remotely, but more that they felt that they couldn’t do it “alone”. To examine this concept of being “alone” I want to quickly indulge in one additional movie reference, for any who have seen the movie “Interstellar” you might recall the climax of the movie involves Cooper the main character of the movie played by Matthew McConaughey trapped 5th Dimensionally in the finite space of his daughter’s bedroom but with infinite time. The beings who placed him in this space were trying to reach out and make contact, but even with the technology to manipulate infinite time they lacked the ability to make a single real connection within any single unit of time. They brought Cooper to this tesseract, this 5th Dimensional space because they believed that his personal connection to his daughter would help guide him to the “present” moment where the changes necessary to save the human race could be made.
With online classes in some ways we have created a similar tesseract of 4 Dimensional space. Our students are all present within the space of a single “shell” of an online course, but log-in and interact with an infinite possibilities of time, some first thing in the morning, some at lunch, some after they put the kids to bed and some burn the midnight oil, some Mondays and Wednesdays, some on Tuesday and Thursdays and some only on the weekends. Analytics are just now beginning to give us the tools as instructors to make the connections within this space to use synchronous technologies including anything from the simple rapid response email or forum activity that you get when someone else is online the same time as you are to Instant Messaging, web conferencing and other Web 2.0 technologies that are bringing the world closer together every day to connect in real-time to our online students and I believe real-time interventions can impact the overall success, completion and persistence of our students.
On of the benefits of moving to Moodle 2.7 has been the availability of the new improved logging features. The question remains though on how best to make use of this available data. In order to ensure student success interventions need not only be accurate in their assessments, but also timely. In a separate post I will expand more on our recent use of the Moodle Engagement Analytic to better track student success within a course and to provide real-time interventions to improve success and completion. For this post however I want to focus on the use of student log-in data to target optimum times to allocate availability for online office hours, tutoring etc… Additionally I would like to propose some rudimentary methods for identifying clustering and wave patterns to better understand the nuanced engagement patterns that students have within specific courses, and propose some potential drivers for student engagement.
First the process for extracting data:
- 1. Within Moodle navigate within the Administration block to Logs under Reports:
2. This will gain you access to the logging data for your individual course within Moodle. This is the first stage where you will need to consider exactly what it is that you are searching for in order to best select your data. Since the option to select a relevant date range is not provided, I generally stick with the defaults and go ahead and pull all users for all dates:
3. Depending on when within the semester you pull your data, this could end up being a pretty large file. The download options (located at the bottom of the first page of data) include .csv, .tsv, Excel and Open formats.
4. Once your file is downloaded (remember .csv can handle more files than Excel) you will need to pull the data into a spreadsheet package that supports Pivot Tables. For this article I will be using Excel. The first column will be labeled Time, and we will need to break the data apart in order to filter for activity by day of the week, period the semester as well as time of day. Below is an example of the default Time Column as well as the columns used to extract the data.
You will want to Insert a blank column beside the column that you are splitting, in this case, you would right-click on Column to the right of the column you ares preparing to expand. For instance when splitting the “Time” column below, you would first right click on Column “B” and select “Insert” from the drop down menu.
As you progress through the “Text to Columns” wizard, Select “Delimited” and “Next” and then choose your delimiting factor, in this case it would be the comma separating the Date from the Time, but in other instances you might need to enter a custom delimiter, such as a colon when you want to separate the hour from the minutes if you wanted to know what hours of the day your students are most active.
Once you have completed the “Text to Columns” and your columns are set correctly you will want to click on the “Insert” tab and select “Pivot Chart”
In the data selection window, select all data you might want to analyze(you can do this by selecting the complete column), remember to give your new columns a heading or they will invalid for use in a Pivot Chart. Go ahead and select the option to open your chart in a new sheet, this keeps things more organized in the event that you want to create a second pivot chart.
You are now ready to do some basic analysis of the data.
By dragging a column header into the Axis Category as well as the Values category. This will automatically generate a “count” which can be analyzed. Below is an example of a course where the “Text to Columns” feature mentioned above was used to identify the hours of the day that students accessed the course over the semester. As you can see from the wave pattern within an average day there on average is a “lunch time” and “evening” spike in activity.
Here you can see where the the activity is being analyzed for all dates of the semester and increased activity in this instance can be correlated with the Due Dates of assignments.
In this case the pattern was less predicable, but it turns out a correlation was able to be drawn not only between increases in student activity around assignment due dates, but also clustering of student activity seemed to mirror instructor activity within the forums:
In this example student activity has been filtered to identify the most active days of the week:
Finally the activity patterns and behavior of specific students can be analyzed. Below is an aggregate view of an entire class, but as you can see in the example above certain students can be filtered out and compared based on grade performance, competency scores etc…
As you can see these are just a few examples of some fairly simple but potentially powerful analytics to help discover a more informed and nuanced understanding of how your students are interacting and performing within your online course. This type of analysis can be used to confirm assumptions, dispel incorrect biases and potentially lead to changes in design and delivery of online content to help achieve higher rates of student success and completion within a course.
Why Does it Matter?
The core question of “”why does any of this matter?” is broad and I have a few specific scenarios of where this type of analysis might immediately be useful, but at a more meta level to paraphrase Socrates, the unexamined course may not be worth teaching.
A specific example of
where this type of analysis might matter would be the direct comparison of the levels of activity over time and behavioral patterns of students who earn an “A” in a course with the activity and behavioral patterns of students who ultimately withdraw or fail the course. Questions about a correlation between cumulative time spent in a course, or frequency of interactions within a week, or high levels of activity early within a course and success could be investigated. If correlations can be detected we can investigate if they appear to be persistent among courses, schools etc…
If we can find a correlation between frequency of contact with a course throughout a week or the semester and success, then an additional analysis of our courses might be to search for a sort of “mirror” effect where our general activity within a course, or specific types of activity as instructors might in fact have some effect in real-time or near real-time in influences the activity of our students. If I believe that increasing the frequency of interaction within a forum throughout a given week helps students remain more engaged and produces better quality discussions and there is a correlation between their activity and the relative levels of my participation in a forum as the instructor this knowledge could empower an instructor to make adjustments in their own interaction and activity to achieving the results in student activity that that they are after.
Without this type of analysis it is left to the instructor’s judgement to determine the levels of student interaction, if we hold that it is possible for the instructor to participate too much and shut down conversation, or participate too little where students feel that no one is really there and they disengage, then objective quantifiable analysis of student engagement over time becomes a very important metric that has largely been relegated to a very subjective tool. Perhaps even more importantly, questions about what types of activity cause students to perceive that we are present and engaged in the course could be very valuable and powerful information in helping faculty that are many times pulled in too many directions already to be more effective in their efforts and more efficient as well. In short this type of analysis might could give us a real picture of how our interaction is in fact affecting our students.
Besides the influence of our perceived participation within a course, the basic instructional design of a course may also have an enormous affect on the activity and engagement of students within a course. If in fact the setting of Due Dates has a quantifiable influence over the activity of students within a course, then by changing the scheduling of assignments each week and over the course of the semester we might be affecting the rates of success and completion within our courses. For instance I found clusters of activity where students were logging-in and interacting within the course doubled once I moved from a single due date on discussions forums on Sunday nights to a discussion format where students needed to post their initial post was due by Thursdays and all posts and responses were due by Sunday. I don’t know if yet if there is a measurable affect of this type of doubling of scheduled activity within a course on success and completion, but with this type of comparative analysis this might can now be measured.
A final example of potential uses for this type of analysis would might would result from detecting a reliable widespread pattern on logging in around lunchtime each day and again at night. Instructionally these might be times that we would want to make ourselves available for online office hours, tutoring, meeting in small groups etc… Additionally we might consider chunking our instructional material and assignments to explicitly accommodate this pattern of interaction. I believe that it is probable that students are wasting time and energy in replicated efforts, beginning assignments they are unable to finish during lunch only to do the same assignment again at night, choosing to tackle complicated material at lunch only to find that they have left the more simple content to work on at night and so forth. It would be interesting to attempt to match the chunking of material into a smaller more manageable piece of instruction designed to be done during a lunch break and a larger piece of content designed to be done in the evenings. Work done during lunch would need to be easily saved for when students inevitably get interrupted so that they can come back to where they left off later in the evening and content could be stacked with intro level content to be interacted with during the day when time is a more of a premium and distractions are more present and the more complex content reserved for evenings when it is easier to concentrate for longer periods of time uninterrupted.
These are just a few examples, in the end these correlations might prove to be false, but I believe that ultimately this type of analysis helps us to understand our students, and that in and of itself holds considerable value. I believe we are entering into a new era of online learning where we can now not only construct a classroom that allows students to escape some of the space-time limitations that we have historically placed on higher education (you have to be in the same physical space as the instructor at the same time) but with increasing analytic tools, the proliferation of real-time communications and 24/7 access to the Internet we are beginning to reliably have the ability to connect with students within this new space in new and meaningful ways.