 One of the great benefits of online instruction is the ability it affords to track the activities and behavior of the students on the virtual campus. By identifying the characteristics of “successful” students and the metrics that measure those characteristics, we can determine which student conditions, activities, and behaviors best predict student success. But even those institutions with the most advanced online learning programs are only just beginning to realize the untapped potential of their student and faculty data. This type of critical inquiry requires a thoughtful design by experienced researchers. Here are some suggestions to get you started.
Student Success At the foundation of nearly all institutional research is the goal of student success. How does your institution define student success? Some common definers include student performance (formative and summative grades), retention, class standing, progression toward degree, satisfaction, etc. The definition for your institution may include degrees of all of these elements. However, the definition may be nuanced and the weighting of these elements is likely to vary greatly from one institution to another. It is important to build a consensus about what the institutional definition of student success is before determining the processes and selecting the metrics to measure it. Choosing Data Sources Once you have a definition for student success, you should take inventory of the data sources available for analysis. You will likely need to input from several departments within your college or university to complete a comprehensive account of the available sources. Consider including the owners or “keepers” of areas such as your learning technologies, learning content, academic information systems, enterprise resource planning systems, etc. You may need multiple levels of expertise for some of these systems, e.g., content knowledge, information technology knowledge, policy knowledge, and more. Now consider what specific sources and systems will be included in your inventory. Typical locations include the Learning Management System, Student Information System, Customer Relations Manager, Enterprise Resource Planning, Education Management Information Software, etc. This will likely begin as a relatively small list. However, the inventory must also include field level data from the sources; these are the actual metrics that will be used in the analysis, either as direct input OR as data for generating input. There are classifications of student data for consideration in your analysis. Consider the following: • Student Conditions: Student conditions are those student metrics that occur outside the classroom. Common student conditions include demographics, academic programs, previous performance, class status/standing, etc. • Student Activities: Student activities are the actions that students take in your virtual campus and online course. Some examples of student activities include the time spent in course (total time, average time per day, average time per session, etc.), time spent in various content areas, performance in course to date, when assignments are completed, etc. • Student Behavior: Some research efforts elect to separate student activities and student behavior. Student behavior examines the “how” of specific student activities. How do students elect to consume instructional content and complete their assignments? What are the sequences they use to complete necessary academic activities? In many instances, how students choose to work online can tell us as much or even more than the actual tasks they have completed. Analysis and Interpretation The processes for analysis and interpretation of your data will vary widely depending on your research questions, available sources, the metrics that you chose, and the output you desire. It is likely that you will want a descriptive statistical report for the key variables of your student population. These may include metrics describing distributions, tendencies, and dispersions. It is also quite likely your analysis will include inferential statistical models. These types of analyses examine the relationship between two or more variables. For example, what is the predictive value of time spent in course content to the final grade in that course? Once again, the quantitative models you select will be a product of the nature of your research and metrics you have decided to use. The interpretation of your statistical output is normally completed in a small team with senior members of academic administration, subject or content experts, the researchers, and experts in relevant statistical areas. The key to quality interpretation is determining a consensus based on the data and articulating that in “actionable” terms. By actionable, we mean a clear and obvious path to change based on the output of the data. Conclusion One of the reasons that colleges and universities have been reluctant to pursue significant research using their online student data is the perception of the complexities and difficulties associated with it. While research of student conditions, activities and behavior is not a trivial thing, it is certainly a realistic goal worth pursuing. Knowing what student actions are likely to predict factors of success such as strong academic performance, retention, persistence, and completion of degrees are clearly invaluable strategic factors. JenEd Consulting has a great deal of experience in the entire research lifecycle. We can help you articulate your research questions, identify key data sources, prepare the data for analysis, select the appropriate research models and methods and complete actionable interpretation of the output. Dr. Rob Sapp Senior Associate rob@jenedconsulting.com Welcome to the JenEd Consulting team, Stacey!
Stacey Garrett Koju brings over 20 years of experience in higher education, healthcare, and government affairs to her work at JenEd Consulting. As a former vice president for online programs, Stacey combines extensive legal expertise with a practical understanding of the contractual and regulatory challenges facing colleges and universities operating in the online space. Stacey is a founding member of Bone McAllester Norton PLLC, and represents several higher education institutions. She also served as Vice President for Online at Cumberland University in Lebanon, Tennessee and was Deputy Chief of Staff for U.S. Congressman Harold Ford, Jr. of Memphis. A graduate of Duke University (B.A.), Stacey received her J.D. from Tulane University Law School and is a member of the Tennessee Bar Association. We're pleased to be a Patron Sponsor at the Appalachian College Association's 2016 Annual Meeting of Presidents and Chief Academic Officers (June 6-7). Come by our table to say hello or sign up for a pro-bono 30-minute consultation regarding your online strategy (john@jenedconsulting.com).
Dr. John E. Neal, President |
RSS Feed