One of the great benefits of online instruction is the ability it affords to track the activities and behavior of the students on the virtual campus. By identifying the characteristics of “successful” students and the metrics that measure those characteristics, we can determine which student conditions, activities, and behaviors best predict student success. But even those institutions with the most advanced online learning programs are only just beginning to realize the untapped potential of their student and faculty data. This type of critical inquiry requires a thoughtful design by experienced researchers. Here are some suggestions to get you started.
At the foundation of nearly all institutional research is the goal of student success. How does your institution define student success? Some common definers include student performance (formative and summative grades), retention, class standing, progression toward degree, satisfaction, etc. The definition for your institution may include degrees of all of these elements. However, the definition may be nuanced and the weighting of these elements is likely to vary greatly from one institution to another. It is important to build a consensus about what the institutional definition of student success is before determining the processes and selecting the metrics to measure it.
Choosing Data Sources
Once you have a definition for student success, you should take inventory of the data sources available for analysis. You will likely need to input from several departments within your college or university to complete a comprehensive account of the available sources. Consider including the owners or “keepers” of areas such as your learning technologies, learning content, academic information systems, enterprise resource planning systems, etc. You may need multiple levels of expertise for some of these systems, e.g., content knowledge, information technology knowledge, policy knowledge, and more.
Now consider what specific sources and systems will be included in your inventory. Typical locations include the Learning Management System, Student Information System, Customer Relations Manager, Enterprise Resource Planning, Education Management Information Software, etc. This will likely begin as a relatively small list. However, the inventory must also include field level data from the sources; these are the actual metrics that will be used in the analysis, either as direct input OR as data for generating input.
There are classifications of student data for consideration in your analysis. Consider the following:
• Student Conditions: Student conditions are those student metrics that occur outside the classroom. Common student conditions include demographics, academic programs, previous performance, class status/standing, etc.
• Student Activities: Student activities are the actions that students take in your virtual campus and online course. Some examples of student activities include the time spent in course (total time, average time per day, average time per session, etc.), time spent in various content areas, performance in course to date, when assignments are completed, etc.
• Student Behavior: Some research efforts elect to separate student activities and student behavior. Student behavior examines the “how” of specific student activities. How do students elect to consume instructional content and complete their assignments? What are the sequences they use to complete necessary academic activities? In many instances, how students choose to work online can tell us as much or even more than the actual tasks they have completed.
Analysis and Interpretation
The processes for analysis and interpretation of your data will vary widely depending on your research questions, available sources, the metrics that you chose, and the output you desire. It is likely that you will want a descriptive statistical report for the key variables of your student population. These may include metrics describing distributions, tendencies, and dispersions.
It is also quite likely your analysis will include inferential statistical models. These types of analyses examine the relationship between two or more variables. For example, what is the predictive value of time spent in course content to the final grade in that course? Once again, the quantitative models you select will be a product of the nature of your research and metrics you have decided to use.
The interpretation of your statistical output is normally completed in a small team with senior members of academic administration, subject or content experts, the researchers, and experts in relevant statistical areas. The key to quality interpretation is determining a consensus based on the data and articulating that in “actionable” terms. By actionable, we mean a clear and obvious path to change based on the output of the data.
One of the reasons that colleges and universities have been reluctant to pursue significant research using their online student data is the perception of the complexities and difficulties associated with it. While research of student conditions, activities and behavior is not a trivial thing, it is certainly a realistic goal worth pursuing.
Knowing what student actions are likely to predict factors of success such as strong academic performance, retention, persistence, and completion of degrees are clearly invaluable strategic factors. JenEd Consulting has a great deal of experience in the entire research lifecycle. We can help you articulate your research questions, identify key data sources, prepare the data for analysis, select the appropriate research models and methods and complete actionable interpretation of the output.
Dr. Rob Sapp
Welcome to the JenEd Consulting team, Stacey!
Stacey Garrett Koju brings over 20 years of experience in higher education, healthcare, and government affairs to her work at JenEd Consulting. As a former vice president for online programs, Stacey combines extensive legal expertise with a practical understanding of the contractual and regulatory challenges facing colleges and universities operating in the online space.
Stacey is a founding member of Bone McAllester Norton PLLC, and represents several higher education institutions. She also served as Vice President for Online at Cumberland University in Lebanon, Tennessee and was Deputy Chief of Staff for U.S. Congressman Harold Ford, Jr. of Memphis.
A graduate of Duke University (B.A.), Stacey received her J.D. from Tulane University Law School and is a member of the Tennessee Bar Association.
We're pleased to be a Patron Sponsor at the Appalachian College Association's 2016 Annual Meeting of Presidents and Chief Academic Officers (June 6-7). Come by our table to say hello or sign up for a pro-bono 30-minute consultation regarding your online strategy (email@example.com).
Dr. John E. Neal, President
JenEd Consulting has been retained as search counsel for this position:
Medaille College seeks an Assistant Vice President for Academic Affairs for Online Learning and Continuing Education to lead and expand fully-online degree programs for post-traditional learners (undergraduate and graduate) as a strategic dimension of Medaille College’s growth, and as a critical component of the college’s future identity and revenue. This position reports to the Vice President for Academic Affairs. For more information, visit jobs.medaille.edu.
Applications and nominations should be electronically submitted to Dr. John Neal, President of JenEd Consulting, at firstname.lastname@example.org.
In one national survey of Chief Academic Officers, 70.8% agreed that online education is critical for their institution’s long-term strategy. If that’s true for your institution, how do you move the online discussion to a more strategic level?
● Be comprehensive
At many institutions, online planning involves people from one specialized area of the institution, but often ignores other functional areas that can help (or hinder) online success. This “silo” approach to planning can lead to fragmented views of preparedness and/or performance, and miss the broader needs and demands of an institutional strategy.
● Identify factors related to success
The word “strategic” refers to those things that help achieve a plan, not just planning for its own sake. With help from my colleague Madeleine Rotrand (www.madeleinerotrand.com), we’ve created the infographic “Should We Take Our Degree Programs Online?” to help you think through (or rethink) those factors that drive a comprehensive online strategy--whether you’re considering fully-online degrees for the first time, or reviewing your traction and momentum after years of online delivery.
● Change the rules of the game
To become strategic about online, we need to change our perspective from that of a poker player (hoping that the hand we’ve been dealt is a winner) to a chess master (moving pieces into alignment over time in advance of a planned action).
So, what’s the first step in being more strategic? We recommend an initial strategic assessment to provide a fresh set of eyes and a new perspective to help answer two fundamental questions (How are we doing? Are we ready for what we’re planning?).
JenEd Consulting offers broad experience in providing strategic assessments that assist institutional planning for online growth and expansion. We have found these assessments to be critical components to three levels of strategic development:
● Short-term operational changes to realize immediate opportunities for growth and improvement
● Mid-range shifts in institutional priorities, structures, and practices to create new markets and significantly larger enrollments
● Long-term concepts to focus strategic priority setting and resource allocation
I hope you’ll take the first step and let us know how we can bring a fresh perspective to your online strategy.
Dr. John E. Neal
JenEd Consulting, LLC
Educationconnex, a leading enrollment solutions provider, and JenEd Consulting, a provider of comprehensive consultation to help colleges and universities expand their reach through mission driven online programs, will work together on select projects to enhance the success of their partner clients.
“Both companies are dedicated to the academic success of our client universities and their students,” said Greg Swinhart, SVP of Educationconnex. “We’ve enjoyed and benefited from our professional relationship with John; he is among the most knowledgeable people when it comes to strategies for adult and online initiatives. We welcome the opportunity to collaborate with JenEd as we work with our partner schools to help them increase online enrollment”.
“It’s always a pleasure to work with people that you trust and respect,” said Dr. John Neal, President of JenEd Consulting. “I’m honored that Jeff and Greg Swinhart would involve me in the important and effective work they perform on behalf of their clients. I also look forward to involving them on projects with JenEd clients.”
About JenED: JenEd Consulting, LLC provides specialized and comprehensive expertise in online learning to colleges and universities through unbiased advice and support services that help institutions serve online students well.
About Educationconnex: Since 2008, Educationconnex has been helping nonprofit universities grow enrollment through strategic marketing and recruitment solutions by developing data driven enrollment marketing and conversion strategies that produce a consistent, predictable flow of student applications.
For more information, visit www.edcuationconnex.com
I’ve been reviewing the Final Report of the MIT Online Education Policy Initiative (Online Education: A Catalyst for Higher Education Reforms, April 2016). While many of our colleagues are already debating and discussing the relative merits of the report and it’s many aspects, I was struck by a question and a comment in the Foreword:
“Where does online education fit in higher education?”
“A quick set of recommendations appeared neither appropriate nor convincing...a deeper dive was necessary…”
That underlying question reminds me of a similar comment I hear from college and university presidents:
"John, I just don’t know what I’m going to do about online.”
To which I give a similar answer:
”Don’t treat online merely as a problem to fix, but as a strategy to develop.”
That’s why we’ve created JenEd Consulting--to provide comprehensive consultation services for colleges and universities seeking to extend their mission through quality online programming--to help institutions do online well.
For many colleges and universities, online programming represents the most critical strategic dimension for their market visibility and financial stability. Yet, most executive leaders lack the experience and/or expertise to create and lead post-traditional programming and expansion. So, institutions look for the elusive “road map” for success in launching, growing, and sustaining online programs and enrollments. Sometimes, they build in-house operations with the aid of an internal online “guru”, or they contract with third-party providers, such as Online Program Managers (OPMs) to bring expertise and resources to the institution. But, the use of gurus and OPMs does not alleviate the institution’s need to create a strategy for online--policies, procedures, structures, leaders, and teams--to build online sustainability and scalability.
That’s our focus at JenEd--to help institutions develop and fulfill strategic goals:
Does higher education need one more consultant? We think so, particularly in the context of this critical--and strategic--dimension:
So, as you’re facing a variety of challenges related to online, we hope you’ll consider JenEd Consulting:
Whatever your current online questions or challenges, I hope you’ll check us out--drop us a line--or give us a call.
Dr. John E. Neal
JenEd Consulting, LLC
Online. Done Well.