 In recent years, the Learning Management System (LMS) has emerged as the enterprise technology for most academic functions in higher education. No longer defined within the narrow parameters of course management, the current generation of LMSs are a powerful amalgamation of many functions including content management, learning analytics, electronic portfolios, program tracking, digital rights management, enterprise academic calendaring, teleconferencing, adaptive instruction and assessment and more.
Your LMS is also likely to be a hub for several other technologies including your student information system, customer relationship management, and a myriad of administrative systems associated with an ERP (Enterprise Resource Planning). The LMS is now used to support traditional face-to-face, hybrid and fully online instruction and it is difficult to overestimate the strategic criticality of this system for current and coming academics. A routine and comprehensive evaluation of your LMS is critical for sustaining and advancing the academic mission of your institution. When conducting such evaluations, you may wish to consider the following questions in your examination. 1. What are my eLearning needs and plans? The evaluation of any enterprise technology must begin with an understanding of its role in your college or university. This includes both the current institutional context and your plans the use of the system in the coming years. While a fully articulated strategic plan is not necessary to evaluate your LMS, you will need a working document that includes a statement of your institutional needs, goals and/or objectives to be used for determining the standards for the evaluation. Since the assessment of your LMS can only be as strong as the criteria you use to evaluate, your consideration of these needs should be a thoughtful and complete representation of the Institution’s requirements. You will likely want to create a small team of five to eight critical staff members involved in the use of the current LMS and with broad institutional perspectives in academic, administrative and technical areas. 2. Does the existing LMS meet the current needs and plans? The evaluation of your existing LMS usually begins with some list of prioritized criteria based on the response to question one. These should reference both your existing needs and your plans for growth and development. Once the criteria are assembled, an honest and thoughtful review of your current system against these standards should be completed. There are a few important points to consider in this evaluation. First, realize that there will be inherent bias in the evaluation of your existing system from anyone who currently uses it in your institution. The bias can occur both for and against the current system and it will occur even if the evaluators are aware of this possibility and are attempting to control for it. You may wish to consider an external evaluator to augment your staff. The answer to question two is often not a clear “Yes” or “No.” A careful review of your LMS against the criteria you have identified will likely show strengths and weaknesses of the system as it stands. The team used to create the criteria for evaluation should meet again to consider the output of the analysis. This final determination of question two is generally provided in terms of a short report with a narrative that describes the performance of the existing system against areas of the criteria and a recommendation to maintain the current system, change or redeploy the current system or consider a new system. 3. Can another LMS better meet my needs and plans? If the response to question two leads you to consider replacing your existing LMS, you should plan to dedicate significant time and resources to the consideration of a new system. Selecting a new LMS can be an intimidating process. The market is large and complex and current systems vary widely. No single LMS is the strongest in all categories of functionality. While you will certainly want to use the criteria you identified for evaluating your existing system, realize that assessing candidate LMSs is more involved than a simple environmental scan using checklists and questionnaires. This task is a more comprehensive process that includes an RFI (Request for Information) or an RFP (Request for Proposal) created by a team of qualified representatives of the important areas of your Institution. You may also want to include expertise that is external to your college or university to assist in building the documents and assessing the responses. 4. Is the change worth the cost? In the final calculus of selecting a new LMS or maintaining the existing one, be sure to consider the total cost. The licensing of the software is only one element of total cost. Be sure to include other hard technical costs in the fiscal evaluation. For example, is the LMS to be hosted externally or internally? What are the total relative costs of each solution including staff training and augmentation, new hardware, infrastructure build-outs, networking upgrades, etc. Try to provide an extensive accounting for the peripheral costs of implementing your new system. Consider the real costs of preparing your students, faculty and administrators. Be sure to reference conversion costs such as the need to maintain concurrent systems if that is within your transition plan. Include expenditures associated with the development of new training materials, help functions, and associated staffing. Consider also the soft but legitimate costs of the transition of your user community. The hours that students and faculty use to learn the new system may not be directly billable to you, but they may have an adverse effect on the process of learning and teaching. This can be mitigated with a thoughtful transition plan and careful communication but there will be some frustration. Conversely, frustration may occur by maintaining a legacy LMS with outmoded functions that no longer meets core needs of your user community. Your final determination between an existing system and a new system will be greatly informed by question four. After the careful consideration you have given questions two and three, be sure you provide the same diligence to your analysis of real costs. The consultants at JenEd Consulting have extensive experience in helping to determine your LMS needs, evaluating your current system and assessing new systems for use in your institution. Whether you need someone to lead the process or just an external set of experienced eyes, we are ready to assist you in your LMS review. Dr. Rob Sapp Senior Associate rob@jenedconsulting.com  One of the great benefits of online instruction is the ability it affords to track the activities and behavior of the students on the virtual campus. By identifying the characteristics of “successful” students and the metrics that measure those characteristics, we can determine which student conditions, activities, and behaviors best predict student success. But even those institutions with the most advanced online learning programs are only just beginning to realize the untapped potential of their student and faculty data. This type of critical inquiry requires a thoughtful design by experienced researchers. Here are some suggestions to get you started.
Student Success At the foundation of nearly all institutional research is the goal of student success. How does your institution define student success? Some common definers include student performance (formative and summative grades), retention, class standing, progression toward degree, satisfaction, etc. The definition for your institution may include degrees of all of these elements. However, the definition may be nuanced and the weighting of these elements is likely to vary greatly from one institution to another. It is important to build a consensus about what the institutional definition of student success is before determining the processes and selecting the metrics to measure it. Choosing Data Sources Once you have a definition for student success, you should take inventory of the data sources available for analysis. You will likely need to input from several departments within your college or university to complete a comprehensive account of the available sources. Consider including the owners or “keepers” of areas such as your learning technologies, learning content, academic information systems, enterprise resource planning systems, etc. You may need multiple levels of expertise for some of these systems, e.g., content knowledge, information technology knowledge, policy knowledge, and more. Now consider what specific sources and systems will be included in your inventory. Typical locations include the Learning Management System, Student Information System, Customer Relations Manager, Enterprise Resource Planning, Education Management Information Software, etc. This will likely begin as a relatively small list. However, the inventory must also include field level data from the sources; these are the actual metrics that will be used in the analysis, either as direct input OR as data for generating input. There are classifications of student data for consideration in your analysis. Consider the following: • Student Conditions: Student conditions are those student metrics that occur outside the classroom. Common student conditions include demographics, academic programs, previous performance, class status/standing, etc. • Student Activities: Student activities are the actions that students take in your virtual campus and online course. Some examples of student activities include the time spent in course (total time, average time per day, average time per session, etc.), time spent in various content areas, performance in course to date, when assignments are completed, etc. • Student Behavior: Some research efforts elect to separate student activities and student behavior. Student behavior examines the “how” of specific student activities. How do students elect to consume instructional content and complete their assignments? What are the sequences they use to complete necessary academic activities? In many instances, how students choose to work online can tell us as much or even more than the actual tasks they have completed. Analysis and Interpretation The processes for analysis and interpretation of your data will vary widely depending on your research questions, available sources, the metrics that you chose, and the output you desire. It is likely that you will want a descriptive statistical report for the key variables of your student population. These may include metrics describing distributions, tendencies, and dispersions. It is also quite likely your analysis will include inferential statistical models. These types of analyses examine the relationship between two or more variables. For example, what is the predictive value of time spent in course content to the final grade in that course? Once again, the quantitative models you select will be a product of the nature of your research and metrics you have decided to use. The interpretation of your statistical output is normally completed in a small team with senior members of academic administration, subject or content experts, the researchers, and experts in relevant statistical areas. The key to quality interpretation is determining a consensus based on the data and articulating that in “actionable” terms. By actionable, we mean a clear and obvious path to change based on the output of the data. Conclusion One of the reasons that colleges and universities have been reluctant to pursue significant research using their online student data is the perception of the complexities and difficulties associated with it. While research of student conditions, activities and behavior is not a trivial thing, it is certainly a realistic goal worth pursuing. Knowing what student actions are likely to predict factors of success such as strong academic performance, retention, persistence, and completion of degrees are clearly invaluable strategic factors. JenEd Consulting has a great deal of experience in the entire research lifecycle. We can help you articulate your research questions, identify key data sources, prepare the data for analysis, select the appropriate research models and methods and complete actionable interpretation of the output. Dr. Rob Sapp Senior Associate rob@jenedconsulting.com Welcome to the JenEd Consulting team, Stacey!
Stacey Garrett Koju brings over 20 years of experience in higher education, healthcare, and government affairs to her work at JenEd Consulting. As a former vice president for online programs, Stacey combines extensive legal expertise with a practical understanding of the contractual and regulatory challenges facing colleges and universities operating in the online space. Stacey is a founding member of Bone McAllester Norton PLLC, and represents several higher education institutions. She also served as Vice President for Online at Cumberland University in Lebanon, Tennessee and was Deputy Chief of Staff for U.S. Congressman Harold Ford, Jr. of Memphis. A graduate of Duke University (B.A.), Stacey received her J.D. from Tulane University Law School and is a member of the Tennessee Bar Association. We're pleased to be a Patron Sponsor at the Appalachian College Association's 2016 Annual Meeting of Presidents and Chief Academic Officers (June 6-7). Come by our table to say hello or sign up for a pro-bono 30-minute consultation regarding your online strategy (john@jenedconsulting.com).
Dr. John E. Neal, President JenEd Consulting has been retained as search counsel for this position:
Medaille College seeks an Assistant Vice President for Academic Affairs for Online Learning and Continuing Education to lead and expand fully-online degree programs for post-traditional learners (undergraduate and graduate) as a strategic dimension of Medaille College’s growth, and as a critical component of the college’s future identity and revenue. This position reports to the Vice President for Academic Affairs. For more information, visit jobs.medaille.edu. Applications and nominations should be electronically submitted to Dr. John Neal, President of JenEd Consulting, at john@jenedconsulting.com. In one national survey of Chief Academic Officers, 70.8% agreed that online education is critical for their institution’s long-term strategy. If that’s true for your institution, how do you move the online discussion to a more strategic level?
● Be comprehensive At many institutions, online planning involves people from one specialized area of the institution, but often ignores other functional areas that can help (or hinder) online success. This “silo” approach to planning can lead to fragmented views of preparedness and/or performance, and miss the broader needs and demands of an institutional strategy. ● Identify factors related to success The word “strategic” refers to those things that help achieve a plan, not just planning for its own sake. With help from my colleague Madeleine Rotrand (www.madeleinerotrand.com), we’ve created the infographic “Should We Take Our Degree Programs Online?” to help you think through (or rethink) those factors that drive a comprehensive online strategy--whether you’re considering fully-online degrees for the first time, or reviewing your traction and momentum after years of online delivery. ● Change the rules of the game To become strategic about online, we need to change our perspective from that of a poker player (hoping that the hand we’ve been dealt is a winner) to a chess master (moving pieces into alignment over time in advance of a planned action). So, what’s the first step in being more strategic? We recommend an initial strategic assessment to provide a fresh set of eyes and a new perspective to help answer two fundamental questions (How are we doing? Are we ready for what we’re planning?). JenEd Consulting offers broad experience in providing strategic assessments that assist institutional planning for online growth and expansion. We have found these assessments to be critical components to three levels of strategic development: ● Short-term operational changes to realize immediate opportunities for growth and improvement ● Mid-range shifts in institutional priorities, structures, and practices to create new markets and significantly larger enrollments ● Long-term concepts to focus strategic priority setting and resource allocation I hope you’ll take the first step and let us know how we can bring a fresh perspective to your online strategy. Dr. John E. Neal JenEd Consulting, LLC www.jenedconsulting.com john@jenedconsulting.com Educationconnex, a leading enrollment solutions provider, and JenEd Consulting, a provider of comprehensive consultation to help colleges and universities expand their reach through mission driven online programs, will work together on select projects to enhance the success of their partner clients.
“Both companies are dedicated to the academic success of our client universities and their students,” said Greg Swinhart, SVP of Educationconnex. “We’ve enjoyed and benefited from our professional relationship with John; he is among the most knowledgeable people when it comes to strategies for adult and online initiatives. We welcome the opportunity to collaborate with JenEd as we work with our partner schools to help them increase online enrollment”. “It’s always a pleasure to work with people that you trust and respect,” said Dr. John Neal, President of JenEd Consulting. “I’m honored that Jeff and Greg Swinhart would involve me in the important and effective work they perform on behalf of their clients. I also look forward to involving them on projects with JenEd clients.” About JenED: JenEd Consulting, LLC provides specialized and comprehensive expertise in online learning to colleges and universities through unbiased advice and support services that help institutions serve online students well. About Educationconnex: Since 2008, Educationconnex has been helping nonprofit universities grow enrollment through strategic marketing and recruitment solutions by developing data driven enrollment marketing and conversion strategies that produce a consistent, predictable flow of student applications. For more information, visit www.edcuationconnex.com I’ve been reviewing the Final Report of the MIT Online Education Policy Initiative (Online Education: A Catalyst for Higher Education Reforms, April 2016). While many of our colleagues are already debating and discussing the relative merits of the report and it’s many aspects, I was struck by a question and a comment in the Foreword:
“Where does online education fit in higher education?” “A quick set of recommendations appeared neither appropriate nor convincing...a deeper dive was necessary…” That underlying question reminds me of a similar comment I hear from college and university presidents: "John, I just don’t know what I’m going to do about online.” To which I give a similar answer: ”Don’t treat online merely as a problem to fix, but as a strategy to develop.” That’s why we’ve created JenEd Consulting--to provide comprehensive consultation services for colleges and universities seeking to extend their mission through quality online programming--to help institutions do online well. For many colleges and universities, online programming represents the most critical strategic dimension for their market visibility and financial stability. Yet, most executive leaders lack the experience and/or expertise to create and lead post-traditional programming and expansion. So, institutions look for the elusive “road map” for success in launching, growing, and sustaining online programs and enrollments. Sometimes, they build in-house operations with the aid of an internal online “guru”, or they contract with third-party providers, such as Online Program Managers (OPMs) to bring expertise and resources to the institution. But, the use of gurus and OPMs does not alleviate the institution’s need to create a strategy for online--policies, procedures, structures, leaders, and teams--to build online sustainability and scalability. That’s our focus at JenEd--to help institutions develop and fulfill strategic goals:
Does higher education need one more consultant? We think so, particularly in the context of this critical--and strategic--dimension:
So, as you’re facing a variety of challenges related to online, we hope you’ll consider JenEd Consulting:
Whatever your current online questions or challenges, I hope you’ll check us out--drop us a line--or give us a call. Dr. John E. Neal President JenEd Consulting, LLC Online. Done Well. |
RSS Feed