Although slow to develop, CCC’s online tutoring service is providing meaningful assistance to students in a variety or courses, from math and algebra to English, psychology and sociology and science courses. Here is a comment from a distance learning student this semester who was having difficulty accessing a computer CD that accompanied her textbook:
“I am amazed of how wonderful you and Mr. Brooks Pace are to us. Thank you for be here for us at anytime we need help.  It can be a stressful here at home sometimes by ourselves when we need help. You guys are our release — thanks!”

One of the best things about the service is that students who have been helped one semester are returning for help with their other course work, and they are referring other students. A few instructors are also directing their students to the Online Tutoring Service for assistance, which serves both the students and instructors.

Whether responding to e-mails or Blackboard Discussion Board postings, we try to be prompt in our responses and to provide meaningful feedback to enhance the students’ understanding of the material they are learning, and that seems to be working as evidenced by this student’s comment:

“I just wanted to thank you again for your help with my essay. I got a 97! I learned a lot from your tips too, so thank you again.”
We want to facilitate learning — not do their work for them.

We also make ourselves available by phone, if necessary, and have even met with a few students face-to-face. In addition to the tutoring, the Blackboard site offers resources and links that help some students “self-serve.”

When necessary, we will e-mail instructors to be sure we are clear on what their expectations are for their classes or a particular assignment. We would like to have even more collaboration with CCC faculty, whether it be to help their students fine-tune their work or spend some extra time with those who need some remedial help.

We all know that many CCC students are juggling, school work and families, so online learning is the best option for their education. It does, however, limit their face-to-face access to instructors during their office hours or to campus academic support. As more and more classes are being offered online and more and more students are taking online courses, the benefits and use of online tutoring is likely to grow.

Lisa Taylor-Galizia
Carteret Community College
Adjunct Instructor

The CCC Online Tutoring Service is a Pilot Program Supported and Funded by Title III.

We are now 3 weeks into the Fall semester and the Distance Learning Team in working on a variety of professional development initiatives, in addition to making sure all online courses at CCC are meeting the Quality Assurance Standards we established last year. We have cross referenced a sampling of our online courses and I must report it is good to see how all the Blackboard training has paid off and so many of our faculty are implementing more creative and innovative instructional design strategies. Our Title III funding and support has played such an integral part in the progress we’ve made over the past 3 years.

Our fourth DL Pioneer team has been selected and we had our kick-off meeting last Friday. We have an excellent team made up of both faculty and staff and they’ve all made commitments to retool and upgrade their online courses, attend professional development activities in their disciplines and essentially take their knowledge of online learning and distance education to a higher level in the next year all made possible by Title III funds. Our staff members on this team will be exploring and implementing new and creative ways (blogging, Youtube, facebook, twitter) to work with our student in the online environment

As Vice President of the NC3ADL Eastern Region I’m also in the process of coordinating a Regional DL mini-conference/workshop for Friday, November 6th here at CCC. Due to travel restraints across the state it looks like there will be no NC3ADL conference this year so we are doing a series of regional conferences. NC3ADL will pay for lunch and I’m going to look to our DL Pioneers from Phase 1-3 to assist with it and have phase 4 attend. The following is a rough outline for that regional conference/workshop.

Workshop Overview – Two Tracks  Friday, November 6, 2009 9:00 – 4:00

2 tracks in the morning 9:30 – 10:30 and 10:45 – 11:45
Lunch 12:00 – 12:45
2 tracks after lunch
1-2 and 2-3 (Optional Track 3-4 on Dl Assessment)

1. Moodle Basics
2. Podcasting for the Technically Challenged
3. Best Practices for Online Teaching
4. Learning Object Repository
5. Social Networking as a Teaching/Learning/ Marketing Tool
6. Blogging for Beginners
7. Incorporating Itunes U and You Tube in your online courses.
8. DL Assessment with Don Staub

My constant goal is to make our online classes every bit as viable, interesting and dynamic as our traditional classroom lectures and labs. Thankfully there are a host of new and powerful communications tools that are making it easier and less time consuming for our faculty to do this. Carteret Community College has a Title III grant resources to help us implement our various professional development initiatives and put these tools in the hands of our faculty so they can take their online teaching to a whole new level of technical and pedagogical excellence.

Retention rates are the proportion of students who graduate or return to their program the following semester.  Usually, retention is calculated from Fall to Fall, but Fall to Spring  metrics can be generated.   Retention is a key metric used to compare college performance throughout the US.

Persistence refers to the proportion of students who complete a course. 

Typically, retention rates are reported for programs; persistence is reported for courses. 

While it makes intuitive sense that persistence would be related to retention, this is challenging to prove.

Therefore, in the true spirit of blogging, I’d like to entertain all ideas regarding the hypothesis that these measures are related. 

• If you don’t think they are related, let me hear why.
• If you think retention and persistence are related, let me know that and how strong you think the relationship is (i.e., if you know one measure, you know the other; persistence explains only part of retention-other factors are more important, etc.)
• And if you think there is a relationship, how can we show it?

Why would a connection between retention and persistence be useful to demonstrate?  

When we know which relationships have the most influence on retention, we are more likely to influence it and therefore, improve retention rates.  Conceptually, persistence is similar to retention in that the student completes something. 

Further, persistence may be easier to influence than retention since it is measured at the course level. 

Both are important metrics.   If retention and persistence are positively related (i.e., as one improves, so does the other), efforts to influence one metric will positively impact the other.  This will enhance our understanding of both program and course level performance using the tools already in place and give us confidence that we are using our resources in the best way.

Submitted by Cindy Schersching, PhD, August 20, 2009

Download a pdf of this white paper by CLICKING HERE.

A key challenge to understanding the metrics of retention, persistence and success (as well as other metrics) associated with a specific course, a program, and/or segment of the student population is to find meaning in the numbers.  Commonly, meaning is found by comparing observed metrics with a standard or benchmark.   The challenge is to determine which comparisons yield the most valuable and instructive information.

Benchmarks can be sourced in a variety of ways.    Let’s review appropriate possible benchmarks in the context of retention and educational institutions.

1.   A benchmark can reflect internal institutional standards. For example, some institutions may refuse to accept retention rates less than 80%; others may find 60% acceptable.  These benchmarks reflect the organizations’ heritage, judgment of key decision makers, financial determinants, etc.

2.  A benchmark may be the best performing institution in the ‘universe’ of all U.S. 2-year public colleges. If, for example, a two-year public college with characteristics similar to Carteret had a retention rate of 85%, we may want to base our progress against this metric.

3.  By definition, an expectation is an average.   Therefore, another benchmark can be an average across years for a specific program. A comparison of each individual year to this average across years will indicate how consistent the retention rate has been.  The challenge is to ‘beat the average.’

If a review of each individual year relative to the average highlights a significantly positive year, additional analyses will uncover the combination of characteristics that differentiates that year from the years that were at-to-below average.

The downside to this type of benchmark is that it is narrow and self-defined.

4.  Another benchmark can be created by averaging across all of the programs with the same base of students (first time, full and part time, degree seeking cohorts) within a year.  This benchmark can indicate whether or not the specific program is different from other programs taken by similar students.   If the program is a stand-out, efforts should primarily focus on analyses of institutional characteristics to identify what is contributing to this program’s success.

Of benchmarks, this approach is quite informative.

5.   An even better standard of comparison is one based on the entire college ‘universe.’ Retention rates based on the diversity of the student population most clearly highlight outstanding programs/majors as well as those underperforming.  These ‘deviations’ from the average based on the total student ‘universe’ can guide further investigations.   Learning what contributes to outstanding retention rates in one program can be leveraged across other programs to raise overall performance.

This benchmark provides very useful direction.   Comparing the same program but across different sample definitions suggests there are student characteristics – either separately or interacting with institutional factors – that are contributing to the observed retention levels.

For the Title III investigation of retention, we are using options 3, 4 and 5 (where we have data) to identify what programs are performing to expectation and to highlight those who are more successful in keeping students.

When using any benchmarks, keep these considerations in mind:

• Learning is strengthened when comparisons to different benchmarks suggest the same outcome.
• Meaningful benchmarks are created on robust sample sizes.  Ideally, the base for these benchmarks is 30+ students.
• It is common to show comparisons for a specific program/major to a benchmark in terms of an index.  The index is created by dividing one percentage by another.  A rule of thumb is those indices of 80 and below and 120 and higher suggest the specific program/major is in some way ‘significantly’ different from the ‘universe’ against which it is being measured.   These indices align roughly with 2+ standard deviations (see below).

To give clarity to the idea of benchmarking, let’s take a specific example from actual data associated with the first time, full and part time, degree seeking cohorts.   We will look at the data for each of the GOT program.   Retention rates are run for fall to fall and fall to spring by years: 2005-2006, 2006-2007 and 2007-2008.  (At the time of this initial analysis fall to fall 2008-2009 data are not available.).

To date, the College has not set benchmarks as defined in options one (1) and two (2).

However, we can calculate the average retention rate for the GOT program from 2005 to 2008 (benchmark option 3).

Submitted by Cindy Schersching, PhD.  Title III

Title III sponsored another well attended Blackboard Boot Camp training workshop on Friday, July 31st in the Wayne West Computer Lab.  We had 18 faculty and staff from across the college attend and 2 graduate students from East Carolina University who requested the training without compensation.

The T3 sponsored Blackboard Boot Camp has been a very successful and benificial training program for Carteret Community College.  It has prepared and certified all (100+) its online faculty including adjuncts to teach effectively in the online environment.

This is  comprehensive “hands-on” professional development that has been supported (funded) by Title III for over 2 years.

East Carolina University Graduate student and photography adjunct instructor Adam Jacono stated that he wished East Carolina University had a certification program for online teachers like our Blackboard Boot Camp.

The Boot Camp is also a catalyst (springboard) for our “one- on-one” training that Instructional Technologist Pre-Ah Hil schedules after new instructors receive their initial training.  She assists them with all aspects of course design, retooling and rich media production.  Pre-Ah continues to go above and beyond the call of duty when it comes to supporting distance learning at CCC.

Title III support has been extremely valuable to Carteret Community College and its distance learning initiatives.  To learn more about Blackboard Boot Camps please check out the CCC DL Blog.

Here’s a brief report/commentary on the 2009 annual Noel-Levitz annual conference on Student Recruitment, Marketing, and Retention in San Antonio, Texas.   Naturally, this is from my perspective…what I was hearing overall, and what I was hearing in specific sessions. Once again, N-L has put on a great conference – lots of very good sessions, a well-organized event, and in a city where it was fun to hang out (San Antonio).

Patrick and I lucked out with the scheduling, as our session one was in the first breakout session. This allowed us to turn our attention to the rest of the conference early on.  Our presentation can be found here.   Afterward, our session moderator gave us a peek at our session evaluations, and, almost to a person, they really liked the blog, and all the materials we are willing to share through it.

Looking back, what I heard as the primary topics of conversation were (beyond the obvious discussions of resources) – Orientation, the First Year Experience course, Early Alert, Web 2.0, Communicating with Today’s Students, and Developmental Education.

Orientation is a must…everyone who is working on retention has an orientation in place. [so why don’t we??] . And, hand in hand with orientation, there was much mentioned about the First Year Experience course, or what we call ACA.  Almost any time a school mentioned success in retention efforts, one of their strategies was to have a well-designed first year experience course. There was also broad discussion about early alerts.  Noel-Levitz has a product that was discussed in a number of sessions (e.g. see “Enhancing Student Success in Developmental Mathematics” below), but most of the talk that I heard was about the importance of an EA system, and how it played into the larger realm of retention activities; there was little focus on the details of a system.  Web 2.0 and Communicating with Today’s Students were not mutually exclusive.  There was a good deal of discussion about harnessing today’s technologies to attract and retain today’s tech-savvy students.  I’ll let Patrick discuss this in greater detail over on the DL Blog in his conference debrief.   Finally, I also heard a lot about Developmental Education (particularly math) and its role in student attrition. One keynoter pointed out: “#1 predictor of attrition is performance in math.” DE, while it certainly has its value in higher education, is often looked to as a major issue when it comes to student attrition.  This is particularly so with Math.  So, the question becomes, how do we provide students with the skills that they need in college without chasing them away from the college (because of too many developmental courses).  One proprietary suggestion is discussed in the final session I attended…see below.

Here’s a quick list of the sessions I’m going to comment on.

Student Retention Strategies at Technical Colleges
Debra Gordon

The Most Effective Retention-Related Strategies for Community/Technical Colleges
Dave Trites

What Works in Student Retention?
Wes Habley

Moving Beyond the Retention Committee to Jump the Plateau
Alderman & Petrusch

Survey of entering Student Engagement (SENSE)
Karla Fisher

Enhancing Student Success in Developmental Mathematics
Corder & Panfil

Student Retention Strategies at Technical Colleges
Debra Gordon
DeKalb Technical College

This session discussed the variety of strategies the college has in place to support retention. The presenter noted that their president is graded (by the board) on graduation, placement, and retention.  She stressed, therefore, that, “Our expectation is that within the class, there is quality.  So, we focus on strategies outside the classroom…If we have a quality curriculum, we need quality support.”

In addition to an early alert system, advising center, academic support center, and so on, one intriguing service they provide is refresher courses for the Compass.  The presenter emphasized that this was not “teaching to the test.”  Rather, these courses provide a refresher in the skills that may be needed earn students a few extra points so they may not end up in developmental.  The sessions are taught by adjunct instructors and FT faculty.

My concern with this session was with the data presented.  Some retention data was provided for the last three years.  And while the school seems to be doing well on average (~65%), there was no significant upward movement in this indicator.  Which is OK, except for the fact that the presenter mentioned on numerous occasions that retention was increasing.  In addition, there was no disaggregated data to show which, if any cohorts, had been affected by the strategies, or which strategies were having the greatest impact.

The Most Effective Retention-Related Strategies for Community/Technical Colleges
Dave Trites
Noel-Levitz

Dave started off his session with some profound points…
• Requiring is a requirement.  Indeed, students want this.
• Expecting students to do the things that we know are important for success
• We need to be intrusive and intentional.

Dave turned the participants loose to discuss with someone nearby, their own stories about what kept them in school.  From there, we discussed the rating sheet handout, focusing on what we thought were important among the list.  Here are some of the best practices identified by the participants:
#29 – front load best instructors & advisors
#8 – first-year experience course

[Someone in the crowd from Davidson CC asked if we at CCC were teaching ACA 120 - Transfer Readiness]

Dave steered the conversation toward the results of the  N-L best practices in retention survey. Two key strategies that came to the top of the list were:
• Academic Support Programs
• ACA-type courses…one suggestion was for an ACA course designed specifically for particular majors.
One school mentioned that they had developed a website for students that listed a lot of the resources that students may access to help them with challenges they are facing.  They then promoted it all over campus. They call it: www.stayinschool.Dixie.edu
(students are encouraged to check out this site before they bail) [I could not find the site, however…but the idea is a good one]

Finally, Dave discussed his own list (based on 20+ years of experience) of what works in retention.  Here is what he presented:
Dave’s top 10
1. Improve teaching
Supplemental instruction
Learning communities (cohort)
Faculty/staff development

2. Academic Support

3. Ensure effective and efficient course management
Online & hybrid instruction

4. Require intrusive developmental advising (appreciative advising)
Use technology (for record keeping)
Orientation
First year experience course (including financial literacy)

5. Campus employment as a retention strategy (work study)

6.  Retention must be someone’s responsibility
Internal marketing and communication
Campus culture (caring for students)

7. Financial aid as retention tool
8. Recruit and admit students who can be successful
Accuracy and integrity: communications and measurement

9. Provide pre- and post-enrollment early alert systems
Pre-enrollment attributes
Attrition predictors (build your own database)

10. Analyzing data to target interventions
e.g. What’s the retention rate of those who register in the last month before classes?

What Works in Student Retention?
Wes Habley
ACT

“The key is to torture the data long enough until it confesses.”

“Retention should not be an institutional goal but rather the by-product of improved educational programs and services for students.”

In 2004, ACT published “What works in Student Retention?” (you can download reports by institution type).  They are publishing a follow-up survey this year – probably be out in September. This presentation was intended to provide the results of the study, but unfortunately, only about 60% of their sample had responded to the survey, so this was couched as “preliminary results” and therefore not representative.

You can download a copy of the presentation (with all of the relevant data) by CLICKING HERE.

You can also view the latest data on retention HERE:

From the survey (again, preliminary results) here is what they gleaned about retention in Community Colleges…

Student Characteristics
Of 18 student characteristics…
15 are cited as making a moderate contribution or higher to student attrition
Student characteristics cited as making the greatest contribution to attrition are:
• Level of student preparation for college-level work
• Level of commitment to earning a degree
• Student study skills
• Level of student motivation to succeed
• Adequacy of personal financial resources

Institutional Characteristics
Of 24 institutional characteristics…
Only 6 are cited as making a moderate contribution or higher to student attrition
• Amount of financial aid available to students
• Student engagement opportunities in the classroom
• Quality of interaction between faculty and students
• Student employment opportunities
• Student access to financial aid advising and information
• Adequate academic/learning support services

Greatest Contributors to Retention
• Learning Assistance/Academic Support
• Academic Advising
• Assessment and Course Placement

Highest Rated Interventions
Nine interventions with a mean rating of 4.0
• Increased number of academic advisors
• Mandated placement in courses based on test scores
• Remedial/developmental coursework required
• Comprehensive learning assistance center
• Math center/lab
• Writing center/lab
• Reading center/lab
• Tutoring
• Programs for first generation students
[presenter stressed that the key is assessment at point of entry and making sure they’re ready to study]

Greatest Impact
Identify the three programs on your campus that you believe have the highest impact on student retention
• Mandated course placement testing (28%)
• Tutoring program (19%)
• Required remedial/developmental courses (12%)
• Academic advising center (12%)
• Summer orientation (12%)
• All remaining practices cited at fewer than 10% of the colleges

If you want to see the data on Voc/Tech Schools, you should check out the PPT (which you can download here)

Moving Beyond the Retention Committee to Jump the Plateau
Alderman & Petrusch

This session was presented by the Retention director and the VP of enrollment at a 4-yr private university…so it didn’t directly apply to us here at a community college.  But, as always, there are some take-aways:

• They highly recommended the IHEP summer academy for strengthening their retention team

They discussed how they reviewed communications with current students – the language (tone) and the means for transmitting communication.  They stressed that they wanted to make sure tha they were showing support and giving students personal attention (e.g. what they found was that previously, they were communicating in such a way that they were setting students up for making late payments).

Their philosophy: We are recruiting not just students, but the whole family

Action: Develop plan for how you will respond to particular groups (cohorts)
• identify those groups now and create a timeline for when we will focus on each group(?)

Karla Fisher—College Relations Coordinator, The Center for Community College
Student Engagement

From the abstract:
“Based on preliminary data from the Survey of entering Student Engagement (SENSE), this presentation will offer insight into students’ earliest college experiences and provide examples from colleges committed to turning the tide of entering student attrition.”

The discussion was not so much about examples, but primarily about the instrument itself…which is OK because it seems like it can provide some interesting information about incoming students.  I struggled with some of the items discussed and in trying to figure out how they were actually telling us more about entering student engagement (e.g. I have learned better test taking skills here).

It’s something we may consider exploring.  It’s administered in the 1st-3rd weeks of fall semester, and done so primarily in developmental courses.
The cost is the same as the CCSSE (currently, for us, $6,000 per administration)

Also to consider is the Entering Students Success Institute
Send a team 3-5 members for 2.5 days.  You need to administer the SENSE first, and the data guides your work at the institute. (therefore, we would need to administer in FA’10 and attend in Spring ’11)

Enhancing Student Success in Developmental Mathematics
Corder & Panfil

I was attracted to this session because they were going to discuss the Noel-Levitz early alert system, coupled with a proprietary product for developmental math – two topics that had been discussed a lot across the campus.  I wanted to see how they were brought together.   So, it was a bit of a sales pitch, but it made me think about how we are supporting our developmental math students.  The proprietary product is software designed by Carnegie Learning and it works to the level of the student – adjusting as the student demonstrates greater or lesser mastery of the content.

The way that the whole concept was packaged was that the student takes the personalized Noel-Levitz survey that reveals a students needs and goals.  Together with placement scores, the counselor, the advisor, and the math instructor may look at a student’s current level in math, and determine where s/he needs to be in order to achieve educational goals.

In the end, the Noel-Levitz product seemed intriguing, and worth having a discussion about piloting.

I could say the same about the Carnegie Learning product…but cost and sustainability would be the thing to keep in mind.

Overall, another great Noel-Levitz conference.

CLICK HERE to download ppt

“Strengthening Our Institution The Power of Title III in Impacting Assessment, Distance Learning, & Advising at a Small, Rural Community College”

A presentation made at the 2009 Noel-Levitz National Conference on Student Recruitment, Marketing, and Retention

Abstract

Carteret Community College is a small, rural college in North Carolina. In 2006, the college was awarded a Title III grant from the US DOE Title III program, which has allowed us to completely alter our approaches to assessment, distance learning, and advising – all in an effort to improve abysmal retention rates.  The College has developed and implemented, from scratch, a comprehensive outcomes assessment initiative for all instructional programs and administrative services. Distance Learning, once the playground of the eccentric, has thrust the College into a statewide leadership role for online student services and instruction. A virtually non-existent advising program now has organization, purpose, and results.  Participants will hear the story, see the outcomes, and discuss organizational culture change.

Below are some of the key handouts from today’s presentation.  Much more can be learned about our efforts through this CCC Title III blog, as well as through the CCC DL blog.   We greatly appreciate your comments and questions.

The Presentation

Carteret Community College has been working diligently for the last 2+ years on the development and implementation of an Outcomes Assessment program. This is partly because of an impending accreditation reaffirmation report and a subsequent visit. But, mostly because it’s the right thing to do: We (like many other community colleges) struggle with retention issues, and improved instruction and services may lead to lower attrition rates. However, the only way to improve instruction and services is to know where you’re starting from and what needs to be improved, along with a continuous cycle of assessment, analysis, and use of results to ensure that you’re moving forward. Thus, the outcomes assessment program.

There is no doubt that a critical piece of this initiative has been funding through Title III. This has allowed our cash-strapped school to bring in experts to provide professional development specifically for outcomes assessment, to release full-time faculty in order to attend professional development, to provide stipends to part-time faculty to ensure that they too are receiving professional development, and to send faculty and staff to relevant, off-campus professional development opportunities.

This is where we have come in the last two years:

• Institutional Level Learning Outcomes (ILLOs…aka, Gen Ed outcomes) identified and assessed at the college;

Click here to download this file: illo-matrix-07-08 If you would like to learn a little about the process of actually selecting our 7 ILLOs, CLICK HERE.

• All 34 instructional programs have identified program level learning outcomes (PLLOs) and are assessing them, analyzing the results, and making use of the results to improve instruction;

Click here to download this sample PLLO from 2007-2008: aqu-07-08

• All 34 instructional programs are on a 4-phase program review cycle; i.e. each program undergoes an intensive program review process every four years (in Fall ’09, we will begin Phase III);

Click here to download a copy of the Instructional Program Review Manual: ipr-manual You can also take a look at sample completed (Phase I & Phase II) program reviews by CLICKING HERE.

• As the instructional programs undergo the review process, they also identify and assess Program and Administrative outcomes, analyze the results, and make use of the results;
• All administrative units at the college have identified administrative outcomes and are assessing them, analyzing the results, and making use of the results to improve the services being provided;

Click here to download this sample Administrative Outcome: academic-support-07-08

• All administrative units are on a 3-phase administrative unit review cycle.

Click here to download a copy of the Administrative Unit Review Manual: admin-unit-review-manual

Download a pdf of this white paper by CLICKING HERE.

This blog serves multiple purposes:

• It provides a forum for sharing the learning from the analyses of retention data.
• It allows each of you time to review these analyses and pose any questions/thoughts you might have
• It documents the work being done under the Title III grant.

The key objective of the data analyses is to provide an understanding of the factors that impact retention rates.   The retention issue is multi-faceted; data will be analyzed from a variety of perspectives.  It is expected that many of the analyses will show similar results (convergence);  when the same result is supported by a variety of analyses,  the stronger the result and more confidence we have in the learning.     Ultimately, we hope to weave a comprehensive story around the factors related to retention and design interventions that keep students in school.

At this date, retention analyses are underway amongst

• Cohorts, first time, degree seeking, full and part time students
• Students taking Gateway classes

The factors that may influence retention amongst these student groups (and the focus of the analyses) includes:

There are challenges to the analyses; every effort will be made to find acceptable solutions:

• There are multiple data bases.  It is difficult to merge information across data bases.
• Some information has never been collected.
• The level of detail associated with specific variables may be missing.
• Some information may be available on a group level only and not on an individual student level.

Retention rates are typically displayed by major, i.e., the total number of returning students in a program is divided by the total number of students enrolled in that program.

This is an alternative way of displaying the same data.  Instead of calculating retention based on the number enrolled in a major, the percentages are based on sum of those returning and the sum of the total number enrolled across major.

There are two advantages to reviewing the data in this format:

• It provides a real perspective on the data in the context of the total student body
• It allows the creation of an index by major.  The index of 100 is expected if, in this case, retention rates observed by major are much greater – or much less – than expected.

The power point presentation that follows this section goes into greater detail on the value of the index.

Posted by Cindy Schersching, PhD  July 19, 2009

Click here for Spring / Summer DL Report 2009

CLICK HERE to download a pdf of the following report.

Introduction
Eight instructional programs and one discipline completed the review process this past year. All have submitted their final reports to the Program Review Sub-Committee, which has, in turned, reviewed each of the final products (using a rubric).  A final report, based on those comments, is forthcoming.

Major findings/suggestions
There were a number of principal findings and suggestions that emerged from the debriefing session.  Regarding data, it appears that thre may have been a bottle neck when it came to data requests; i.e. when everyone asks for data at once, it may be overwhelming for Jennifer.  One suggestion was to give each of the review teams a window during which they could work with Jennifer on their data needs.  Another suggestion is to begin requesting data in May or June, so that she may have more time to generate a report, and the team has more time to analyze.

In terms of the process, some suggestions were to begin the process in the spring. Another suggestion was that the manual was not as useful as it could be.  The sample was not aligned with what was required, and section descriptions were sometimes vague.  There are plenty of samples now available in the repository, but the manual itself could have more elaboration, with relevant samples in each section.  The group also agreed that for the final presentation, it should be the CAC who makes it.  Curriculum Committee liaisons felt uncomfortable about fielding questions for a program that was not theirs.  Finally, the group felt that it would be useful to forgo the August orientation for more one-on-one/small group discussions with the leadership of each review team.  One of the outcomes of that meeting would be the selection, with program input, of the full team composition.

The longest discussion centered on the review team and roles and responsibilities.  Perhaps the number one suggestion was to revise the way that teams are chosen.  It was suggested (and echoed) that the program under review play a greater role in selecting review team members.  This would be based on the needs of the program in terms of the review.  There was general consensus that the curriculum committee liaisons had difficulty grasping the process, until about the half-way point.  One suggestion was to have the PRASC serve as the external readers, providing feedback throughout the year, as sections are submitted.  Another suggestion was for a true external stakeholder to be a review team member, thus providing a state-of-the-industry perspective.

Main points from debriefing

Data
• Waiting for data could be frustrating
• Not knowing what data to ask for could be frustrating; part of this was the nature of the research…sometimes it wasn’t until late that you realized what data you needed
• Provide review teams with a list of available reports, so that they know what’s available, without having to go fishing for it on the L drive
• Start process in summer – gives you time to start pulling together data so that you have information to go on in the fall;
• Some data requests went unfulfilled, and team had to shift the way it responded to certain items
• Inconsistency in sources of data (What Susan provided some times did not agree with what Jennifer provided).
• Suggestion: Stagger start for data requests…a window of opportunity for each program

The Process
• There should be no exceptions for faculty and staff in fulfilling the responsibility of being on a team.  I was excused last year, and was in the dark this year.
• The model/sample in the manual is not aligned with what is requested
• What to provide up front for illustration: Not necessarily a template, but perhaps samples that fit broad categories (e.g. sciences, humanities, etc…) [of course, there are now 16 samples in the repository to select from].
• A template for formatting…not fill in the blanks, but one that has the basic formatting (e.g. fonts, headings, etc…) in place [ask Mary to help with this?]
• [working with Patrick to have some crossover between Phase IV pioneers and review team members…goal would be to have one or two reviews (including process) that are completely online]
• Final presentations:  to encourage more dialogue, make sure the curriculum committee understands their role for this process; Or, maybe presentations in a different venue
• CAC should be making the presentation – they can field questions a lot easier than the review chair

Roles & Responsibilities (the team)
• I was confused the whole time…CAC had been involved in a review the year before, “so she did all the writing and data collection and everything…I was just kind of doing a little editing…even though I was the chair of the committee, in name, I felt like I was just following.”
• It wasn’t until half-way through the process that our chair (i.e. curriculum committee liaison) realized what their role was to be.
• [as the curriculum committee liaison] “I have a real hard problem coming up with questions for a program that I know nothing about”
• I don’t think those who were not from the program area had a clear understanding of their role.  I think it was confusing for them to be the chair of a committee that’s not their program…it was confusing to them as to what they should be doing…they wanted to help, but didn’t know how.  And, in frustration, the program people end up taking the whole thing and writing it themselves.
• I had an SER person who provided data.  Otherwise, I wrote the entire thing.
• The SER person and the library person probably knew what they were doing, but for the others, the contribution was, “Oh yeah, that sounds good.”
• A lot of times, they are not in a position to know what to write…how to approach it…what questions to ask.
• People are busy as it is…to ask them to come in from other disciplines was busy work for them, and not useful for those who had to sit down and write it anyway.
• Have a true “outside” reader…someone from the community and the same profession who can bring educated objectivity…they know the lingo, they understand the needs of the industry, they know what’s cutting edge
• Could the faculty from the program under review, choose the team…the body of people who would be of greatest use to them
• Have the program review sub committee review the product – in part, and finally, in whole – throughout the year, using a rubric that is the same, or similar to, the one the PRASC is using already.  They could see trends, make suggestions (make clarifications, etc… to strengthen products throughout the year…as opposed to the end.
• The first meeting could be between Don (or the person arranging teams) and the review chair and the program chair… and the three of you brainstorm who you would need.  What approach does that program need to get the information that is needed.

Working toward a useful product
• The three would also work on shaping the review (within the broad parameters).
• A discussion could take place about the program…strengths, weaknesses, questions….which would provide an understanding of the process and the bigger picture (the outcome).
• Someone who could help, at least in this initial meeting, would be someone who has just gone through the process…someone who’s already been down the path.
• The [orientation] was not effective…the focus was on who was on what team…”what’s our title?”  etc….
• The best feedback was when we sat down with you [Don] and we had focused discussions
• The orientation, with the powerpoint, doesn’t make sense…there’s no context for us yet.
• Feedback, other than from the group, was not as consistent as it could be. I submitted everything, as prescribed, but I was not always sure it was what was needed.
• The manual should suggest the flexibility inherent in the process
• More leading questions and examples built into the manual…guide me a little bit more…particularly for the analysis sections…there needs to be more clarity about the difference between the sections preceding the Analysis, and the analysis itself.