Creating Rich Transcripts for Career Activation

Around the world, many people have questioned or criticized the impoverished traditional transcript. A 2017 report from the Higher Education Quality Council of Ontario reported, for example, that the “Current credential and accreditation system does not serve students well.” The fact is that institutions should be embarrassed by the standard transcripts they have been issuing, unchanged for a century, and students should demand better.

Some colleges have launched programs to revise transcripts so they represent students’ co- and extracurricular experiences, attaching to the standard transcript some electronically provisioned add-on that shows students’ activities outside their coursework. Unlike the traditional transcript, where most courses are titled in terms of their content, such supplements often emphasize skills. Other tweaks on the standard transcript have supplemented the abbreviated course titles with some other information or, in the case of electronic transcripts, links to student production or a course website.

In alignment with those efforts and other broader initiatives to assist students in the transition to careers, the political science department at the University of British Columbia, where I am a professor, has worked to develop for its graduates a supplementary “rich transcript” that includes:

• The student’s courses’ full titles;
• A word cloud built from the instructors’ detailed course descriptions for their courses (not the generic calendar descriptions);
• Aggregated statistics for each student on the number of writing assignments, pages written, peer reviews, oral presentations, hours of group work, research designs, primary research, internships and service learning; and
• A list of 23 skills showing in how many of the student’s courses each skill was a key learning outcome.

Every student deserves this kind of report on their learning and skill development. In this article, I describe the five-year process that produced the rich transcripts for a pilot project for graduates of one major at my university in hopes it can inform others to consider how they might improve their students’ transcripts, as well.

In our case, it all began when I started thinking about university transcripts and wondered if students could be given a better summary of their learning experiences. Those transcripts had the official stamp of the institution, some overabbreviated course names and, of course, grades -- and nothing more. Not a great parting gift for students who had worked so hard, had a variety of learning experiences and developed a wide range of skills.

Fortunately, it was around this same time that the faculty of arts at the university was defining program learning outcomes and building a data mart including enrollment and course information. In addition, the career office was looking for more tools to help students articulate the skills they develop pursuing their degrees.

So the time seemed right to propose a pilot project: the Course Characteristics Census. The idea was to inventory our courses to find out how the instructor had designed the learning, what students were being asked to do in the course and what skills those instructors intended students to develop. Among the many benefits, we imagined that the course data could be aggregated by student to summarize each person’s learning experiences, academic output and skill development.

Gathering the Data: A Course Inventory

We started by examining how we might best collect the relevant information about each course. The natural place to start was to gather syllabi and code them. The dean’s office and other department heads suggested that approach because they thought the syllabi would contain a clear indication of the learning activities and skills to be developed in the class. Moreover, faculty members wouldn’t have to be asked to do anything.

But it soon became clear that the syllabi were extremely varied, with only a few providing the information required, especially on learning objectives in terms of skill development. Some were even inaccessible because departments had not been required or encouraged to archive their syllabi, and some former instructors were unreachable.

We decided to try a different approach to gather a broader set of, and more reliable, data. In consultation with experts on pedagogy and assessment and, of course, the department itself, we built a questionnaire to gather detailed information on characteristics of courses: learning outcomes defined as skills, teaching modalities, learning activity structure, assessments, work time expectations, technology use and special features like community service learning or primary research.

But unlike an ordinary internal university survey that might have value with, say, a 20 percent response rate, we realized we would need a 100 percent response rate if we were going to be able to aggregate the course information for each student covering all of their learning in the political science major, without any gaps. In other words, we would need to conduct a census, not a survey.

The final questionnaire typically took 20 minutes per instructor per course. Obviously, we didn’t ask instructors to repeat it if they had not significantly changed their course design from year to year. Still, it was a tremendous challenge getting information on every course taught in the department by full-time and part-time faculty over the course of four years. We only succeeded because we had the strong participation of the department chair and I was able to twist my colleagues’ arms in a way that only a colleague can do. Even then, we were missing a few courses, particularly those that sessional lecturers and postdocs taught.

An academic unit trying to gather this information should consider the time lag from beginning to collect the data to the point when the rich transcript can be issued for the first time. It is theoretically possible to start by trying to gather the data back five years to cover the course history of the students about to graduate. In our case, we gathered it over four years and then did the data processing and issued these rich transcripts. Even then, at the end of those four years, we had to go back to fill in some missing responses and try to collect data on courses as far back as seven years.

Lesson 1: To provide rich transcripts to students based on instructor-provided data, you must obtain full buy-in from the department chair to use all methods to ensure that every instructor fills out the questionnaire for all of their courses over a few years.

Lesson 2: Different strategies are required to ensure the participation of different types of instructors. We had to write a census response requirement into the contract of part-time instructors. And we had to tell full-time faculty repeatedly that this was mandatory because, without full participation, students would not get a complete rich transcript and any reputational or student-satisfaction benefit to the department would not be realized.

Manipulating the Data: Creating the Transcripts

We needed course information for 237 distinct course-instructor pairs. That is, if both Professor Apple and Professor Plum taught POLI367, we needed data from each of them. But if Professor Apple taught the course four times, we only needed that one data point from her unless she significantly changed the course design. Our goal was to match that course-instructor data to the enrollment histories of two cohorts (550 graduates) from our political science major.

We first defined what indicators we wished to include in the rich transcript -- what features of the courses could be aggregated and be most useful for students. Broadly, those were output, experiences and skills.

Lesson 3: Define what indicators you want to surface for students and for the program before composing the questionnaire. Consider how the data will be aggregated, and formulate questions to facilitate easy data processing.

We settled on these items as the content of the rich transcript:

• A list of the student’s courses with the real substantive titles, not the official calendar titles;
• A word cloud from the course descriptions written by the instructors, not the calendar description;
• For each student, a tally of their output: the number of writing assignments; written output in pages; number of peer reviews and oral presentations; hours of group work; and the number of some enriched learning activities like a research design, primary field research, community and global service learning, and so on;
• A tally of the number of courses for that student in which 23 different skills were a key learning objective, including common ones like “Write clearly and effectively” as well as others like “Develop or clarify a personal code of ethics” or “Perform mathematical or formal/logical analysis.”

After we settled on the content, we had to design the final product. The approach was to create a two-page infographic-style document. A designer was brought in to work with our data scientist who would be batch-producing the rich transcripts in Tableau, a data science and data visualization tool.

The result is a document that looks official as well as engaging and is suitable for both print and screen. We designed it to be a document that a reader can digest in just a minute or two but with enough detail that a student can point to it in a job interview and provide examples of their output, learning activities and skill development.

The team’s data scientists merged the course and enrollment data and then produced PDF files for each student. Finally, the PDF transcripts were attached to individual emails to students through a mail-merge operation.

Surveying Student Reactions

We sent a follow-up survey to the students to gauge their reactions as well as to prod them to look at their transcript again and use it in their career-development activities and job searches. We asked students if they saw value in their rich transcript and how they imagined using and sharing it.

All but a handful of those who responded had opened their transcript, and, in fact, two-thirds had opened and reviewed it more than once. The mean overall rating of the value of the transcript was four on a zero to five scale, while the rating for “usefulness for career” was slightly lower, at 3.4. When asked on a zero to 10 scale if they would recommend that other colleges offer this to their students, three-quarters gave a response at seven or above, with 30 percent giving a 10.

About two-thirds of the students had already shown it to friends or family when we followed up with them a few weeks after they first received it. Sixty percent said they thought they were likely to show it to a prospective employer at some point.

Many of the student reactions spontaneously referred to the benefits of their rich transcript for career activation. One person said, “I was originally hired as a panel administer for a broadcast measurement company and was promoted in March to a position that requires a lot of writing and teamwork. For my job application and interview, I was able to use the specifics in the transcript to identify what I did during undergrad. Thank you and the department for putting it together!”

We received suggestions for further development of the rich transcripts. One of the most common was to allow students to access the report online as they progress through their degree, so they can see what skills they are accumulating. Some even suggested that the information about learning activities and skills development be available while students are selecting courses so that they can “fill gaps” in their skill set by choosing courses with particular learning activities and outcomes. We intend to pursue those suggestions. We will also reach out to employers for their views about the value of the current rich transcript document and any suggestions for improvement.

Future Directions

Going forward, we will now systematize and streamline the data-gathering and production processes for scaling up to multiple departments. We’ll also work to integrate the rich transcripts with efforts to support students and alumni in their career preparation and job search by building a set of accompanying materials to help students understand how to use their rich transcript as they start their career journey.

During this consolidation phase, we will elaborate a process framework, providing a kit that any department can use to conduct a course characteristics census and produce rich transcripts. The process phases will be, roughly:

• Definition of desired rich transcript content;
• Development of the instructor questionnaire through consultation;
• Questionnaire administration, including timing;
• Data manipulation;
• Joining course data to student enrollment data;
• Rich transcript visual/graphic design;
• Production of rich transcripts;
• Distribution;
• Integration with student and alumni career resources; and
• Evaluation

Over the long term, we should also be able to first, issue interim rich transcripts at the end of each semester so students can track their learning experiences and skills development, and second, make the course census information available to students as they choose their courses. Students could then more consciously and accurately build their own program to acquire and develop a wide range of skills -- and perhaps deep competency in one or two.