A Bird's-Eye View of Online Program Quality

Laura Dicht has since last November served as manager of online program quality in New York University’s Tandon School of Engineering. Before that, she spent two years as an instructional designer for the school. In the nine months in her new role, she’s gathered some newfound insights into the online teaching experience, shaped new processes and policies for improving the school’s online output, and set a precedent for a more rigorous, structured approach to online development.

Dicht oversees a portfolio of 65 online courses as well as a course development team that includes four instructional designers and three educational technologists. She’s working on building relationships with instructors and planting seeds for future efforts to refresh an expanding number of existing online courses. She’s also piloted a program, soon to expand, in which full-time instructors serve as “course directors” who provide feedback and tips to adjunct instructors.

“Inside Digital Learning” asked Dicht a few questions about her job and her impressions of the online learning landscape at her slice of the institution. Her answers offer insights that could be helpful whether you’re a faculty member struggling to teach online or an administrator hoping to create more formal processes.

Q: How did this position come about?

A: Our new associate dean for online learning, Nasir Memon, when he started in our department about a year and a half ago, his big interest and what he really, really cared about is maintaining quality. He always talks about what will set us apart -- our courses are high quality, our faculty are high quality, we provide a high-quality learning experience.

Q: The word “quality” has many different definitions depending on how you ask. What does it mean to you?

A: This is an area which took our department some time to define what we meant. It is a bit of a buzzword, and it can mean a lot of different things. It can mean high production value, really great modules. It can mean the efficiency of our teaching. What we really cared about is making sure that the online environment, the online classroom, felt really welcoming, that a sense of community was created. Students knew what was expected of them, knew how to achieve the goals of the class, felt a good sense of community in the classroom, felt they could contact their instructor. Just the overall experience was really enjoyable and high impact for the students.

Q: What previously underserved needs does this new position address?

A: When I was an instructional designer, we spent a lot of effort working with new faculty to develop high-quality course modules, high-production-value materials. But I always kind of wondered, ‘OK, we developed this course -- what now? How do we make sure that the faculty is successful teaching this course when we don’t have that same kind of hands-on close relationship with the faculty anymore?’ In my transition to this role, it has been really important to try and develop a relationship with our existing faculty. For courses that are not newly developed, I am the immediate contact for faculty who maybe did not necessarily develop their own course, or faculty who developed their own courses not through our course development team. I can provide them with training and support through that process in case they have any questions, or I sometimes provide them with suggestions or tips or ways they can go about doing things, because I know our LMS really well, inside and out.

Q: Besides answering faculty requests for consultation, what else falls under your purview?

A: My responsibilities can be a little overarching. Right now we are getting ready for the fall semester. A lot of what I do is I track the enrollments -- I work with the different departments to make sure our courses have high enrollments or meet our enrollment criteria for running courses. I do outreach to our faculty to let them know when their course sites are available to set up and if they need any help with that. I let them know about tools and technologies at their disposal, and I provide training to use some of those tools as needed. That’s what I’m doing actively as we get ready for the fall semester.

I do periodic check-ins with our faculty -- I schedule some brief phone conversations just to find out what’s going well, what’s not going well, if they need our assistance with anything. I look to see if there’s an overarching need that has been acknowledged from our faculty. For instance, we had a bunch of faculty express that their TAs didn’t know how to be TAs for an online course. So we developed a training program, which we’re launching this fall. Another example of that is an online student guide, so students can more readily be acclimated to being students.

In the spring, when I did this series of phone conversations, I noticed that a lot of these issues faculty are dealing with were not unique. We could as a department take measures to address this across the board. When I check in again in the fall, we’ll see if these measures had any impact.

We also pull out this information from student evaluations. We did surveys to our students to gather some of the student perspective. I also read the final course evaluations to see what students would like to see, what worked well and didn’t work well.

Another part which we have only started to do, and will be doing much more in the future, is leading our course refresh efforts. That is the fully developed courses that need to be revisited. That will be a much bigger role in the future as a lot of our full course developments age a little bit more. A big part of this is working with our faculty to create a culture of continuous improvement of courses and course materials.

Q: What have been the biggest challenges in this position?

A: Because it is a new role, when I started it was not very well defined. It really took me a lot of reflection, and a lot of looking at our courses, talking to our faculty, figuring out where the need areas are. It’s still a work in progress. I’m still figuring it out as we develop this role. We’re less than a year in. I’m looking forward to what we can accomplish in the year ahead.

Q: You’re getting a widescreen look at what faculty members experience when teaching online. Have you noticed any common threads tying their experiences?

A: A big challenge with our online faculty has been the idea of community building. It’s a lot more natural in the classroom environment to be able to start discussion among their students because they’re literally right there in front of them. Similarly, faculty have noted that it’s much harder to assess learning because they don’t have that visual eye-to-eye marker of, ‘I can see my students seem to be getting it.’ Our faculty have noted that they have had to evolve their approaches semester to semester -- they’ll try something, realize that didn’t work that well, have to do something different next time. A lot of faculty do weekly webinars, but it’s not as natural as getting up in front of the classroom. That’s something we definitely work on.

Through these conversations and talking to various faculty, I can see this is what one person’s doing, it seems to be working well, so sometimes I suggest that to faculty who are struggling a little. We’re trying to build a knowledge-sharing network among our faculty to address some of these issues.

Q: How about among students?

A: The students are sometimes missing that sense of community and engagement, too. We try and make sure our class incorporates some sense of discussion and/or group work to overcome those challenges, to make sure that students don’t feel so isolated.

Q: Faculty attitudes toward online education vary widely. How do you reach instructors who are less interested in working with you, or more wedded to their traditions?

A: We do have a lot of faculty who are really devoted to being good educators, and really willing to learn how to improve their teaching methodology. And then we have some who are a little more resistant or maybe have been teaching online for a very long time, and maybe are less willing to take feedback and change their style. We really just try and meet them where they are, find some kind of middle ground when we can maybe push the needle. We acknowledge that we can’t take a faculty [member] who’s teaching one way and completely make them do something different.

A lot of our faculty do have their teaching style, but we work with them and we are accepting that that is their style. Our associate dean is an academic, a computer scientist. He has a close relationship with a lot of faculty. Sometimes we use him as a very valuable resource in speaking to faculty as a fellow academic, not as an administrator.

Q: How does this job differ from being an instructional designer?

A: I would say as an instructional designer, I created a lot of tools that I still utilize in my current job. There were a lot of checklists and rubrics that I created for myself and for my own processes. But the main difference now is that I get to create resources and guidelines for our whole course development team to help them look at the course more holistically, make sure they're including all the necessary components for a high-quality course. In addition to working with faculty in ongoing maintenance, I work with the course development team to make sure all the new courses that are being developed are high-quality courses and that all of our course materials are meeting high quality standards.

I think it has been a good experience being in this new role. One of my first things I did was acquaint myself with all of our online courses, look very closely at course materials, how faculty were teaching our courses, help to frame what I was doing from there. I think it’s been a good learning experience. It’s been a big shift from being an instructional designer because I worked very closely with a small group of faculty, and now I’m working in a more broad capacity with all of our faculty, and have a lot more oversight into the big picture of what’s going on in our courses and programs and with our student body.