Tagged ePortfolio

Interview: Bret Eynon, Joseph Ugoretz, Laura Gambino

Editorial Note

This interview took place on July 7th, 2016 at LaGuardia Community College in Long Island City, NY. The transcript below has been edited for readability and for length and the 90-minute conversation has been edited down to a 66-minute recording. For those who may want to skip to specific topics in the interview, we have provided section headers and time markers to allow for more efficient listening. However, we also wanted to give our readers and listeners the opportunity to experience both the recorded interview and the written transcript in their entirety.

Link to archived content on archive.org

Getting Started: Launching ePortfolio at CUNY

Dominique:
I’d like you to start with a brief introduction about your institution and how you’ve used ePortfolio–your campus ePortfolio history.
Bret Eynon:

I’ve been at LaGuardia since 2000. In 2001, I helped Paul Arcario write the first grant that supported our ePortfolio work. When we got that grant, we first focused on pedagogy, influenced in part by the work I’d done with the New Media Classroom Project[1] and the Visible Knowledge Project[2]. In those projects we emphasized linking constructivist pedagogies with multimedia authoring as a way to help students take ownership of their learning and develop their identities as learners. As our ePortfolio work at LaGuardia grew, we started taking on assessment. We did little around assessment for the first eight years. Around 2009, as our Middle States visit approached, Paul asked us to strengthen our work with the assessment side of ePortfolio, so that grew in importance.

The development of our new First Year Seminar has in some ways shifted the balance back to pedagogy, and has added new emphasis on advisement. Each year 10,000 to 14,000 LaGuardia students are active on their portfolios. ePortfolios are used in some way in most credit programs, and to some extent in Adult and Continuing Ed.

In 2008 we held our first national conference. That same year we launched the first of six years of grant-funded work through our Making Connections National Resource Center. We worked with 70 to 80 colleges and universities nationwide around ePortfolio pedagogy and practice and built the Catalyst for Learning site (http://c2l.mcnrc.org/).

Joseph Ugoretz:

When I came to Macaulay in 2007, I found a really unique institutional structure. It’s a consortial program where the students are on eight different CUNY campuses. We don’t have our own faculty. We draw faculty from the different campuses. We don’t have our own curriculum except for four seminars. So it was an unstructured and flexible setup that required an unstructured and flexible ePortfolio solution. I came with the same orientation Bret mentioned from the Visible Knowledge Project; I was focused on pedagogy and on reflection for students, and especially on students taking ownership of their own learning and being able to design the representation of their learning. And that was a nice match for the Honors College. So we launched with all our incoming freshmen in 2008, 500 at once, and grew from there.

What has characterized ePortfolio at Macaulay is this flexibility, opening to many different pathways. It’s rhizomatic in a way, because we weren’t able to impose a structure or a plan. As a result, we’ve seen the flowering of things we didn’t expect and that may not even fit a strict definition of ePortfolio. “What is an ePortfolio?” is an interesting question that we ask all the time.

It’s been fun to see the way that students have taken hold of the concept in order to create different approaches: literary journals, travel blogs, curriculum plans, career placement portfolios, poetry collections, etc. The other exciting thing is the number of students who use the platform for group or collaborative work. There’s a large number of class ePortfolios or small groups within a class, or clubs, or student groups that put together a portfolio based on their interests that are shared. Many voices can be included, with a consensus on overall design.

I’ll say one more thing: From the beginning, we felt that visual graphic design was critical, that students express their individuality that way, and that they make the portfolio itself be a representation. A paper portfolio tends to be a black binder. But then we see someone putting stickers on it. They want it to be more than a container, to be a thing that has meaning in itself, as an object.

Laura Gambino:

I’m at Guttman Community College, which opened in 2012 as the New Community College. I joined the college two weeks before we opened. One of the great opportunities we had was to rethink the community college experience for urban students. We built on a lot of the great work that we saw at LaGuardia and Macaulay and elsewhere across CUNY. And one of the things we did was to implement ePortfolio at scale, right from the start. So on the second day of our Summer Bridge program for our first cohort of students, every student created an ePortfolio.Because we’re a little newer, we focused on all different aspects of ePortfolio at once. So we’re very focused on the pedagogy and having students create a learning portfolio that spans their entire academic experience from Bridge right through commencement, connecting their curricular, co-curricular and experiential learning activities that take place out in the city. It’s also the primary vehicle for our assessment of student learning.

All students submit their portfolios at three milestones: the end of Bridge, the end of the first year, and the end of the second year. And all faculty engage in assessment where we look at not just the artifacts of student work in the portfolios, but also the reflections that go along with them. That’s the most engaging piece for faculty: the reflections. ePortfolio is also our course content delivery system. (We use Blackboard only in limited ways.) Most faculty maintain professional ePortfolios. Our student clubs and student government use it. But we also use it as an institutional repository– course teams share assignments and materials through ePortfolio. So, we’ve had that same experience where its use has blossomed in ways we didn’t foresee. But it’s safe to say ePortfolio is central to our learning culture; and that presents great opportunities and great challenges.

Joseph Ugoretz:
Laura makes a good point. People are hungry for ways to bring things together and publish and share them. When you give people something that allows them to do that, they seize it and run with it.
Laura Gambino:
In Bridge, students create a group ePortfolio. They’re doing a group research project, and they construct a portfolio together, synthesizing different components into one, collaborating and sharing it. And it really engages them. It doesn’t just introduce them to ePortfolio; it engages and shows them what the Guttman learning experience will be like.

Pedagogy first, platform second

(11:50)

Bret Eynon:

I want to pick up on a couple of things. First, Laura and Joe’s emphasis on collaboration shows how our practice has changed over the past 10 to 15 years. The notion of a social pedagogy for ePortfolio has transformed our notion of portfolio practice. When we first encountered ePortfolio, it was private and individual. People were wary about going public. That has really changed. We now know that the more portfolios are used as a site of exchange, communication, conversation, and collaboration, the more powerful they are. I also want to comment on Joe’s point about visual expressiveness. I’ve long been a believer in the importance of visual design in ePortfolios, and have seen that as crucial for student ownership. But it’s now more difficult than it used to be.

Our first platform required all students to learn HTML authoring. They created visually stunning portfolios, but it took a huge amount of work to train them, and that was a barrier to broad adoption. So we moved to a platform that emphasized ease of use and promised the possibility of interaction–but that required templates and came, to some extent, at the cost of visual expressiveness.

There are complex factors around the platform, related to visual expressiveness, ease of use, interaction and organization for assessment. As a result, there is no such thing as the ideal platform.

Laura Gambino:
When we speak at conferences, the question we always get is: What platform should my campus use? As Bret was saying, there is no perfect platform, and there will never be a perfect platform. ePortfolio is so much more than that technology. But that’s what people get stuck on.
Joseph Ugoretz:
The question of which platform to use is sort of a “how long is a piece of string” question. The point is to start with the philosophical goals, and the pedagogical goals, and really think those through. And then see what kinds of technologies are going to best meet those needs. We’re all saying the same thing here, that the solution that’s best for one college is not best for another.
Bret Eynon:
I’ve seen an interesting tension as institutions take on portfolio. To understand your goals for ePortfolio is difficult without experience. I’m a deep Deweyan and I believe in experiential learning. But you can’t get the experience you need without a platform. So it’s a chicken and egg conundrum. I guess the solution is to jump in some place and be willing to adapt.
Joseph Ugoretz:
One of the things I admire about LaGuardia is that you started with a model and weren’t afraid to radically alter the approach as you learned. And I think that’s really critical. We don’t have to commit everything to one approach or one platform and then stick with it. But we do have to have something to work on. And then we can learn from it and see what the next best step will be.

What’s unique about ePortfolio as an educational technology?

(18:18)

Dominique Zino:
What exactly have you seen change in terms of the technology? What have been the most beneficial changes? Do you see other changes on the horizon in terms of platforms and technology?
Laura Gambino:

ePortolio technology is different from other educational technologies or learning management systems because of its ability to span courses, semesters, and co-curricular spaces. It can bring together the entire student learning experience. Most educational technology is course-centric. But a student can create an ePortfolio and carry it with them across all of their learning experiences, and see the way those experiences fit together and integrate. And they can reflect on the integration of all of those different pieces. As far as I know, no other software or educational technology is designed to do that. That, to me, is what it makes it so unique and so valuable to the student learning experience.

Joseph Ugoretz:
And I’ll double confirm that. It’s always infuriated me that our learning management systems think of learning as something that starts in September and ends in December, and then you can never see that course again ever. That’s not the way that people learn. It’s not the way people want to learn. Learning goes across courses, across time, and across institutions.
Bret Eynon:

Over the last fifteen years we’ve seen growing recognition, across higher education, that we must help students build their abilities to integrate their learning. Fifteen years ago, I don’t remember anybody talking about that. But now it’s a common discussion. We were just down at the US Department of Ed for this big national symposium on the future of higher education, and everyone in the room was talking about it: how do we help students connect their learning? How do we help them use what they’re learning? How does education add up to more than a set of discreet, isolated experiences?” It’s a big change. That’s one reason why I think the moment for ePortfolios is arriving. To some extent, at our three schools we’ve been ahead of the curve on this. But for most educators, the need for the portfolio, the need for integration, is only now becoming clear. That puts us at a really interesting moment. It’s not to say that the ePortfolio will solve all problems–the challenge of integration is not just a matter of the software. It even goes beyond the pedagogy and practice of individual faculty members. Integrative education requires a different level of institutional integration and collaboration, a common envisioning of the educational project.

We have a long way to go on that front. But integrative ePortfolio technology can help. And, as more educators recognize the need for integration, I think our integrative portfolio practice can grow increasingly sophisticated and powerful.

Laura Gambino:
One of the biggest technological changes is related to what Bret was talking about earlier, the rise of social pedagogy. The ability to comment on other folks’ portfolios, to have conversations in portfolios, and connect to other social media–we didn’t see that ten years ago. So that’s emerged and it’s great because it has helped facilitate a new and effective classroom pedagogy.
Joseph Ugoretz:

As the web has developed, we’ve seen a move to curation, pulling in lots of different sources, evaluating them and mashing them up. ePortfolios are a place where that can happen. That includes curating videos and images quickly and easily. Students want to be able to put something up right away and maybe reflect on it later. Portfolios are places where we can, through comments and interaction with peers and professors, push them to go beyond the first response to the deeper learning.

Laura Gambino:
Another facet that’s evolved is the assessment system. Think about assessment systems ten years ago. Now, in many platforms, you can easily capture snapshots of student portfolios, store them for as long as you want, distribute them to different faculty, score them with rubrics, and get the data easily and quickly–wow, I’m sounding like a real geek! That’s a huge advantage.
Joseph Ugoretz:
We’ve moved to a broader and deeper view of assessment. The portfolio platform we developed at BMCC focused on filling in boxes for specific tasks. You get the box filled in, boom, that’s a positive assessment. That’s no longer an interesting form of assessment for anyone–if it ever was.
Bret Eynon:

I’d mention two other technology developments. One is the intersection of badging, and portfolios. Badging seeks to capitalize on the motivational skill-building, level-conquering processes found in digital gaming. It can also create discreet pathways into the portfolio for external viewers. The badge lives in the portfolio, and provides a way to spotlight a particular part of the portfolio, a salient skillset, let’s say, for a particular audience. We’re now experimenting with this at LaGuardia. In the digital learning environment, students have the opportunity to “learn everywhere.” Take a course on another campus, or online. The key is, don’t leave it disconnected. Connect it with a common repository of learning. I think that’s part of what the portfolio is going to offer, that place of intersection, where diverse learning experience is connected. Badges and ePortfolios could be a part of facilitating and representing that integration.

The other thing to mention is the connection between portfolios and analytics. Clearly, learning analytics and algorithmic analysis of student data is upon us. But as of now, it’s happening at a pretty superficial level–how many clicks here and how many clicks there. That doesn’t actually tell us very much.

I see the potential for a more interesting form of analytics, linked to portfolios. Doing a machine reading of portfolios, and surfacing data patterns could support a more integrative understanding of analytics. That could be a very interesting area to explore in the future. But we have a ways to go on this.

Asking Students to Make Choices about their Digital Identities

(31:05)

Joseph Ugoretz:

My friend Patrick Masson says “I’ve been enrolled in this really fascinating MOOC for the past 20 years. It’s called the internet.” People don’t necessarily go to institutions to learn anymore. They don’t necessarily go to sources that the academy decides are credible. Many of our students go to YouTube as their first source of information. And often they learn more from YouTube than they do from college. So the important skill for students going forward is learning the right kinds of questions to ask, and how to evaluate answers; how to look for sources, and how to judge sources. That’s really critical. If I’m in a science fiction mind, the idea of intelligent agents assisting that process is both brilliantly utopian and a bit frightening.

I don’t want a Siri to give me the answers based on what I’ve always found before. Because the loss in such directed searching is the serendipity and discovery we used to get by browsing library shelves. For our students, the skill to learn is how to ask questions and how to judge sources.

The other skill is becoming aware that these are choices. That what Google gives you is not a transparent, unmotivated choice. Help them think about who’s controlling their information and how they can control their own information.

When we start with our incoming first year students and introducing portfolios, we do an exercise about digital identity because students come to us, 17, 18 years old, and they’re nervous about being out there. And portfolios can help them learn that the best way to have a positive digital identity is to make your own digital identity. You don’t keep yourself off the web. That can’t be done. But you make yourself searchable in ways that you approve of by putting yourself out there in ways that you approve of.

Laura Gambino:

We do the same thing, and we ask: What do you want that identity to look like? How might your academic identity be different from your Facebook identity or your Instagram identity? Your ePortfolio is the place where you get to design and construct and develop who you are as a learner and a scholar. And that can be a powerful thing to students, and for many of our students. And that’s a key part of the portfolio: they own it. It’s not institutionally owned. They’re the owners. Back to that idea of customization and ownership. The goal is helping students not just own the portfolio, but become owners and agents of their own learning process.

Bret Eynon:

We mentioned digital identity and learning identity. Both are increasingly salient needs for today’s students. For me, the idea of digital identity goes back to the early ’90s, wanting students to be more than consumers of knowledge, wanting them to be creators of knowledge; more than consumers of the web, creators of the web. That idea–linking constructivist learning and multimedia authoring—was part of what was behind the portfolio. Now we also understand that your identity as a learner is crucial to the learning process. We’ve seen a broad shift in our understanding of learning: first it was just content acquisition; then it was cognitive skills development; and now we increasingly understand that affective dimensions are involved as well. The whole person is engaged in the learning processes.

As students learn, they craft a new sense of self, engaged with the world, engaged with the knowledge of the world. We now realize that we must help them develop their capacity to do that throughout their lives. That’s increasingly clear as a goal. That’s another reason we may be on the cusp of an ePortfolio moment.

ePortfolio practice is well suited to address this need. ePortfolio practices can help students engage in this critical process: making choices about who I am, who I want to be, what I need to know, how to learn it. How to become an adaptive critical thinker and always coming back to that sense of self: the deeper question of who I am and how I take ownership of myself.

The scholar who helped me think about that is Marcia Baxter Magolda, who talks about purposeful self-authorship. All three of those words are important: purposeful, self, and authorship. That concept helped me think about what is going on at the most sophisticated level of portfolio practice.

Maybe it’s a LaGuardia thing, or CUNY community colleges, or community college more broadly; or maybe it’s CUNY, I don’t know. But when you work with first-generation students, you see how the process of self-authorship is crucial for their success. It makes a big difference for students. There is very solid evidence on this.

Our LaGuardia data–and data from other campuses–shows that the ePortfolio process has a significant positive impact on retention and student success. Other factors matter, too: pedagogy, peer mentoring, etc. But I think that the process of self-authorship is a powerful factor for our students.

Joseph Ugoretz:
It’s a good point. We’ve all had the experience of being at a national conference or at places where students have a different level of home education, experience with college, academic skills and privilege in the world. It’s sometimes hard for people on other campuses to understand the reality of CUNY and CUNY students.
Laura Gambino:
When we introduce ePortfolio to students, as I mentioned before we intentionally use that language: This is the space to construct and develop and create who you are as learners and scholars. And it’s powerful, because our students often don’t see themselves that way. The ePortfolio becomes the space for them to do that.

Specific examples of faculty and student work

(40:59)

Dominique Zino:
Can you talk about a case where you’ve seen this sense of self emerge? Maybe in a bridge program, or maybe in a capstone course, where a student has done something with portfolio that fulfills this vision?
Bret Eynon:

I’d point to our Graduation Plan. In our First Year Seminar, we use ePortfolio to introduce all students to a process of self-examination, self-assessment, and purposeful planning. It starts with simple steps–identifying and reflecting on your goals in life, your values, strengths and interests. Then we ask them to connect that to real choices, thinking about career, transfer, and ultimately, “What courses should I take next semester? How am I going to move forward in this building my education in concordance with my goals and values?” The ePortfolio template gives faculty and peer mentors a set of prompts to use with students. It’s a practical exercise in purposeful self-authorship. Done well, this process is very powerful. And some faculty go further. In Natural Sciences, after doing the Graduation Plan, Preethi Radhakrishnan has her students do a video of a digital story. And the videos students create are incredible. In their own voices, students connect their education to their new sense of self. This opens the opportunity for a recursive examination, as students revisit this process, looking at how they’re changing over time.

Joseph Ugoretz:

So while we were talking about this, I looked up one student’s ePortfolio, called “The Utopia of Daniel” (http://macaulay.cuny.edu/eportfolios/utopiaofdaniel/). So Daniel is the student. If you look across the top, Daniel has his categories for what is in his portfolio, and it’s Home, In The News, Cars, College, Gaming, Technology, TV and Movies, and Random. And so you notice, college is one tiny little piece of him and his learning. And then he’s got this rich list of subcategories for each one, and experiences that led him to internships. Daniel was interested in cars and went to Texas to visit the Delorean factory. And boom, he got an internship with Delorean, in part because he had been thinking about how his interest in cars matched his interest in filmmaking and digital production and computer science.

Looking at his ePortfolio you get a real picture of Daniel as a person. Lots of movie reviews, and technology reviews, like which external hard drive is the best for video editing. And he gets questions from people interested in buying technology.

Here’s one: “How to Upgrade Your Ram on Your Mac Book Pro: Super Simple and Dirt Cheap.” That’s not something that he learned in school, but it connects to everything he’s interested in learning and doing. The idea is to collect and integrate all these different pieces of student learning and then make that available to a wider audience, so that you can engage with that audience. That’s been a key advantage for our students.

Another example is a student who was assigned to write a review of a play without seeing it just by reading other critics’ reviews. She did it, she posted it, and the playwright saw it, and left a comment saying, “Wow, I’m really interested in your ideas. There will be two tickets waiting for you at the box office. Please come talk to me afterwards, and so we can discuss what you thought.” So that changes the learning–it’s very different from just learning in an insulated way. It makes the learning process permeable to the outside world. That’s very powerful for students.

Laura Gambino:

I’ll tell you two quick stories. First, I had student a couple of years ago whose initial goal in Bridge–he was very, very clear–was to get his degree and become an auto mechanic. That’s just what he thought he should do, and what his family thought he should do. He took our First Year Seminar, our Ethnographies of Work course, and learned that he could be a successful student. And we got him to delve into his goals a little more, to do some career exploration. Meanwhile, he started to share his drawings in his ePortfolio. We started to see them, and then I saw him doing them in class too. And we started to talk about that. Students in his class and his advisor and I encouraged him to talk to someone in the art field and what his possibilities could be.

Over the course of the year, as he was doing his journaling in his portfolio, and getting all this feedback, he did this research project in his portfolio about what art careers are. And he realized that art was something you could actually pursue as a career. He’s now finishing his Bachelor’s in Graphic Design.

And here’s a second story that’s a little different. Back when I was at Tunxis Community College in Connecticut, I taught an Intro to Programming class, a very tough class. One year I had a very quiet young woman who would sit in the back. She never really spoke in class. It seemed like she wasn’t sure if this is what she wanted to do. In their ePortfolios, I had them do reflective journaling on their design process; I had them use the portfolio to share with each other and comment on how they solved particular problems.

I remember looking at this student’s portfolio–she had changed the color; she put up all this comic artwork. It was one of the most beautiful portfolios I‘ve ever seen. She’d done this all on her own.

So they all did their group portfolio review process. And I come in the next day, and there are all of these students sitting around her going, “Show us how to do that.” “How did you do that?” “That is like the coolest thing I have ever seen.” And you could see her mindset shift: I can do this, I can be one of them, I can hold my own, I can be a computer scientist. It was an amazing experience that helped craft her sense of self and her sense of purpose in this field. And I will never forget that moment walking in and seeing that, and then just watching her grow and move through the program all because of what she had done on her portfolio and the social interaction it supported.

And I guess there’s ways you can replicate that without the portfolio. But here, the portfolio was the vehicle that helped her craft her identity, because she had taken ownership and put that sense of herself in her portfolio. That helped her see how she could connect to the work she was doing, connect to a place in the field.

Key ePortfolio practices: making, sharing, integrating

(51:13)

Dominique Zino:
What pedagogy produced such rich experiences and products,? If you had to name two or three tenets of ePortfolio pedagogy, what would you point to?
Joseph Urogetz:

That’s tough for us because our ePortfolios are 100 percent student driven, student motivated, student designed. It’s sort of build and they will come. Or let them build it, and then they’re there. It’s a respect for the individual student as a creator and as a person and as a learner. And whether that happens in the classroom or somewhere else, the message that comes across is that you can make something here. I think one of the things that make us human beings is that we like to make things. It’s like if you give a child a set of Legos, they want to make something with it. So it’s making that set of Legos available and usable. And then maybe showing them a few examples of what other people have made.

Laura Gambino:

I would add reflection to that, the reflective pedagogy. To me, that’s where students articulate their learning and see it for themselves. One of my students talked about this in their portfolio more eloquently than I can–I wish I could quote him here– but he said what he loved about ePortfolio was its ability to combine process and product. Doing the work is interesting, but reflecting on it and thinking about how he got that product and the process he went through to get there, that’s much more interesting. Carol Rodgers and others say reflection is where learning takes place. We need to stop and think and to articulate our learning process. Portfolio practice builds the ability to do that, to reflect and share reflection. Reflection is not always a solitary process because we need to reflect in community and share as well.

Joseph Ugoretz:
Like Laura said, it’s not just sitting alone and reflecting on your work. It’s saying, “Bret, let me show you this,” and then Bret asking questions, and then me answering these questions. And that’s really where a social pedagogy for ePortfolio motivates and multiplies the reflection.
Bret Eynon:

The core of what’s pedagogically valuable about portfolios is the way it makes learning visible. It makes the learning visible to students themselves, supporting the reflective process. It also makes the learning visible to peers, for a social pedagogy. And it makes the learning visible to faculty for individual and collective assessment. Engaging in this process collectively, using ePortfolios can help faculty better understand that what students learn in my course connects to what they learn in your course, and how it all adds up. Finally, by making learning visible, it opens the way for integrative pedagogy. That’s the frontier we’re working on. Seeing how students learn over time, across courses and disciplines, how what’s happening in one area connects to another. Integration between the learning experience and the evolving sense of self.

Integrative pedagogy points to all those connections. Of course, it’s a deeply challenging pedagogy. It’s an unfamiliar pedagogy, in that it asks faculty to step outside their area of expertise and think about all these other things happening outside their classroom.

There’s a learning curve for faculty, for students, and for everyone else, too. It’s a slow learning curve, but an essential one.

Integration won’t happen for students unless faculty scaffold and support it, help them see the connection: how does this class relate to you?  To who you are?  To who you’re becoming?

It takes faculty a while to learn how to do that. It’s kind of scary, risky. It requires us to provide scaffolding and support to faculty as they think it through, learn how to do it, step by step. We need to encourage it. Reward it. Support faculty as they learn from others and try it out themselves. Get used to looking at students’ portfolios as they enter the class, seeing who they are, what they bring to the class, what they’re doing in other classes. It’s really building a culture where it’s widely understood – yeah, this is what we do here. We as a campus are working together to support the learning and growth of these complex individuals, our students, who we value so much. That takes time and practice.

That’s why building meaningful ePortfolio practice is challenging. The technology is simple.  But the pedagogy is not easy to learn.  It requires an extended process of engagement and professional learning, a sustained collective engagement.

That’s difficult to do in higher education.  Our institutions are not set up to do this.  We’re not set up as integrative institutions.  Faculty are not familiar with or prepared for an integrative approach.  And students aren’t accustomed to it, either.  It’s a paradigm shift, across the board, that requires a high degree of institutional support, institutional intentionality, institutional integration.  That’s what we’re working on now.  We’re all inventing a new practice for higher education, and it takes time.

Integrating ePortfolio pedagogy and assessment

(1:00:15)

Dominique Zino:
Is there a tension between that kind of pedagogy and the need for assessment?
Joseph Ugoretz:
This doesn’t have to be, no. There isn’t if the assessment is authentic and really feeds back into the teaching and learning enterprise.
Laura Gambino:

I don’t see them in tension at all. In fact, I think it’s just the opposite. They complement each other nicely. When we have our Assessment Days, where all of our faculty are getting together, it’s not just looking at artifacts of the work. The work is great. But faculty find the greatest value in the reflections, and looking for integration. Because that’s one of our core outcomes. We want students to integrate and apply and critically think. Because we have the portfolio, because we have not only the artifacts, but also the reflections and all the different pieces of their learning, we’re getting a much richer picture of the student learning experience. And we’re able to see what students are learning and how they’re learning it, which enables us to do a better assessment of our learning outcomes.

Bret Eynon:

When we think about assessment, it’s important to distinguish between assessment for accountability and assessment that’s designed to enhance student learning, faculty learning, and institutional learning. When we talk about assessment at LaGuardia we’re talking about that latter category. Assessment for accountability can be done more simply in ways that are ultimately not very satisfying or meaningful. But we’re talking about assessment that has a deeper purpose–to help us work more collectively and effectively with students to transform the learning experience.

Grounding assessment in the real work of the classroom makes it more authentic, makes it easier to connect the insights generated by the assessment back to the practice that generated them. “Okay, here’s the work, here’s the assignment, here’s the result–let’s go back and redesign the assignment. Let’s go back and look at the work again so that we’ll see a real adaptive cycle.”

The portfolio can facilitate that by making the artifacts available. And it can facilitate the next step of that, which I think Laura is pointing to, which is thinking about those discreet components of the work as part of a larger picture with the student. That’s powerful for educators–seeing the whole student, seeing the evolution of the whole student.

Laura Gambino:
The ePortfolio brings the student into the process. I heard a colleague at Tunxis say this a long time ago, and I’ve used it many times since: the Latin root of the word “assess” means to sit beside. So assessment really means to sit beside the student. We can’t physically have the students sit beside us. But through their ePortfolios, they are sitting beside us, because their voice is now in the process. And they are more active in the process through the story they’re telling us in their ePortfolios. So to me, it’s the truest form of assessment, where students are sitting beside us.
Joseph Ugoretz:
Reflection includes a kind of self-assessment. Putting that power in the hands of the students is really valuable.
Bret Eynon:

Pedagogy should inform assessment and vice versa. They have to dovetail. Integrative reflection is key to both. Reflection is the site of connection and crossing boundaries. It opens all sorts of possibilities for learning and change. We’re asking students to reflect and learn about themselves as learners, to think about how they learn best, how they can get stronger as learners. We’re asking faculty to reflect and understand what’s happening in the learning process, what students bring into the classroom, and how they’re changing. And we’re asking the institution to reflect and learn on a broad scale: who are our students? Are we serving them well? Are we meeting their needs? Are we really doing what we say we’re doing? And when we ask those questions, we must constantly bring it back to action: “What does this tell us about what we should do?”So assessment is not an end unto itself. It’s really about thoughtful change. How should we change what we’re doing to make it better? That’s essential if we’re going to be successful as 21st century educational institutions. We have to do that. We’re not given the choice to just stay as we’ve been. We must adapt to very new situations. The new learning ecosystem is filled with powerful new players. If we’re not figuring out how we can best help our students–what we have to offer them and how we can do it better–we’ll be left behind. To my mind, it’s imperative that we in higher education figure out how to do that. There are lots of facets to that work, but it’s now increasingly clear that ePortfolio can play a crucial role in helping higher education rise to this moment.

Notes

[1] A pioneering, teaching-with-technology faculty development program coordinated by CUNY’s American Social History Project from 1996-2002, the NEH-funded New Media Classroom served educators from schools, colleges/universities, and cultural institutions nationwide. Under Eynon’s leadership, NMC helped educators develop strategies for using new digital resources in history and culture classrooms.

[2] Co-led by Eynon and Georgetown University’s Randy Bass, coordinated by Georgetown’s Center for New Designs in Learning and Scholarship, and funded by Atlantic Philanthropies from 2000-2005, the Visible Knowledge Project engaged 70 faculty from 22 campuses nationwide in Scholarship of Teaching and Learning projects focused on the use of new digital resources to support adaptive, embodied, and situated learning. For more info, see: https://blogs.commons.georgetown.edu/vkp/

About the Participants

Bret Eynon is a historian and Associate Provost at LaGuardia Community College (CUNY), where he is responsible for strategic planning and oversight of collegewide educational change initiatives related to learning, teaching, curriculum, technology, advisement, and assessment. The founder of LaGuardia’s Center for Teaching and Learning and its internationally known ePortfolio project, Eynon also co-directed (with Georgetown’s Randy Bass) the national Visible Knowledge Project and directed the FIPSE-funded Connect to Learning project, which from 2010 to 2014 worked with twenty-four diverse campuses nationwide and produced a unique international resource site, Catalyst for Learning: ePortfolio Resources & Research (http://c2l.mcnrc.org).

Eynon’s many articles and books include The Difference that Inquiry Makes (with Randy Bass); Freedom’s Unfinished Revolution: An Inquiry into the Civil War and Reconstruction; and 1968: An International Student Generation in Revolt; as well as Who Built America?, an award-winning series of textbooks, films, and CD-ROMs created with CUNY’s American Social History Project. His most recent book, co-authored with Randy Bass, Open and Integrative: Designing Liberal Education for the New Digital Ecosystem, was published in June 2016. High Impact ePortfolio Practice: Catalyst for Student, Faculty, and Institutional Learning, co-authored with Laura M. Gambino, will be released in January 2017. A national faculty member for the Association of American Colleges and Universities since 2006, Eynon has been honored for his work by the American Association for Higher Education, the American Council on Education, the Community College Futures Association, and the Carnegie Foundation for the Advancement of Teaching. The national Community College Humanities Association has recognized him as a Distinguished Humanities Educator.

Laura M. Gambino is Associate Dean for Assessment and Technology and Professor of Information Technology at Guttman Community College (CUNY).  In her role as Associate Dean, Dr. Gambino oversees the College’s institution-wide ePortfolio program and the Integrated Planning and Advising for Student Success (iPASS) initiative. She serves as Principal Investigator for Guttman’s EDUCAUSE/Achieving the Dream iPASS, GradNYC College Completion Innovation Fund and Title V grants.  She also leads the assessment of Guttman’s institutional student learning outcomes; her work in this area focuses on the intersection of assessment, pedagogy, and assignment design.  Gambino, a leading ePortfolio and assessment practitioner and researcher, serves as a DQP/Tuning Coach for the National Institute for Learning Outcomes Assessment (NILOA).  She is co-author with Bret Eynon of High Impact ePortfolio Practice: A Catalyst for Student, Faculty, and Institutional Learning.

Joseph Ugoretz is currently Senior Associate Dean for Teaching and Learning (and Interim Chief Academic Officer) at Macaulay Honors College of the City University of New York (CUNY). He is also an adjunct faculty member of the CUNY Graduate Center’s Certificate Program in Interactive Technology and Pedagogy. Dr. Ugoretz has taught high school English, served as a professor of English at a large urban community college and has led initiatives across the liberal arts, particularly in the STEAM (Science, Technology, Engineering, Arts and Mathematics) disciplines. He has taught fully online courses, first-year honors seminars, graduate courses, and high school English, as well as faculty development programs from workshops to retreats to unconferences. He currently serves on the board of the Association for Authentic, Experiential and Evidence-Based Learning (AAEEBL) and is a leader in open flexible eportfolios and the scholarship of teaching and learning.

At Macaulay, Dr. Ugoretz supervises the Instructional Technology Fellows, the faculty and curriculum of all the Macaulay seminars, and frequently teaches the Arts in New York Seminar as well as upper level seminars in Science Fiction and the Future of Education. Events and activities like the Night at the Museum, Snapshot NYC, BioBlitz and the Science Forward video series are examples of what he and his team provide to Macaulay students.

Aside from the scholarship of teaching and learning, Dr. Ugoretz’ research interests include Urban Legends and Internet Lore, Science Fiction, and Oral Performance Art (the subject of his fieldwork with pitchmen at county fairs and carnivals, and of his essay, “Quacks, Yokels, and Light-Fingered Folk: Oral Performance Art at the Fair” in the 2006 collection Americana: Readings in American Popular Culture).
He blogs at https://prestidigitation.commons.gc.cuny.edu.

Building a Place for Community: City Tech’s OpenLab

Charlie Edwards, New York City College of Technology

Jody Rosen, New York City College of Technology

Maura A. Smale, New York City College of Technology

Jenna Spevack, New York City College of Technology[1]

Abstract

For the Fall 2011 semester, New York City College of Technology (City Tech) launched the OpenLab, an open digital platform for teaching, learning, and collaboration that everyone at the college—students, faculty, and staff—can join. Built by a City Tech-based team using the open source software WordPress and BuddyPress, it provides a space where members can connect with one another in an academic social network, create profiles and portfolios, and collaborate in courses, projects, and clubs, sharing their work with others at City Tech and beyond. As its name suggests, the OpenLab is both open on the web and a place for experimentation. It is also experimental itself, in its goals to increase student engagement and reduce fragmentation at a large, diverse, and sometimes impersonal commuter institution. What we have seen so far is promising: in only two years the OpenLab has already become an essential part of the life of the college with, at this writing, over 9,400 members. This article traces the path that led to the creation of the OpenLab, shares successes, challenges, and lessons learned along the way, and outlines future plans. We invite readers, and visitors to the OpenLab, to consider what creating such a space—open, shared, experimental, and democratic—might achieve at their own institutions.  

 

The OpenLab in Context

Institutional challenges

City Tech is a senior college of the City University of New York (CUNY), the largest urban public university in the U.S. The college has gone through several transformations since its founding in 1946 as a trade school, most importantly when it joined the CUNY system in 1964 and when it became a senior college—awarding both Associate and Baccalaureate degrees—in 1983 (New York City College of Technology n.d.b). City Tech offers a wide range of professional and technical programs including engineering, computing, architectural and construction technologies, hospitality and tourism, communication design, and the allied health sciences, as well as a variety of general education and upper-level courses in the liberal arts and sciences.

Students at City Tech differ from the popular media image of the traditional American undergraduate in several important ways. There are no residence halls available at the college, and, like most CUNY undergraduates, the majority of City Tech students live in New York City with their parents, guardians, or other family members. The college is a majority-minority serving institution, with self-reported race/ethnicity at 34% Hispanic, 32% Black, 20% Asian/Pacific Islander, and 11% White students. Just over half of City Tech students report a household income of less than $30,000 per year, and just under half are in the first generation of their families to attend college. Like undergraduates at other commuter colleges, City Tech students often have responsibilities in addition to their studies; they may work part- or full-time, or provide care for children or other family members (CUNY Office of Institutional Research 2012, New York City College of Technology 2012).

Enrollment at the college has soared in recent years; by Fall 2012 the college enrolled over 16,000 undergraduate students, almost two-thirds of whom attend full-time (CUNY Office of Institutional Research 2012). While its location, in busy Downtown Brooklyn adjacent to the Brooklyn and Manhattan bridges, facilitates easy access via public transportation, the density of the neighborhood constrains opportunities to enlarge the college campus. A much-needed new building is currently under construction, though its completion is not anticipated until Spring 2017. Meanwhile there is a lack of attractive, communal gathering space at City Tech: there are few lounges for students to use to socialize, relax, or study, no quadrangle or lawns, and only one small outdoor space.

City Tech’s student body, then, is constantly in motion between home, work, and school, with limited opportunities to make connections with and at the college—though such connections are key to student success.

 

OpenLab_Homepage-1024x725Figure 1. OpenLab Home Page (click for full-size image)

 

A living lab

The OpenLab is a major component of “A Living Laboratory: Revitalizing General Education for a Twenty-First Century College of Technology,” a five-year project (2010-2015) funded by a grant from the U.S. Department of Education’s Title V Strengthening Hispanic-Serving Institutions program.[2] The Living Lab project aims to improve student engagement, specifically by addressing the difficulties that students encounter in making connections between their required General Education courses and the highly specialized coursework in their majors, even as these courses function as gateways to their fields. The project seeks to collaborate with groups throughout the college to re-envision General Education across the disciplines by using City Tech’s Brooklyn waterfront location as a living laboratory, employing place-based learning and the hands-on modes familiar to our students in their majors to increase student achievement, measured by grades and persistence, retention, and graduation rates.

The OpenLab was conceived as a key element of the Living Lab project that would both support the Living Lab’s goals and seek to address these larger institutional challenges of disconnection and fragmentation by strengthening the college’s social and intellectual fabric. As an open digital platform the OpenLab can make the college curriculum visible to City Tech students, and encourage them to make connections among their strengths, interests, and goals. The OpenLab creates avenues for community members in different classes, departments, and schools to connect with one another. It offers a virtual place for students, faculty, and staff to meet and work and an opportunity to strengthen the college community unfettered by the limitations of our physical plant. The OpenLab provides virtually through its online platform what the City Tech campus cannot always provide physically: a beautiful, inspiring space for communities to gather and grow.

 

Why Open? Why Lab?

The name OpenLab speaks to two important and pervasive themes of this initiative. Its open nature fosters community and connection. Unlike closed online systems such as Blackboard, the learning management system available throughout CUNY, the OpenLab allows members across the college to communicate with one another and the world outside City Tech. OpenLab membership is available to everyone at City Tech through a simple registration process. Members can see what other members are doing, as can visitors from outside of the City Tech community. Such openness has great potential. This open-by-default design provides greater transparency for students looking to see what goes on in other courses, offering them a unique insight into their choices for registration and choosing a focus or major. Faculty members can benefit from the innovative pedagogies their colleagues employ, and can see not merely which courses their students have enrolled in but also what they study in those classes. Staff members can get a glimpse into course activities that would not otherwise be accessible to them. For all members, the possibility of observing and participating in curricular, co-curricular, and extracurricular activity is an exciting prospect, one that is only possible because the system is open.

As a college of technology, City Tech uses throughout its curriculum a laboratory model that is both familiar to and successful for its students. The OpenLab fosters the kind of learning espoused in the City Tech mission statement, developing both intellectual curiosity and practical experience in career fields by creating active, hands-on, and process-oriented experiences. In so doing it pairs with the college’s educational goals for the students to apply problem-solving skills to their work in their professions and develop communication skills (New York City College of Technology n.d.a), fostering intellectual curiosity and practical experience in career fields. All members of the OpenLab—students, faculty, and staff alike—benefit from the opportunities for experimentation available to them through use of the tools it provides: blogging capabilities, collaborative documents, shared discussions, and the ability to integrate external tools easily and seamlessly. By using WordPress for content creation, OpenLab members gain valuable experience with software they will likely encounter again, whether for work or in their own extracurricular pursuits. We encourage students specifically—and all OpenLab members in general—to take advantage of the expertise they can develop in using WordPress, and to promote it as a skill when they move on from City Tech.

The OpenLab, in adopting the open source software WordPress and BuddyPress as its foundation, joins a worldwide laboratory of open source design and development. In creating the OpenLab, the team has been fortunate to build on the experiments and experiences of others at City Tech, CUNY, and beyond who have implemented WordPress and BuddyPress as open pedagogical tools over the past decade. The open digital platform that the OpenLab is most similar to is the CUNY Academic Commons, an academic social network launched in 2009 that connects faculty, staff, and graduate students across all 24 campuses of CUNY (the team has recently released the Commons In A Box, which puts a full-featured WordPress and BuddyPress implementation within reach for any institution). The OpenLab team also counts as inspiration UMW Blogs, built at the University of Mary Washington by Jim Groom and his team, and the University of British Columbia’s WordPress installation spearheaded by Brian Lamb. We have learned from WordPress projects at other CUNY colleges as well, including Baruch College’s Blogs@Baruch, Macaulay Honors College’s and York College’s student ePortfolio platforms, and Queens College’s undergraduate writing initiative. In building the OpenLab as a place for community at City Tech, then, we are proud to join a community of educators and technologists dedicated to opening education through the use of free and open source software and a commitment to open pedagogical practices.

 

The OpenLab in Theory

In addition to the strong lineage of inspiring open source projects that paved the way for the OpenLab, the project has theoretical grounding centered on the idea of student engagement as critical to ensuring that students are successful in their college careers (Kuh et al. 2007, xi). Certain pedagogical and social practices contribute to and bolster student engagement; these include frequent interaction between students and faculty, especially outside of class time, both academic and social interaction with peers, feedback received from faculty in a timely manner, and “time on task”—the amount of time students spend on their academic work (Kuh et al. 2007, 43). Further, a strong engagement with the college community may be especially important for commuter students such as those at City Tech, who spend less time on campus than do undergraduates at residential colleges (Krause 2007, 29).

Research also indicates that technological interventions like the OpenLab have the potential to strengthen student engagement at commuter institutions (Krause 2007, Kinzie et al. 2008). The OpenLab offers a platform for faculty-student and student-student interaction in courses and extracurricular activities, as well as multiple avenues for faculty and peer feedback on student work in a variety of media. Since the OpenLab is a web-based platform, the ability to accomplish coursework while off-campus may also increase the amount of time that students spend on their academic work. With the multiple responsibilities that our students navigate throughout their days in addition to their coursework, the opportunities for 24-hour, off-campus access to the OpenLab may be especially beneficial.

The OpenLab can also have an impact on student engagement beyond flexible access to coursework. Recent data collected for the National Survey of Student Engagement (NSSE), which samples over one million students at hundreds of four-year colleges and universities about their participation in programs and activities for learning and professional development, reveals that engagement rose among “[f]irst-year students who frequently used social media to interact with peers, learn about campus events and opportunities, and interact with faculty and advisors” (National Survey of Student Engagement 2012, 18). While the NSSE asked students about proprietary social media platforms, this finding corresponds with the observations of those working with open academic social networks. Matthew K. Gold, for example, notes that these platforms provide multiple pathways for connection via what he terms “low-stakes engagement”: “Students may visit the course site because they receive a friend request from a fellow student, and in visiting the site, they may quickly respond to a status update, contribute a link to a group, respond to a forum post, send a private message, or update their own status” (Gold 2011, 70-1). These academic social networks provide multiple avenues to attract students into a course, and in doing so can strengthen their ties to the college community.

The OpenLab can foster opportunities for our students to work and study together, to encourage each other, and to support each other’s success, which is far less likely when students do not know or feel connected to one another. Beyond this, though, as Gold and Otte report in their discussion of the CUNY Academic Commons, faculty and staff members too can similarly benefit from working with a platform that encourages collaboration, a networked structure that encourages interdisciplinary, intercollegial, and nonhierarchical conversations (2011, 10). Such a structure, however, does not come about automatically when WordPress is installed; rather, it is the result of conscious design choices made with community-building in mind.

 

Design of Place: Structure and Features

Customizing for community

The OpenLab and much of its user-created content is open and visible to the public, allowing prospective members and site visitors to browse and comment. Although only members of the City Tech community can join the site and actively create content, the site structure and home page content feed invite visitors to experience the dynamic learning environment and community that is not always visible on the physical campus.

BuddyPress, the open source social networking software integrated into the OpenLab’s WordPress Multisite installation, allows us to define a virtual place that supports City Tech’s community-building needs and provides the infrastructure necessary for an active, high-impact, connected learning environment. It offers social networking features like activity streams, member profiles, group and site creation, discussion, and collaborative document editing, and is at the core of sibling sites such as the CUNY Academic Commons. BuddyPress powers one of the most distinctive features of the Commons, its home page, which highlights recent activity by members, groups, and blogs, and gives visitors an instant snapshot of the vibrant Commons community. This integration of members and their activity, and the collaboration and cross-pollination it fosters, inspired the OpenLab’s home page design and organization.

 

OpenLab_Navigation-1024x725Figure 2. OpenLab “Group” configuration: Courses, Projects, Clubs, and Portfolios

 

There are, however, significant differences between the OpenLab and its sibling installations. The biggest modification, implemented in partnership with our consulting developers, was to customize BuddyPress and subdivide its default “Group” configuration into several discrete types, Courses, Projects, Clubs, and Portfolios:

  • Projects on the OpenLab can range from student research projects and collaborations to college committees to grant-funded academic journals, and can be created by individual or multiple members.
  • Clubs allow any member of the College community to grow an online space related to a particular social or academic interest.
  • Courses are set up to offer faculty members the tools to facilitate class collaborations, engage students in meaningful peer-to-peer learning, and provide a digital repository for course documents.
  • Portfolios are an extension of the College’s student ePortfolio initiative and allow for personal portfolio development for all OpenLab members, including faculty and staff. The member Portfolio is a WordPress or external site linked to the member Profile page. Members are encouraged to use the site to present their teaching, academic, or creative work.

These appear prominently in the navigation, along with People, the OpenLab’s directory of members. This organizational architecture supports usability and navigational aesthetics and facilitates group creation, interaction, and findability by separating groups into defined areas of focus that are familiar to our college community. This important customization helps to communicate a clear path for visitors and offers a foothold for members less experienced in open, collaborative systems.

 

OpenLab_People-1024x725Figure 3. OpenLab People Page

 

The People section of the OpenLab brings together all our members—students, faculty, and staff—and provides access to their Profiles. Any member of the campus community with a City Tech email address may sign up and create a Profile on the OpenLab.[3] The member Profile presents the member’s academic interests, department or area of study, and contact information, as well as recent activity on the OpenLab including posts, comments, and discussion topics. Profile visitors can view the Projects, Clubs, and Courses that the member participates in, allowing them to make connections with others who have similar interests and providing opportunities to share learning, social, and research experiences with the greater OpenLab community. The member Profile also allows for a pseudonymous display name and allows the member to choose the information they share with others. Use of pseudonyms allows members to protect their identities without compromising the public nature of the OpenLab. We believe that giving members control over their self-presentation is an essential component of any open platform.

 

OpenLab_Profile-1024x725Figure 4. OpenLab Member Profile

 

The benefit of the OpenLab’s customized navigational sections is that it gives new or prospective members a place to start: browsing these general areas guides the member who might be interested in joining a campus club or seeing an example of a member Portfolio. Although not all groups on the OpenLab are public, those that are present the most recent data on the group Profile page. Although the OpenLab encourages openness, it also supports a range of fine-grained privacy options that enable groups to have public or private spaces as desired, and even a combination of the two. Each group Profile, similar to the member Profile, displays recent posts, discussions, collaborative document activity, group membership, and a link to the group’s associated WordPress site.

 

Listening to Campus Voices

The OpenLab team follows an iterative approach to design and development, the standard open source software methodology. After releasing a version of the site, we listen for feedback from members and observe how they embrace or ignore new features. Critical here is the work of the OpenLab’s Community Team.[4] All are advanced graduate students with expertise in both the technical and pedagogic aspects of using open source platforms in an educational context. Some of their work takes place behind the scenes: testing site updates, reporting bugs, evaluating and recommending plugins and themes, and developing the OpenLab’s Help content. They are also the friendly faces of the OpenLab, answering members’ questions via email and online, running workshops, visiting classrooms and offices, and maintaining open channels of communication between the OpenLab’s members and the rest of the team.

This community-minded method of development accepts the unpredictability of member needs and behaviors and allows the OpenLab to evolve as the community grows and develops. Following are several examples in which member requests and actions have translated directly into exciting new functionality and have greatly improved the design of the OpenLab as place to learn, work, and share.

 

My OpenLab

In the early releases of the OpenLab, its navigation and usability needed significant enhancement. Members repeatedly requested easier access to their personal settings and dashboards. A simple name, My OpenLab, was adopted for the member’s logged-in state, which was identified in headings and dropdown menus. My OpenLab makes the member’s content more readily available and improves overall usability, but also conveys a sense of place and ownership.

 

OpenLab_MOL_External_Portfolio-1024x725Figure 5. My OpenLab navigation, Portfolio integration, external site linking

 

External site linking

The ability to link a WordPress site to a group is available in BuddyPress, but this functionality did not extend to sites outside the OpenLab’s network. Many faculty members, some students, and a few campus clubs had however already developed web sites and were hesitant to start a site on the OpenLab if it meant recreating their sites there. Customizing the OpenLab’s BuddyPress install with the ability to link an external site to a group’s Profile now allows members with sites on other platforms to become part of the OpenLab community space, making this externally hosted content visible in activity feeds and on the home page. This seemingly small customization is actually a powerful intervention that simultaneously promotes member agency and inclusion. It allows the OpenLab to support those who are committed to maintaining a “domain of their own” (more formally, a “personal cyberinfrastructure,” Campbell 2009). We see this as particularly important for encouraging OpenLab participation among adjunct faculty, who may have their own independent teaching sites to accommodate their teaching on a number of campuses.

 

Cloning courses

As the number of courses on the OpenLab continued to grow, to over 600 current and past courses by the spring of 2014, it was clear that faculty needed tools to replicate their previous course structures, rather than starting from scratch. One key feature of the OpenLab is that course content remains available after the course ends; consequently, it was important to develop an alternative so faculty members would not empty courses for reuse by deleting past student data. This BuddyPress customization now offers faculty the opportunity to clone a course and establishes a best practice of maintaining an archive of past OpenLab courses available for student and department reference.

 

Portfolio integration

When the OpenLab was initially developed, the college was also looking for a new platform to host its student ePortfolios. The OpenLab was able to provide a home for this project, incorporating student ePortfolios into the site structure. A year later an initiative was launched to digitize faculty teaching portfolios and, once again, the OpenLab was the obvious place to turn. In response, we now offer all members of the college community the opportunity to create an online Portfolio, linked to the individual member’s Profile. Student ePortfolios, faculty teaching portfolios, and staff professional portfolios can now all be found together in the Portfolios section of the OpenLab. Through such design choices, the OpenLab becomes not only a solution to a logistical problem but a place for sharing, since students, faculty, and staff can see and learn from one another’s work.

These examples demonstrate our efforts to work toward a method of inclusionary innovation that encourages meaningful growth. We rely on member input to make the OpenLab the best it can be, with the aim of creating an inspiring, supportive gathering place that everyone at City Tech can share. The OpenLab’s designers strive to support this aim through compelling visual design that reflects well-considered information architecture and usability heuristics. Visitors and members alike regularly praise the OpenLab for its beautiful design. This is a tribute to the efforts not only of the current team but of the (now former) City Tech students who worked on the OpenLab’s original look and feel. Their design vision supports the hope that creating a beautiful place for learning, sharing, and belonging will foster community engagement and increase retention both within the OpenLab and at City Tech.

 

The OpenLab in Practice

Since the first semester of its use in beta, the OpenLab’s ambitious goals continue to be fulfilled in practice. The member-generated content of the OpenLab is a testament to the value of openness and of the laboratory model, and makes evident the OpenLab’s contribution to the college community. Examples of the benefits of the open, shared, democratic, and experimental avenues abound on the OpenLab.

 

Open

Charting new territory with a site on the OpenLab is a great opportunity for anyone interested in making content more visible. The OpenLab can offer a new and very real audience for that work because of its open nature. The openness of the platform encourages browsing through the shared materials, both in formal and informal ways—exemplified by a student who asked our Community Team why some courses were available for her to see whereas others required membership. She was following some of the courses in her major that she had not yet taken, or had taken with different professors, found reading these course sites a valuable resource, and began conversing with the faculty member to expand her education further.

Some courses assign what this student did voluntarily: to draw on the OpenLab’s openness as a source for learning. For example, a professor asked his Introduction to Poetry students to explore the OpenLab and to comment on materials on other courses’ sites. This not only challenged students to move beyond the comfort of their class’s community, but also reminded others that the OpenLab is open for this kind of cross-pollination. An English Composition professor asked students to examine other members’ avatars and report back about how they perceived members based on their avatars. This served the course as an introduction both to visual literacy and to the construction of an online persona. Another faculty member in Restorative Dentistry used the fact that the OpenLab is readily available to non-members by inviting colleagues from an international professional organization in the field of restorative dentistry to view his students’ work and comment on it. Here, students were exposed to an audience beyond the college, professionals with expertise to benefit their growth in the field.

Courses are not the only venues to benefit from being open. The needs of various college committees and organizations have been met by sharing information on the OpenLab. These additions have not been top-down, but rather reflect the community’s adoption of this much-needed place to collaborate. For instance, the college-wide Assessment Committee, in an effort to increase transparency and inform and involve more of the community, established an OpenLab Project that includes reports from the most recent assessment cycle for all participating departments. College Council has had a public site on a college server but is now developing its OpenLab presence. Committees that are open to all college faculty members might wonder what added value there could be in sharing their materials in the open forum of the OpenLab, rather than just with its voluntary members. In both of these examples, the openness of the site matters because the sites are part of the network, not hidden away from their potential audiences. Unlike stand-alone sites, the networked nature of OpenLab sites creates a broader audience for the work of these committees and organizations. Not only can faculty and staff members benefit from seeing what other committees are doing, but students as well can begin to see a clearer picture of the college, its infrastructure, and its concerns.

 

Shared

Community building is a central component of the OpenLab, creating venues for shared spaces in courses, among courses, and across the college. Learning Communities are a natural fit for the shared spaces of the OpenLab. These communities enroll a cohort of students in two or three courses that collaborate on a shared project or consider overlapping themes. In one of the first learning communities on the OpenLab, two professors in Hospitality Management and one in English connected their courses using the OpenLab via custom menus that seamlessly linked students from one course to the next. This connection fostered the sense that all three faculty members collaborated to create this more dynamic, more engaging experience for these first-year students. In another Learning Community, professors from Humanities and English similarly linked courses using custom menus in that first semester, and then subsequently began sharing their space in one Course site to facilitate their community’s joint project, a place-based endeavor that drew on semester-long projects that culminated in a virtual walking tour. This tour used various external technologies as it collated students’ on-site speech videos by embedding them in a shared Google map that was itself embedded on the shared OpenLab site.

The open nature of the platform can easily facilitate the development of unofficial learning communities. One instance came when two math professors joined efforts when each taught a section of the same course in the same time slot. They shared one OpenLab site for students to exchange reflections on their joint field trip, a walk across the Brooklyn Bridge to provide the basis for mathematical calculations, and even booked a larger classroom for one class session to accommodate both classes in a discussion of the results from their joint field trip. Other faculty members have experimented with unofficial learning communities, such as one bringing together a hybrid section with a fully online section of the same course. The opportunity to communicate and collaborate with other students is so important for the otherwise disconnected online students; pairing them with students who do have the chance to get to know each other face-to-face enhances the learning experience overall among the members of the created community.

 

Experimental

With the capability to expand educational opportunities through incorporating multimedia components into posts and pages, the OpenLab fosters the hands-on, active learning so many in our community strive to integrate into their courses. The availability of spaces to contribute content in a variety of media on the OpenLab can inspire disciplines not traditionally thought of as visual to make use of the site’s capability to incorporate images and videos, or those not typically considered writing-intensive to include expository blog-posts or collaborative-document assignments. In one math course on the OpenLab, students were asked to reflect on the term infinity both in its colloquial usage and as a mathematical term. To further support their understanding of these definitions, students were asked to relay a story about when they first learned about the term, and to incorporate a photograph that encapsulated the meaning. Writing, visual literacy, and a sense of play became vital skills to complete this math assignment, a great example of how the OpenLab creates a place for creative expression and innovative pedagogy.

In an Introduction to Hospitality Management course, students were asked to consider Brooklyn as the site of tourism. Imagining they were each the concierge in a boutique hotel in Brooklyn, students were asked to develop videos that guests would watch to pique their interest in a nearby location. Students could creatively cast their videos, considering not only the details of a location, but also the intended audience of the video and how they would want to represent the location to the clientele. This project took advantage of YouTube as a resource for storing videos that were then easily embedded into posts. It also identified for students their role and their audience, and offered a creative medium for students to consider the role of the concierge, and to experiment with place-based learning in a field-specific way.

In each of these examples, students had opportunities to use openly available digital tools and to transfer knowledge and skills from their college experience to work in their disciplines. Faculty members could continue to cover course content while expanding their pedagogical practices and affording students additional avenues for experimentation and creative expression. One student took on this challenge independently, conceiving of her ePortfolio not merely as a way to record her achievements in Restorative Dentistry but also as a customizable, thoughtful reflection of her approach to her field. Using the extended metaphor of a city to describe the mouth full of teeth, she showcased her dedication to her work and her interest in experimenting both in the lab and in her representation of that work.

 

OpenLab_ePortfolio-1024x725Figure 6. Student ePortfolio: Restorative Dentistry

 

Democratic

All members have the ability to create Projects, Clubs, and Portfolios—only faculty can create Courses, which is the only difference in permissions among the types of members—and to read or actively participate on public sites throughout the OpenLab. This means that in addition to officially sanctioned committees and clubs, there are numerous unofficial, unchartered, or even previously unformed groups coalescing in OpenLab Projects and Clubs. In a recent survey of OpenLab Clubs, the Community Team found that more were unchartered than officially City Tech-sanctioned, indicating that the OpenLab has given a place for students—and all members—to gather around common interests regardless of official support. Thus, in addition to the notable presence of the Student Government Association and the City Tech newspaper, there are also unchartered clubs such as Anime Gaming Underground, or the Mobile Application Developers. We are eager to provide support—both technical and community-building—for these ventures to become fully realized.

Projects such as the First Year Writing and Developmental Writing Resource Archive, which offer resources for faculty teaching the college’s sequence of composition courses and developmental writing, respectively, encourage full-time and adjunct faculty to join, share, and benefit from the wealth of resources that have been compiled. Providing opportunities for full-time and adjunct faculty members to share space and resources has been one of our main goals for the OpenLab, making it a space for all to get involved, regardless of rank or position. Many new projects are being developed to collate materials for faculty working on a given course or initiative, and to share discussion space to collaborate on the issues at the heart of that endeavor. These spaces open the conversation to adjunct faculty, creating avenues for their involvement that have not existed before. OpenLab workshops also create such shared spaces, since they provide support for faculty and staff, full-time and part-time alike, and can facilitate collaboration across and among departments and offices. As part of the Title V grant, adjunct faculty members receive a stipend to participate in OpenLab workshops; this further facilitates their inclusion in the OpenLab.

 

The OpenLab in Progress

These are just a few examples of the kind of work that the OpenLab enables; every day visitors to the site can find more. It has been exciting to see how the OpenLab’s members have grasped its potential for connection and collaboration, openness, sharing, and community-building. But of course technologies do not determine outcomes. Indeed, as Gold and Otte note in their discussion of the CUNY Academic Commons, any intervention in the mode and means of social interaction—and perhaps especially in education—is potentially disruptive and destabilizing (Gold and Otte 2010, 10). Inevitably, then, with all the OpenLab’s successes, there have been challenges along the way. Below are three of these and the approaches taken to address them.

 

Growing pains, growing community

The first challenge has actually been the OpenLab’s success: its dramatic rate of adoption. As the Living Lab’s external evaluator noted, the project’s original usage target (1,000 active members by the end of the grant) quickly looked “quaint.” Such vertiginous growth naturally brought technical concerns: with the site rapidly becoming essential to many of its members, we hardened its infrastructure for reliability and have recently optimized the database to ensure it can support this rate of growth for years to come. The biggest impact, though, has been experiential—a site with thousands of members simply feels different, and the individual members and the work they create can easily be lost in the sheer volume of activity and content. If we fail to attend to the affective dimension of site use, we could find ourselves simply replicating online the problems of disconnection and fragmentation that the OpenLab hopes to address.

Early in the development process, we made adjustments to the site design to showcase member content more effectively and make it more readily findable: the home page now shows the four most recently active groups of each type (Courses, Projects, Clubs, and Portfolios), and search and filtering capabilities have been added. Another important addition to the home page is the “In the Spotlight” section, which allows us to call attention to a different example of the exceptional work on the OpenLab each week. Members have reported their excitement at having their work recognized so prominently; when we featured the Restorative Dentistry student’s ePortfolio described above, its author responded that this appreciation of her work had not only provided encouragement but also “made it something real” for her, “something that a living person actually looked at and read (as opposed to an odd exercise similar to jotting things down, folding them into paper airplanes, and launching them into the cybervoid)” (Jes Bernhard, pers. comm.). We are planning other ways to celebrate member contributions and enable all members to participate in the selection process.

Evident in the above is the interplay between software and social. A social network should not be conflated with community;[5] building a sense of community among OpenLab members is an ongoing process that requires conscious and intentional individual and collective effort. The Community Team plays an important role here in helping members feel welcome on the site. With such a large member population, though, the Community Team cannot interact with everyone—members must create community by and for themselves. And they are. Many members complete public profiles and make “friend” connections. New faculty have set up a private space where they can share their experiences, ask questions, and discuss concerns. Faculty members continue to use the OpenLab to share ideas and resources, and discuss best practices with colleagues; adjunct faculty, who often find themselves marginalized at their institutions, are joining these conversations. Important student-facing administrative offices—Veterans Affairs, for instance—see the OpenLab’s potential for giving the students they serve a space of their own to communicate. Students have created initial presences on the site for key organizations and informal interest groups, too.

We believe, though, that there is more we can do to foster community on the site. We will be working to understand, through quantitative and qualitative approaches, how the OpenLab’s members (especially its less vocal members) use and experience the site, what works and does not work for them. We will be creating ways to bring members together on the OpenLab for shared activities—including, simply, fun.

 

A place for students

We are particularly interested in understanding the perspectives of one key group: students. Students make up the majority of OpenLab members (over 90% at this writing). However, despite the success of Courses on the OpenLab, the wide array of individual and group Projects created by students for their coursework, and strong adoption of student ePortfolios, we have yet to see widespread adoption of the site for extracurricular activities. There are many external factors that undoubtedly influence student use of the OpenLab. In addition to the time and space constraints we know affect City Tech students, another factor that may have an impact on their voluntary use of the OpenLab is access to technology. Although many own or have access to their own computers, tablets, or smartphones and Internet access from off-campus, others do not, and must rely on shared computers in their homes or at the college. As enrollment has risen, campus computer labs, like the campus overall, strain to accommodate the increasing number of students who wish to use them (Smale and Regalado, 2011). Thus, students may not always be able to access the OpenLab at a time and location that is convenient for them and conducive to using the OpenLab for extracurricular reasons. Additionally, while there are convincing arguments for using the OpenLab rather than closed, proprietary platforms, students may see Facebook or other social networks as more appropriate for co-curricular and extracurricular activities since they already socialize there.

We have no desire to construct a “creepy treehouse” to lure in students (see Gold 2011 for a discussion of this term [73-4]), but are concerned to help them understand that the OpenLab enables them to create places of their own, and that they can be co-creators of the OpenLab itself by participating in our collaborative development process. In response, we are making concerted efforts to engage City Tech’s students beyond the classroom. We have recently recruited a group of Student Community Team members who contribute regularly to a site they have named “The Buzz,” and are looking to expand the team to continue to provide dynamic content by and for our student population. Student team members have also begun working with our Community Team to conduct proactive outreach to student clubs and other organizations and provide them with help and guidance on incorporating the OpenLab into their activities. We hope that these efforts will encourage deeper integration of the OpenLab into student life at City Tech.

 

OpenLab_Buzz-1024x725Figure 7. Student Community Team Blog “The Buzz.”

 

Understanding the OpenLab

We are deeply committed to the OpenLab as a democratic space where students, full-time and adjunct faculty, and staff are equally welcome and valued, a space that they own and which is shaped by their needs, that encourages openness, connection, and experimentation while respecting its members’ privacy and their right to share, hide, create, or delete their work as they see fit. We have found, however, that conveying this vision requires continued work. Although many members have been with us for months or even years, each semester brings new students and faculty to City Tech and new members to the OpenLab. This means that, just as we explain the site’s features and functionality to new members, we must also seek to convey the aims and ethos of the project. Some members regard the site as another Blackboard or Facebook and complain that its features are lacking in comparison. We explain that their missions could not be more different—that Blackboard is a learning management system primarily focused on courses whereas the OpenLab is a space that anyone at City Tech can use (and that the platforms can readily co-exist). In regards to Facebook, we might observe that our efforts to build a social network at City Tech are ultimately aimed at improving student outcomes, not monetizing their data.

Members can certainly use the OpenLab without knowing or perhaps even endorsing its underlying argument. Accustomed to working with closed, rigid, and proprietary products over which they have little control, many people are not likely to expect educational and social media software to be open, flexible, and dedicated to enhancing their agency. We have learned, though, that the lack of a shared understanding of the project can limit its potential. For instance, if faculty members do not know the pedagogic benefits of the OpenLab’s open and collaborative features, they are less likely to use them; students will be reluctant to provide feedback if they have no expectation that their concerns will be heard and acted on; adjunct faculty may not join an open interest group if it is unclear that it is truly open to all.

Finding effective ways to build this shared understanding, then, is critical to our outreach activities. We have created a brainstorming game that faculty members can use to generate assignments that employ the affordances of open platforms such as the OpenLab to support specific learning objectives. This game has been used for City Tech professional development, and we have run interactive sessions at conferences to further promote open digital pedagogical practices. We are continuing to enhance the OpenLab’s Help content, and we have established an Open Pedagogy Project as a place to discuss and share best practice examples of OpenLab usage—explicitly articulating the project’s goals and guiding principles in spaces such as this allows us to hone them and debate them with the OpenLab’s members, and challenges us all to live up to them.

 

Sharing the OpenLab: at City Tech and with you

Each new semester allows us to revisit, revise, and improve. Not only our development approach but our entire project process is iterative, so the next few years promise to be as eventful as the last. We will continue to evolve the OpenLab in response to member feedback. A generative platform generates ideas; members are constantly coming up with new ways to use the site. One recent and exciting request is to provide support for alumni mentoring of current students. Additionally, the student team has come up with important suggestions that could help build community on the site, such as highlighting members with similar interests and mutual friends. Finally, like any active development project, we have a laundry list of improvements, large and small, that would benefit the OpenLab’s members.

Key for the project’s relationship with the college is the idea that members can work with the OpenLab, not just on the OpenLab. In other words, an open platform, built by and for City Tech, offers opportunities for collaboration, participation, and co-creation that are unthinkable with closed, proprietary software solutions. For example, the OpenLab creates rich possibilities for curricular integration across a wide range of disciplines. We have already developed an internship program in partnership with our consulting developers and City Tech’s Advertising Design and Graphic Arts department that has given students real-world experience in documenting and enhancing the OpenLab. We can imagine other synergies: students in City Tech’s new professional and technical writing major could create Help content that would be read by thousands of people; classes in web development could create plugins for the OpenLab and thus for the BuddyPress and WordPress communities; students in education fields could analyze and learn from best practices they find on the site. Thus, in addition to reaching out to student organizations, we are also working with academic and administrative departments, other grant-funded projects, committees, and so on, to help them explore what the OpenLab can mean for their work.

Looking beyond City Tech, we are also committed to sharing both what we have built and what we have learned along the way in creating, using, supporting, and sustaining the OpenLab. We have begun that process here, and welcome questions and comments. We have identified discrete items of OpenLab functionality that can be packaged as software plugins for release within the scope of the current grant funding. Our plans for future development include contributions that will benefit the wider WordPress and BuddyPress communities. Most excitingly, we are seeking to partner with others to make the OpenLab’s key features—such as, for instance, our customizations of BuddyPress group functionality—freely available to all.

The OpenLab has joined other dynamic initiatives in offering members opportunities for opening education. With the strength of this experience behind us, we close with a challenge. Free and open source software is proven, robust, and sustainable. A community of practice freely shares recommendations and advice, and the OpenLab team benefits from and seeks to give back to this growing and generous group. Every day the OpenLab provides a living demonstration of what students, faculty, and staff can achieve with an open platform that is built for community. Members across the college have seized the opportunities that the OpenLab provides to create open and shared spaces, to experiment and innovate, and have provided innumerable examples of the benefits of using and contributing to an open platform. We ask readers, and visitors to the OpenLab, to imagine what the use of open digital tools can do to strengthen their communities. The tools to open education are already at hand.

 

Acknowledgements

The authors would like to thank all members of the OpenLab team, present and past, for their contributions to the project. We are grateful to the OpenLab’s many supporters at City Tech, including all our colleagues in the Living Lab initiative. Finally, our deepest thanks go to the OpenLab’s members—without whom, of course, the site would be an empty shell. The OpenLab is enabled by “A Living Laboratory: Revitalizing General Education for a 21st-Century College of Technology” (2010-2015), a $3.1 million project funded by the U. S. Department of Education under its Strengthening Hispanic-Serving Institutions (HSI) Title V Program.

Bibliography

Campbell, Gardner. 2009. “A Personal Cyberinfrastructure.” EDUCAUSE Review 44 (5): 58-9. Accessed October 20, 2013, http://www.educause.edu/ero/article/personal-cyberinfrastructure. OCLC 42937303.

CUNY Office of Institutional Research. 2012. “2012 Student Experience Survey.” Accessed October 20, 2013, http://www.cuny.edu/about/administration/offices/ira/ir/surveys/student/SES2010FinalReport.pdf.

Gold, Matthew K. 2011. “Beyond Friending: BuddyPress and the Social, Networked, Open-Source Classroom.” In Learning Through Digital Media Experiments in Technology and Pedagogy, edited by R. Trebor Scholtz, 69–77. New York: Institute for Distributed Creativity. Accessed October 20, 2013, http://learningthroughdigitalmedia.net/beyond-friending-buddypress-and-the-social-networked-open-source-classroom.

Gold, Matthew, and George Otte. 2011. “The CUNY Academic Commons: Fostering Faculty Use of the Social Web.” On the Horizon 19 (1): 24–32. OCLC 39502849.

Gorges, Boone B. 2012. “Sowing the Seeds.” Teleogistic. Accessed October 20, 2013, http://teleogistic.net/2012/03/sowing-the-seeds/.

Kinzie, Jillian, Robert Gonyea, Rick Shoup, and George D. Kuh. 2008. “Promoting Persistence and Success of Underrepresented Students: Lessons for Teaching and Learning.” New Directions for Teaching and Learning (115): 21–38. OCLC 6541567.

Krause, Kerri-Lee D. 2007. “Social Involvement and Commuter Students: The First-Year Student Voice.” Journal of The First-Year Experience & Students in Transition 19 (1): 27–45. OCLC 40245359.

Kuh, George D., Jillian Kinzie, Jennifer A. Buckley, Brian K. Bridges, and John C. Hayek. 2007. Piecing Together the Student Success Puzzle: Research, Propositions, and Recommendations. ASHE Higher Education Report v. 32, no. 5. San Francisco: Jossey-Bass. OCLC 123013011.

National Survey of Student Engagement. 2012. “Promoting Student Learning and Institutional Improvement: Lessons from NSSE at 13”. Bloomington, IN: Indiana University Center for Postsecondary Research. Accessed October 20, 2013, http://nsse.iub.edu/html/annual_results.cfm.

New York City College of Technology. n.d.a. “About City Tech: Consumer Information.” Accessed October 20, 2013, http://www.citytech.cuny.edu/aboutus/consumer_info.shtml.

———. n.d.b. “About City Tech: Heritage & History.” Accessed October 20, 2013, http://www.citytech.cuny.edu/aboutus/heritage_history.shtml.

———. 2012. “Facts 2012-2013.” Accessed October 20, 2013, http://www.citytech.cuny.edu/files/aboutus/facts.pdf.

Smale, Maura A., and Mariana Regalado. 2011. “‘Sometimes I Type Papers on My Cell Phone:’ Mobile Digital Technologies and CUNY Students.” Paper presented at MobilityShifts: An International Future of Learning Summit, The New School, New York, New York, October 10-16. Accessed October 20, 2013, https://ushep.commons.gc.cuny.edu/results/.

 

[1]The OpenLab is the work of many hands, as will be evident here and the Credits page shows. The “we” used in this article thus seeks to represent not only the authors but the entire OpenLab team.

[2]The Living Lab includes three activities in addition to the creation of the OpenLab: conducting a faculty development seminar focused on using place-based learning and high-impact educational practices to infuse general education into courses; integrating the assessment of general education student learning outcomes across the curriculum; and building an endowment for the Brooklyn Waterfront Research Center. More information is available on the Living Lab project site.

[3]A valid City Tech email account is required to register with the site. Members retain access to their accounts during periods of inactivity and even after leaving City Tech; we believe members should have agency in controlling the content they have created on the site.

[4]Note that the Community Team concept is borrowed from the CUNY Academic Commons. We also benefit from the OpenLab’s other sibling installations in that our team members gained their experience by working with several of these initiatives. Additional support for students and ePortfolios is provided by the college’s Student Help Desk and ePortfolio teams.

[5] Boone Gorges, one of the lead developers of BuddyPress and a member of our team, has also made this observation (2012).

 

About the Authors

Charlie Edwards is Program Manager of the “Living Laboratory” initiative at NYC College of Technology, CUNY. After twenty years in commercial IT, she is now also a graduate student in the English PhD and Interactive Technology & Pedagogy Certificate programs at The Graduate Center, CUNY, with research interests in the Digital Humanities and Victorian popular fiction. She can be reached at [email protected].

Jody Rosen is an Assistant Professor of English and OpenLab Co-Director at NYC College of Technology, CUNY. Her research focuses on communication-intensive instructional practices in the classroom and in professional development, as well as representations of gender and sexuality in early twentieth century Anglo-American literature. She can be reached at [email protected].

Maura Smale is Associate Professor/Coordinator of Library Instruction and OpenLab Institutionalization Lead at NYC College of Technology, CUNY. Her research interests include undergraduate academic culture, game-based learning, open access publishing, and critical information literacy. She can be reached at [email protected].

Jenna Spevack is Associate Professor of Creative Media and OpenLab Co-Director at NYC College of Technology, CUNY. As an artist, designer, and educator, her projects and practices explore how interactions with and connections to ecological systems support resilience in the shifting natural-social-political landscapes. She can be reached at [email protected].

Images are for demo purposes only and are properties of their respective owners. ROMA by ThunderThemes.net

Skip to toolbar