Running Header: METHODS PAPER

Methods Paper

Ryan Bradshaw

George Mason University

EDRS 897

Dr. Call Cummings

METHODS PAPER

Methods Paper

The project I am looking to complete is a mixed methods study, with a qualitative

phenomenological study portion that examines the motivations and barriers that affect a club

sports alumnus(a)’s decision to donate to their alma mater. The hypothesis for the study is that

alumni who are former club sports student athletes will have had a more positive student

experience than alumni who did not participate in club sports as a student, which in turn will

result in a higher propensity to give.

This hypothesis is based on the quantitative research of Taylor and Martin (1995), Monks

(2003), Hoyt (2004), Drew-Branch (2011), and Rau and Erwin (2015) who all found correlations

between student engagement, particularly in student organizations, and donations as alumni.

None of the above studies have looked specifically at the subsection of club sport athletes.

McDonough’s (2017) qualitative study identified involvement in extracurricular activities, in

general, as a motivating factor for alumni to donate their time back to their alma mater, but did

not examine financial contributions. Shapiro and Giannoulakis (2009) interviewed former varsity

student athletes and identified that their experiences as student athletes were actually a barrier in

their decisions to donate.

A literature gap exists surrounding former club sport student athletes and their decisions

to donate. This proposed study will be conducted using a mixed methods format. The first part of

the study will confirm that a higher percentage of former club sport student athletes, as alumni,

donate back to their alma mater than non-former club sport participant alumni, and that they do

so more frequently and that they give greater amounts of money, on average. The second portion

of the study will involve interviewing former club sports student athletes who have donated back

2
METHODS PAPER

to the institution to determine if their experience as a club sport student athlete affected their

decision to donate.

This paper will focus solely on the qualitative portion of the paper and discuss the

methods for collecting, analyzing, and representing this data. Special focus will be paid to the

use of Qualitative Data Analysis Software (QDAS).

QDAS

QDAS platforms are a relatively recent development in qualitative data analysis. Until

the 1970s, audio recordings of interviews were difficult as portable recording devices were non-

existent (Markle, West, & Rich, 2011). ATLAS.ti, a popular QDAS platform, claims to be one of

the original QDAS systems and was only established in 1989 (Radivojevic, 2018). Prior to the

advent of QDAS, most qualitative research was conducted by hand, with notes and colored

pencils for coding themes (Ravitch & Carl, 2016). QDAS systems are helpful as they allow

researchers to organize and sort data, specifically large datasets, electronically, in one location

that is searchable rather than sorting through pages of manual transcripts and notes (Hatch, 2002;

Ravitch & Carl, 2016). Zhao et al. (2016) found that the majority of researchers who utilized

QDAS platforms did so to analyze their data more efficiently. Researchers are able to store and

analyze large sets of data in QDAS, which would be impossible to analyze by hand, and afford

researchers the opportunity to make new senses of data and to collaborate with others (Paulus,

Lester, & Britt, 2013). While efficiency and speed are important factors in the decision to use

QDAS, researchers must still be careful to not rely on software to conduct the analysis for them

(Hatch, 2002; Ravitch & Carl, 2016). As Hatch (2002) states about a QDAS platform, “It is not

and never can be a satisfactory alternative to doing the mindwork associated with analyzing and

interpreting data” (p. 207). Zhao et al. (2016) suggest that this can be overcome by ensuring that

3
METHODS PAPER

the researcher is aware of their interactions with the software and to consistently pay attention to

how the software impacts their own methodological position and decision-making process.

The use of QDAS has become much more prevalent in the past 25 years. Woods et al.

(2016) found, in a review of 763 empirical articles published from 1994 to 2013 that used either

the ATLAS.ti or NVivo QDAS platforms, which were identified as the two longest established

platforms currently in use, that the number of published articles per year increased exponentially

from 1 or 2 per year in the 1990s, to over 220 in 2012. The researchers additionally found that

73.3% of the published studies used the QDAS platforms to analyze interview data, by far the

most prevalent type of data analyzed with these platforms. The primary way that researchers

used the QDAS platforms was to analyze and manage data, with 99.6% of articles mentioning

using the platform for this purpose, followed in a distant second by 10.4% of researchers using

the software to data visualization. A mere six articles over the 20-year period claimed using the

platforms for data collection and creation.

Literature review

A literature review and a proposed methods section, which include information on the

researcher’s epistemological and ontological slants, have previously been produced and can be

viewed via the above hyperlinks.

To keep track of citations in the literature review, methods section, and any other paper

that is created in the future, the ‘References’ feature in Microsoft Word has been used by the

researcher. While RefWorks, EndNote, Zotero, and Mendeley are all more popular citation

management tools (Hensley, 2011), the researcher has a previously established library of

hundreds of citations in Microsoft Word. The references travel with each document and can also

be shared via a downloadable file and integrated into any other copy of Word. At this point in the

4
METHODS PAPER

researcher’s process, the action of transferring all references to another citation management

software is an unnecessarily cumbersome process that will be avoided.

Research site access and participant recruitment

This project will be completed at a large university that has at least 15 years of historical

club sport participant data which also has a university Foundation that is willing to share

aggregated data for the quantitative portion of the project and contact information for

purposefully selected former club sport alumni who have previously donated to the alma mater.

Institutions with this data will be identified via online inquiries on the NIRSA - Leaders in

Collegiate Recreation, the association for collegiate recreation professional’s, online community.

Messages and posts in the community can be specifically targeted to relevant staff members at

the member institutions, such as Recreational Services Directors and Club Sports Directors.

Once institutions with the required 15 years of historical records are identified, requests to the

institution’s Foundations will be made to gain access to the aggregated donation data and for the

contact information of selected alumni who had previously donated, in order to contact these

individuals to request interviews.

Purposeful sampling will be used in order to ensure that alumni from a broad range of

backgrounds, including recently graduated and older alumni, male and female alumni, alumni

who participated in different sports, and alumni who make different sizes of financial donations

are included in the interviews. Potential interview participants will be contacted via email and

offered a digital gift card as compensation for participating in the interview, which will be

delivered shortly after completing the interview.

5
METHODS PAPER

Human subjects approval

Institutional Review Board (IRB) approval will be acquired from both George Mason

University and the institution at which the project is taking place. The researcher is Citi Program

certified via the online training, which is a requirement of the George Mason University IRB

process. The researcher has experience utilizing the George Mason University IRBNet electronic

protocol management system from previously approved IRB requests.

Data collection process

As alumni who are up to 15 years post-graduation tend to move around the United States

and the world, and the cost of traveling to these various venues would be prohibitive, interviews

will be conducted virtually using video chat. Video chat was identified by Gratton and

O’Donnell (2011) as an efficient mean of conducting interviews and focus groups with

participants in distant and/or remote locations. To conduct the video chat interviews, Skype

and/or Google Hangout will be used, depending on the interviewee’s preference.

The video interviews will be recorded using SnagIt, a fee based screen capture video and

audio recording application. Participants in the interviews will be informed of the recording

ahead of time and submit an electronic consent form. Participants will also verbally be informed

that the interview is being recorded at the start of the interview and will be asked to again

provide their consent.

SnagIt will enable the interviewer, who will also be the researcher, to record any section

of their computer’s screen, which will be limited to the Skype or Google Hangout window, plus

their webcam, the audio from the computer, which will include the audio from the interviewee,

and the audio from any attached microphone, which will capture the interviewer’s voice. Once

the interview is complete, the SnagIt recording will be stopped and any dead air portions of the

6
METHODS PAPER

recording will be edited out. The file will then be converted to a .mp4 video file and saved on the

researcher’s password protected computer in a password protected file. No data will be saved

within SnagIt or Skype/Google Hangout.

As a back up to the new technology, in case an error does occur with the video recording

or file transfer, an audio recording of the interview from an external device will also occur. The

device, the researcher’s password protected iPhone, will be placed next to the computer to

capture both the interviewee and interviewer’s voices. The recording will take place using the

VoiceRecorder app for iPhone. Following each interview, the iPhone will be connected to the

researcher’s password protected computer and transferred to a password protected .mp3 audio

file on the researcher’s computer. The original recording will then be deleted from the iPhone.

Interviews will be conducted until saturation is achieved and no new information is

forthcoming from interviews, based on Lincoln and Guba’s (1985) definition of saturation. I

anticipate data saturation to be achieved after approximately 15 interviews.

Transcription process

In keeping with Woods et al.’s (2016) findings of the most popular uses for QDAS

platforms, the interviews will be uploaded to a platform and coded and analyzed using that same

platform. Depending on the results of the analysis, the data may also be visualized using the

same QDAS platform or another data visualization platform, such as Tableau. A visualization

will only be included in the reporting of the data if that format would increase the reader’s ability

to comprehend the data and not for the purposes of the visualization being a novel or ‘cool’ way

of representing data.

Transana is described as a QDAS that is designed specifically for the analysis of video

and audio data (Woods & Dempster, 2011). Saldana (2016) notes that Transana is a specialty

7
METHODS PAPER

QDAS software that allows researchers to code digital audio and video documents in a single

stored file, while Paulus, Lester, and Dempster (2014) note that Transana allows researchers to

reference their audio, video, and transcript records again easily in the future. Woods, the

developer of the software, states in Woods and Dempster (2011) that Transana works with

researchers in order to identify ways that the software can assist the researchers to perform

analyses of data that other QDAS platforms are unable to complete, such as analyzing multi-

layered datasets with multiple overlapping video and audio threads. Transana will be used to

facilitate both the transcription and analysis of the data in this project.

The transcription process in Transana will occur by first uploading the .mp4 media file,

generated by SnagIt following the recording of each interview, into the Transana media file for

the project. Transana allows the researcher to time code the video, audio, and transcript or

transcripts, as it allows users to create multiple simultaneous transcripts of the same audio and/or

video recording, of the interview simultaneously and to then sync all components together.

Transcription for this project will utilize this feature as a way to make the transcripts, audio, and

video searchable.

Currently, speech recognition software is not included in the Transana software.

However, the developers of Transana recommend a process to utilize the Windows Dictation

software that is included in the Windows 7 and above operating systems. The process involves

opening the Windows Dictation software over top of Transana, utilizing a blank transcription

window in Transana, and then playing the audio and video combinations in short segments and

then repeating the audio portion into the Dictation software while time stamping every instance

in which a participant or interviewer speaks (Transana, 2018). The process of utilizing the

8
METHODS PAPER

dictation software is believed to cut the transcription time by up to half while maintaining a

quality transcription (D. Woods, personal communications, June 12, 2018).

Data analysis process

The transcripts and videos will then be coded and analyzed using a thematic analysis.

Transana will be used for this process. Following transcription, a first reading of the transcripts

will be used to identify thematic codes. Codes are often “a word or short phrase that

symbolically assigns a summative, salient, essence-capturing, and/or evocative attribute for a

portion of language-based or visual data” (Saldana, 2016, p. 4). Ravitch and Carl (2016) identify

coding as a component of the data analysis process that involves both an inductive, data that

comes from the transcripts, and deductive, data that comes from other sources, including prior

research and existing theories, components. The authors believe that there is no right or wrong

way to determine these codes or to complete the coding process and use the example of Miles

and Huberman (1994), who suggest creating a provisional “start list” of codes based on past

findings. In this light, a provisional start list of codes will be used based on Shapiro and

Giannoulakis’s (2009) previous findings on former student athlete motivations and barriers,

however, special attention will be paid to ensure that these codes do not dictate the coding

process and allow for the discovery of additional new codes during this first reading. If any of

the start list codes do not emerge as a potential theme during the first reading, they will be

eliminated. A simple definition of coding is that it “involves reading for regularly occurring

phrases, terms, interactions, among actors, strategies and tactics, consequences, and patterns of

participation” (Miles, Huberman, Saldana, 2014, as cited in Ravitch and Carl, 2016, pp. 249-

250). This process will be undertaken in the first reading.

9
METHODS PAPER

Transana allows researchers to create their own codes in the ‘keywords’ tab. The

keywords can also be color coded. A summary report of keywords can also be created in the

software, which can be used as part of the verification process.

A second and third reading of the transcripts and synced audio and videos will be

completed in Transana. The software allows the researcher to identify sections of text, audio, or

video that relates to a theme (called a keyword), and to then create a ‘quote’ for text data, or a

‘clip’ for audio or video data, that is tied to the theme. The quote or clip can be classified as

belonging to multiple themes. The data is then visualized with a color coded segment of the text

being shown in the transcript and in a separate visualization window in the Transana software.

By the end of the third reading, saturation, the point where no new themes are forthcoming from

the readings (Ravitch & Carl, 2016), should be achieved. As suggested by Ravitch and Carl

(2016), the researcher’s advisory committee, some project participants, and some colleagues will

also be consulted with at this point to verify that saturation has in fact been achieved.

Transana is capable of producing multiple types of reports based on the way the data was

coded through quotes and clips, including reports that include all quotes and clips per identified

keyword, word frequency reports, word clouds, and digital keyword maps. These reports will be

used by the researcher in the writing process.

Digital tools to represent the qualitative data

Potential methods of digital representation of the data include using the downloadable

word cloud and digital keyword map from Transana. The keyword map displays the identified

clips and quotes along the timeline of the transcript, color coded by the color associated with the

keyword (Figure 1). This visualization would only be used if the data produces an interesting

map that tells a story of when certain themes are mentioned during the course of the interviews.

10
METHODS PAPER

Figure 1. Keyword map produce from Transana. From “Transana’s Keyword Map Reports,” by
Transana, 2015.

The word cloud feature displays words in a very quantitative fashion, with the words

used most often appearing the largest. This feature would also only be used if the resulting data

was relevant to the results of the study, not just to present an interesting new visualization item.

Another way that data may be graphically represented would be if there are any trends

that are identified, such as if females have different motivations than males, or alumni who

participated in different sports have different motivations. These representations could also be

visualized through the Piktochart platform, an online information graphic creation software, that

allows users to create documents that incorporate text and data visualizations.

Ethical concerns of utilizing digital tools

Ethical considerations must be a part of every qualitative researcher’s inquiry process

(Paulus et al., 2014). The Belmont Report, which outlined ethical considerations and guidelines

in biomedical and behavioral research in the United States, recommended that researchers follow

three basic ethical principles: 1) respect for person, by acknowledging individual’s autonomy

and protecting individuals with diminished autonomy; 2) benefice, which encompasses the

principles of “do not harm” and “maximize possible benefits and minimize possible harms”; and,

3) justice, which the report demonstrates as a formulation of ensuring that each person receives

an equal share, based on their individual need, effort, societal contribution, and merit (US

11
METHODS PAPER

Department of Health and Human Services, 1979). As new technologies emerge and are utilized

by researchers, they must consider the ethical implications and remain reflexive to ensure that

research is conducted in ethical manners (Paulus et al., 2013; Paulus et al., 2014).

For this project, a major ethical consideration is the use of audio and video recordings in

the interview process. Hatch (2002) references concerns for protecting research subject’s privacy

when video recordings are being used. When only audio recordings are used to analyze

transcripts, the subject’s privacy is better protected as it is more difficult to determine their

identity by hearing their voice than by seeing their picture in the video. In deciding to use audio

and video, rather than only audio, for this research project, this concern for privacy was

considered. The benefit for the researcher to use the video of the interviews is the potential to

feel better connected to the subjects by analyzing their body language and facial expressions as

they discuss their experiences as a former club sport athlete and their decision to donate to their

alma mater. The researcher believes that the use of video will allow him to delve deeper into the

data and the experience.

The principles of the Belmont Report have been considered in this decision. The

interview subjects will all be adults who are capable of making their own decision on whether or

not to participate in the project. The Belmont Report recommends the use of implied consent

with research subjects (US Department of Health and Human Services, 1979), which will be

done both by electronic signature form ahead of the interview and again verbally at the start of

the interview. The interviewee will be informed in both instances that audio and video recordings

of the interview will be occurring and that they have the right to refuse to participate, refuse to

answer any questions, and to end the interview at any time. As an added protection for the

12
METHODS PAPER

subjects, the interviewer will also inform the interviewees that they may request the deletion of

the video component of the interviews, but not the audio, at any time following the interview.

The subject of the interview has minimal risk for the interview subjects, which was also

considered as part of the decision to capture the video of the interviews. This follows the

Belmont Report principle of “do no harm” (US Department of Health and Human Services,

1979). The greatest risk will be the potential of the interview questions reminding the

interviewee of a negative experience. If this were to occur, the researcher would ask the subject

if they would like the video recording to be deleted.

Additional safeguards for the interviewees’ privacy will include only the researcher, his

advisory committee, and potentially other researchers being the only individuals with access to

the audio and video recordings. The recordings will not be saved on the SnagIt application, the

files themselves will be password protected, they will be kept on the researcher’s password

protected computer, and if they need to be shared, will only be done so in password protected

formats using secure digital data transfer means, such as the application Box.

The interviewees will also be given pseudonyms in all published reports of the data. One

potential risk for individual participants will be if their alma mater gains additional information

about them as individuals, not as a general societal group, that encourages the institution to target

that individual for additional financial giving. The use of pseudonyms adds a layer of anonymity

for the subject (Paulus et al., 2014) and will help diminish this concern.

13
METHODS PAPER

References

Drew-Branch, V. L. (2011). Student engagement, alumni satisfaction, and alumni donations at a

public four year institution: An analysis of how the undergraduate experience influences

alumni donations. Graduate Thesis.

Gratton, M., & O'Donnell, S. (2011). Communication technologies for focus groups with remote

communities: a case study of reserach with First Nations in Canada. Qualitative

Research, 11(2), 159-175. doi:10.1177/1468794110394068

Hatch, J. A. (2002). Doing qualitative reserch in education settings. Albany, NY: State

University of New York Press.

Hensley, M. K. (2011). Citation management software: Features and futures. Reference & User

Services Quarterly, 50(3), 204-208.

Hoyt, J. E. (2004). Understanding alumni giving: Theory and predictors of donor status. Paper

presented at the 44th Annual Forum of the Association for Institutional Research.

Boston, MA.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Thousand Oaks, CA: SAGE.

Markle, D. T., West, R. E., & Rich, P. J. (2011). Beyong transcription: Technology, change, and

refinement of method. Qualitative Social Research, 12(3).

McDonough, K. M. (2017). Young alumni perspectives about philanthropically supporting their

alma mater. Graduate Theses and Dissertations(15370). Retrieved from

lib.dr.iastate.edu/etd/15370

14
METHODS PAPER

Monks, J. (2003). Patterns of giving to one’s alma mater among young graduates from selective

institutions. Economics of Education Review, 22(2), 121-130. doi:10.1016/S0272-

7757(02)00036-5

Paulus, T. M., Lester, J. N., & Britt, V. G. (2013). Constructing hopes and fears around

technology: A discourse analysis of introductory qualitative research texts. Qualitative

Inquiry, 19(9), 639-651. doi:10.1177/1077800413500929

Paulus, T. M., Lester, J. N., & Dempster, P. G. (2014). Engaging in reflexive practice and

making ethical choices. In Digital tools for qualitative research (pp. 12-27). London,

UK: SAGE Publications Ltd. doi:10.4135/9781473957671

Radivojevic, I. (2018, April 10). Overview of ATLAS.it 8 Windows. Berlin, Germany. Retrieved

from https://youtu.be/5JNpB8vgZcs

Rau, N. E., & Erwin, T. D. (2015). Using student engagement to predict alumni donors: An

analytical model. Journal of Nonprofit Education and LEadership, 5(2), 101-111.

Ravitch, S. M., & Carl, N. M. (2016). Qualitative Research: Bridging the conceptual,

theoretical, and methodological. Thousand Oaks, CA: SAGE Publications.

Saldana, J. (2016). The coding manual for qualitative researchers. London, UK: SAGE

Publications Ltd.

Shapiro, S. L., & Giannoulakis, C. (2009). An exploratory investigation of donor constraints for

former student-athletes. International Journal of Sport Management, 10, 207-225.

Taylor, A. T., & Martin, J. C. (1995). Characteristics of alumni donors and nondonors at a

research I, public university. Research in Higher Education, 36(3), 283-302.

15
METHODS PAPER

Transana (Producer). (2015). Transana's Keyword Map Reports [Online video]. Retrieved from

https://youtu.be/LDbchRml8c0

Transana (Producer). (2018). Using speech recognition with Transana [Online Video]. Retrieved

from https://youtu.be/bVdpUyv2tx8

US Department of Health and Human Services. (1979). The Belmont Report. Washington, DC:

Author.

Woods, D. K., & Dempster, P. G. (2011). Tales from the bleeding edge: The qualitative analysis

of complex video data using Transana. Forum: Qualitative Social Research, 12(1).

doi:10.17169/fqs-12.1.1516

Woods, M., Paulus, T., Atkins, D. P., & Macklin, R. (2016). Advancing qualitative research

using qualitative data analysis software (QDAS)? reviewing potential versus practice in

published studies using ATLAS.it and NVivo, 1994-2013. Social Science Computer

Review, 34(5), 597-617. doi:10.1177/0894439315596311

Zhao, P., Li, P., Ross, K., & Dennis, B. (2016). Methodological tool or methodology? Beyong

instrumentality and efficiency with qualitative data analysis software. Qualitative Social

Research.

16