Chapter 6- Discussion and Conclusion
4. RESEARCH METHODOLOGY
4.2 Data Collection
4.2.2 Measures of Validity and Reliability of Survey Instrument
Validity is assessed in a spectrum, and these include; content, face, construct, convergent, and discriminant. These forms of validity are discussed next.
4.2.2.1 Validity and Reliability of study
Validity is concerned with the extent to which the data collected actually measures the intended phenomenon (Leedy & Ormrod, 2001). Employing measures to ensure instrument validity is an imperative phase of conducting a meticulous study in the IS domain (Gefen & Straub, 2005;
Straub et al. 2004). Instrument validity is of high contemporary importance because data captured with instruments that lack rigour would threaten the scientific basis of the profession (Straub et al.
2004). A researcher therefore has a duty to ensure integrity of the instrument that captures required data for his or her investigation. The validation measures employed in this study are discussed in the subsequent section. Conversely, reliability focuses on the regularity of the research model in use (Leedy & Ormrod, 2001). It can be established through the use of multiple items to reflect each underlying construct and to statistically conclude on the internal consistency of responses to each item (Hair, Anderson, Tatham, & Black, 1998; Allen & Yen, 2002).
Measures taken to ensure reliability in this study are also discussed in the next section.
4.2.2.1.1 Content Validity
Content validity focuses on the extent to which items in a survey instrument measure their intended target, according to their conceptual definition (Rogers, 1995; Hair, Black, Babin, &
Anderson, 2014a). This form of validity is attained based on literature reviews or expert opinion
(Boudreau, Gefen, & Straub, 2001; Straub, 1989). To ensure content validity in this study, a review (pre-test40) of the survey instrument was conducted within the University of Cape Town’s- IS department. Three professors and three doctoral associates participated in the review. These individuals’ were selected based on their involvement in authoring prior related studies that are similar to the present study’s context. Additionally, they have served as journal reviewers, presented conference papers, and published on the topic of consumer use of technology-based services. Thus, their feedback aided in ascertaining the extent to which the study’s measures were appropriate for the objective of the study.
4.2.2.1.2 Face Validity
This form of validity ascertains whether the instrument captures data about its intended target (Eachus, 1999), thus an important point of protocol for the data collection phase of research (Hair et al. 2014a). To confirm the presence of face validity, a pilot-test was conducted in the city centre of Nairobi, also known as Nairobi central business district. The location was selected because it is a central business area within the city, has a several (approximately. 35941) M-pesa agent stores within its vicinity, and was thus likely to enable easy access to present users of M-pesa. The pilot study therefore ensured that the participants understood the instructions given to complete the questionnaire, could adequately interpret the items and answer the questions, were able to respond to scale items, and that actual response time was within range of the estimated 10-15 minutes required to complete the questionnaire.
For the pilot study, responses from 30 individuals’ were collected. This allowed for a preliminary statistical analysis to infer the likely proportion for the main survey (Saunders et al. 2009). For example, to determine the variance in responses; which helped identify potential issues that required rectification prior to the actual data collection phase of the study. The time frame required to complete the number of questions in the survey was assessed and determined.
Following the pilot study, revisions to item wording were made to improve clarity, and the length of time required to complete the survey was estimated to be 10 minutes. The revisions made to the survey instrument are presented in table 4.4.
40The basis for conducting a pre-test is to realize how the data collection protocols and survey instrument work in a simulated environment (Fowler, 1993, p.100.)
41 http://www.safaricom.co.ke/personal/m-pesa/m-pesa-agents/agent-locations-pdfs
Table 4.4 Revisions to survey items
Item Source Operationalized item Rephrased item post-pilot study
Task Technology Fit TTF (Larsen et al.
2009; Goodhue &
Thompson, 1995)
1. In enabling me send money, the functions of M-pesa are
appropriate
1. The money transfer deposit function of M-pesa allows me to send money.
2. In enabling me receive money, the functions of M-pesa are appropriate
2. The money withdrawal function of M-pesa allows me to receive money.
3. In general, the functions of M- pesa fully meet my money transaction needs
3. In general, the deposit (send money) and withdrawal (receive money) functions of M-pesa fully meet my money transfer needs.
Post-Usage Usefulness PUU (Bhattacherjee et al. 2008)
1. Using M-pesa for sending money increases my productivity (e.g., makes sending money faster)
1.Over the year(s), I find that using M-pesa for sending money increases my productivity (e.g., makes sending money faster, efficient, and cost effective) 2. Using M-pesa for receiving
money increases my productivity (e.g., makes receiving money faster)
2.Over the year(s), I find that using M-pesa for receiving money increases my productivity (e.g., makes receiving money faster, efficient, and cost effective)
3. Using M-pesa for sending money improves my money transfer performance (e.g., makes sending money safer)
3. Over the year(s), I find that using M-pesa meets my monetary transaction needs.
4. Using M-pesa for receiving money improves my money transafer performance (e.g., makes receiving money safer)
Confirmation (Bhattacherjee et al.
2008; Larsen et al.
2009)
1. My experience with using M- pesa was better than what I expected before using it
1. My experience using M-pesa was better than what I expected before using it.
2. The service level provided by M-pesa was better than what I expected before using it
2. The service(s) provided by M- pesa is better than what I expected before using it.
3. Overall, most of my
expectations from using M-pesa were confirmed
3. Overall, most of my
expectations from using M-pesa were met.
System Quality (DeLone & Mclean, 1992; Zhou, 2013).
1. M-pesa quickly loads all the
text and graphics 1.M-pesa quickly loads all text and images
2. M-pesa is easy to use 2.M-pesa is easy to use 3. M-pesa is easy to navigate 3.M-pesa is easy to navigate Information Quality
(DeLone& Mclean, 1992; Zhou, 2013).
1. M-pesa provides me with information relevant to my transaction needs
1. M-pesa provides me with relevant information for/about my transactions.
2. M-pesa provides me with
accurate transaction information 2. M-pesa provides me with accurate transaction information (e.g., account balance)
3. M-pesa provides me with up-to-
date information. 3. M-pesa provides me with current transaction information (e.g., account balance).
Service Quality (DeLone & Mclean, 1992; Zhou, 2013).
1. M-pesa provides on-time
services 1.M-pesa provides me with
services in a timely manner (e.g., quick response time)
2. M-pesa provides prompt
responses 2. M-pesa provides quick
responses to transaction queries (e.g., account balances) 3. M-pesa provides personalized
services 3.M-pesa provides personalized
services Utilization
(Larsen et al. 2009;
Junglas et al. 2009)
1. I utilize M-pesa for sending
money 1. I use M-pesa for sending
money 2. I utilize M-pesa for receiving
money 2. I use M-pesa for receiving
money
3. I am very dependent on M-pesa 3. I am very dependent on M- pesa for monetary transactions.
Trust
(Zhou, 2013) 1. M-pesa service provider is
trustworthy 1. M-pesa service provider
(safaricom) is trustworthy 2. M-pesa service provider keeps
its promise 2. M-pesa service provider
(safaricom) fulfils its promise(s) 3. M-pesa service provider keeps
customers’ interest in mind 3. M-pesa service provider (safaricom) keeps customers’
interest in mind Flow
Zhou 2013 1. When using M-pesa, my attention was focused on the activity
1. When using M-pesa, my attention is focused on the activity
2. When using m-pesa, I feel in
control 2. When using M-pesa, I feel in
control of the activity 3. When using m-pesa, I find a lot
of pleasure 3. When using M-pesa, my
attention is not easily diverted
4. When using M-pesa, I enjoy it.
Satisfaction
(Bhattacherjee, 2001a;
Zhou, 2013)
1.I feel satisfied with using M-
pesa 1. I feel satisfied using M-pesa
2. I feel content with using M-
pesa 2. I feel content using M-pesa
3. I feel pleased with using M-
pesa 3. I feel pleased using M-pesa
Continuance Intention Bhattacherjee (2001a);
Bhattacherjee et al.
(2008).
1. I intend to continue using M- pesa rather than discontinue its use
1. I intend to continue using M- pesa to send money
2. My intentions are to continue using M-pesa than use any alternative means
2. I intend to continue using M- pesa to receive money
3. If I could, I would like to
discontinue my use of M-pesa. 3. My intentions are to continue using M-pesa rather than use any alternative means
4. I would like to discontinue my use of M-pesa
4.2.2.1.3 Construct Validity
Construct validity is the extent to which a factor measures its target measure, and is often required where there is operationalization of constructs (Straub et al. 2004). It is concerned with whether selected items are in harmony and also if they can jointly reflect the core of the labelled construct, void of the substance of the items (Straub, 1989; Boudreau et al. 2001). Thus, construct validity is attained by eliminating the odds that latent constructs are captured by the options in the measurements, and is established by testing convergent and discriminant validity, which are discussed next.
4.2.2.1.4 Convergent and Discriminant Validity
Convergent validity is evident where each measurement item correlates strongly with its posited theoretical construct, whereas, discriminant validity is evident where each measurement item correlates weakly with all other constructs besides the one posited theoretically (Gefen & Straub, 2005). In essence, convergent validity is an assessment of whether two constructs posited to have a theoretical relationship, indeed possess such a relationship. Whereas, discriminant validity tests whether unrelated constructs posited to have no theoretical relationship, actually are distinct (John
& Benet-Martinez, 2000).
Convergent validity can be assessed by viewing the factor loadings of the items of a given construct (Chin, 1998a), the composite reliability (CR42) of each construct, and the average variance extracted (AVE43) of the constructs. Additionally, convergent validity is attained when the AVE for each construct is above 0.50 (Chin, 1998b). Thus, convergent validity requires three conditions to be met: cross-loadings should be above 0.7 and at least equal to 0.5 (Hair et al.
42 “Composite reliability is a measure of internal consistency reliability but unlike cronbach’s alpha, does not assume equal indicator loadings (Hair et al. 2014a). Indicators should be above 0.7 but levels between 0.6 and 0.7 for exploratory studies are acceptable (Hair, Hult, Ringle, & Sarstedt, 2014b, p.115)
43AVE examines the level of variance that is accounted for by a construct in view of the level of variance resulting from measurement error (Fornel & Larker, 1981)
2014a); reliability should exceed 0.7 and average variance explained (AVE) should be at least equal to 0.5 (Hair et al. 2014a). Conversely, discriminant validity is tested by identifying the square root of the average value for each construct and comparing them against their correlation with other constructs (Chin, 1998b). This requires that the AVE of a determinant should be larger than the squared correlation of itself in comparison to other determinants. Where the AVE for each construct is greater than its shared variance with any other construct, discriminant validity is established (Fornell and Larcker, 1981).
4.2.2.1.5 Reliability
Reliability is the extent to which a set of items are consistent in measuring a given construct (Straub et al. 2004). It illustrates that the operations of a study can be replicated across similar settings and yields the same results (Yin, 2003). It is also an account of measurement accuracy that seeks to eliminate the possibility of inconsistent and flawed results (Rogers, 1995). This study employs an accepted test of inter-item consistency reliability, known as Cronbach’s alpha coefficient (Cronbach 1951; Nunnally 1978; Sekaran, 2000). Cronbach’s alpha is a test of the consistency of responses to all items measuring a construct, and is reflected in the extent to which independent items of a construct correlate with each other (Sekaran 2000). Cronbach’s alpha reliability coefficient ranges from 0 to 1, and while the coefficient has no lower limit, the closer it is to 1, the stronger the reliability of the instrument (Gliem & Gliem, 2003). Accordingly, leading statisticians; Hair et al. (2014a) recommend reliability scores of 0.7 or higher as an ideal threshold. The recommended threshold of 0.7 is applied to this study.