AN EXPLORATORY STUDY OF FACTORS INFLUENCING M-LEARNING SUCCESS

INTRODUCTION

The rapid development of information and communication technologies (ICT) has led to the increased use of ICT in instruction and learning (8). The convergence of the mobile devices with existing educational technologies provides learners with greater flexibility by making learning available and accessible. The term m-learning is coined to describe the convergence of mobile technologies with e-learning. The development of m-learning products and the provision of mlearning opportunities are expected to be rapidly expanding. In business, for example, the importance of m-learning has been raised as many companies look into mobile technologies to support mobility of their Knowledge Management (KM) activities. The use of ICT facilitates knowledge sharing and cooperative learning among KM participants (24, 30). As the primary goal of KM is to connect people to information and people, integration of mobile technology with KM is quite natural.

Although m-learning is generally considered to increase the performance of learners by making learning accessible, no research has been done in relation to m-learning success factors from the learner's perspective. Previous researches on mobile technologies are mostly concerned with technologies such as small keyboards, limited battery life, and unreliable network connections. In contrast to the technical research, this study investigates key determinants of m-learning success perceived by users. The users are knowledgeable about what factors in Information System (IS) are affecting their satisfaction (6). Studying m-learning success from the user's perspective is critical to understand the value and efficacy of management actions and investment in m-learning.

Several measures of e-learning success have been reported, which include improvements in sales/scores and reduction in turnover (5). However, too many exogenous variables can be involved in measuring these variables. For example, Lucas (17) has shown a weak relationship between the economic performance of sales personnel and their information system utilization.

In this study, the construct, user satisfaction, is used to measure m-learning success. In IS (Information Systems) research, user satisfaction is considered to be the most critical and widely used measure of IS success (11). Since m-learning contents are delivered via either wired or wireless Internet, they are information-oriented products and services. Thus, the previous research on IS success could be applicable to the understanding of m-learning success. Yet, previous studies are geared toward the traditional data processing, which makes this study necessary.

In this study, we used previous studies and focus group interviews to identify potential determinants of m-learning success. In m-learning, goods and services are not clearly divided. M-learning users seek PDA machine and contents as well as installation assistance, software training and support. Three dimensions of user satisfaction were identified. Those were system quality, contents quality and service quality. 32 questionnaire items were generated for these dimensions. A sample of customers were asked to rate each item on a five-point scale anchored on strongly disagree (1) and strongly agree (5). This initial questionnaire was used to assess the satisfaction of customers who had recently purchased learning products of three different m-learning vendors. The data collected by questionnaire were analyzed to find the valid constructs. Then hypotheses describing the relationships between the identified constructs and user satisfaction were formulated and tested.

Understanding the determinants of m-learning success provides valuable guidance to both vendors and customers. M-learning vendors can benefit from this study by focusing on improving the factors that affect user satisfaction. Customers can benefit from this study in selection of a desirable m-learning vendor who can provide as many m-learning success factors as possible.

THEORETICAL BACKGROUND

User Satisfaction: M-learning Success Measurement

IS success are categorized into three different levels (25, 11): (1) technical level success in relation to the accuracy and efficiency of the system, (2) the semantic level success in relation to the information in delivering the intended meaning, and (3) the effectiveness level success in relation to the effect of the information on the user. Separate success measures are used for each level (11): (1) 'System Quality' for technical level, (2) 'Information Quality' for semantic level, and (3) 'User satisfaction' for effectiveness level. Among many different measures of IS success, user satisfaction is considered to be the most critical and widely used measure of IS success. Since mlearning contents are delivered via either wired or wireless Internet, they are information-oriented products and services. Thus the previous research on IS success could be applicable to the understanding of m-learning success. While user satisfaction is considered a dependent variable in this study, system quality

and contents quality (that is reconstructed from information quality) are considered as independent variables that cause the user satisfaction. This study also investigates service quality as an independent variable. These variables are discussed in the subsequent sections.

System Quality

In IS area, multiple constructs to measure the system quality have been developed. Among them, the most frequently used constructs are reliability, ease of use, flexibility and accessibility of the system (2, 3, 4, 11, 14, 27).

Reliability of the system represents the user's feeling of assurance or certainty about the system (11). In the current mlearning environment, contents are downloaded to the PC first, then moved to PDA. The following are included to measure reliability of m-learning system: (1) uninterrupted download of the contents from Internet to PC as well as from PC to PDA and (2) uninterrupted learning session. Interruption usually occurs in relation with the video contents.

The construct ease of use is associated with user-friendly design of the system (11). In this study, the following are asked to measure the ease of use in m-learning system: (1) the readability of contents in a small PDA screen, (2) the usability of the screen for inputting data and commands, and (3) understandability of on-line manual.

Flexibility of the system represents the capacity of the system to adjust in response to new conditions, demands, or circumstances (11). In this study, flexibility of the system is measured in relation to (1) upgradability of hardware and software and (2) maintainability of the system. Accessibility in mobile environment means the easiness of access to the computer system.

Content Quality

In IS effectiveness research, the information quality is considered to be important to IS effectiveness. The information quality is concerned with quality of IS output, mostly in the form of reports, which is measured by accuracy, reliability and timeliness of the information (1, 2, 16, 20, 29). Viewing the contents of m-learning system as IS output, this study includes the following to measure contents quality: accuracy, reliability and currency of the contents.

Accuracy of the contents is concerned with the correctness of the contents such as grammatical correctness. Reliability is related to the dependability of the contents such as the contents authored by the subject experts and user's questions answered by skillful instructors. Currency of contents involves the contents reflecting the latest trend in their domain.

Mainly based on the interviews with experts in m-learning company, the following items on contents quality are identified: (1) uniqueness of contents - the same contents cannot be obtained from another source; (2) appropriateness of volume the volume of the contents is appropriate for studying; (3) provision of test - tests on contents are provided (13); (4) provision of multimedia - contents consist of variety of the media such as video, audio and text; (5) variety of the difficulty level - the levels of difficulty in learning materials are appropriate; (6) variety of the subject - subjects of the contents are various to meet various needs of users; and (7) provision of edutainment - contents are interesting.

Service Quality

Virtually all tangible products have intangible attributes, and all services possess some properties of tangibility (21, 26). In many cases, a product is only a means of accessing a service. Pitt et. al argues that service quality needs to be considered as an additional measure of IS success (23). M-learning users do not just want a PDA machine and contents. Rather, they seek the system that satisfies their mobile learning needs. Thus, the following abilities of the vendor can influence user satisfaction: installation assistance, product knowledge, software training/support and online help.

In the marketing literature, the dimensions of the service quality include reliability, responsiveness and assurance (22). Reliability means ability to perform the promised service dependably and accurately. In order to measure the reliability of m-learning service, we include the following items: (1) the degree of the vendor showing a sincere interest in solving the user's problem in relation to hardware or software, and (2) the degree of the vendor providing the services such as installation and after-service at the promised times. Responsiveness is willingness to help customers and provide prompt service. Thus, the degree of vendor being responsive to user's request such as PDA usage and learning material is included to measure the responsiveness of m-learning vendor. Assurance represents knowledge and courtesy of employees and their ability to inspire trust and confidence. To measure the assurance of m-learning service, this study uses the following items: the easiness to request customer service and the correctness and understandability of FAQs.

Based on the interviews of experts working in m-learning company, we found out that users have strong wishes to communicate each other for sharing knowledge. Thus, the following items are included: (1) provision of Internet community service and/or private web-boards, (2) provision of chatting service and/or web-board for discussion, and (3) provision of web-boards for knowledge sharing.

Intellectual Ability

Individual characteristics have important effects on how people adapt to their environments (7). People differ in how they prioritize goals, how they search feedback, and how they control themselves (12, 18, 28). Thus, the model of human decision making can be enhanced by incorporating variables on the individual difference (10). In this research, learning ability of an individual learner is used as a moderator. Learning ability is measured by the class rankings of the respondents in percentage.

RESEARCH METHODOLOGY

Expert interviews and survey instrument were used in this study. Interviews with the managers of m-learning vendors were performed to verify the questionnaire items. The final questionnaire consists of 32 items for system, contents and service qualities and 1 additional item for user satisfaction. Participants responded to 5 Likert scale items where the end labeled "strongly disagree' was assigned a value of 1 and "strongly agree" was assigned at the top end. Users of three independent m-learning vendors were asked to respond the questionnaire. While two vendors provide course contents for k7-k12 curriculum, the other vendor provides foreign language instruction for foreign adults. Of the 2000 survey instruments distributed, 547 were returned and usable (27.4 percent response rate).

ANALYSIS

The validity and reliability of the instrument were evaluated. Construct validity was examined by factor analysis, using the principal components method with a varimax rotation. The factor analysis resulted in six factors, each of which has eigenvalue greater than 1. Factor loadings for each item of six factors were over 0.4, Accordingly, each item is considered to be important in interpreting the factors. Six factors were named as Contents Relevance, Contents Assurance, System Usability, System Assurance, Service Commitment and Membership Community.

Table 1 reports the factor loadings, eigen values and explained variance for each of the factors. Factor reliabilities, as represented with Cronbach's alpha in Table 2, were between 0.76 and 0.89 for each factor. The reliability coefficients above 0.60 are typically considered satisfactory (19).

TEST OF HYPOTHESES

Pearson correlation coefficients - (1) between dependent variable and each of independent variables and (2) between each pair of independent variables - were significant (p<0.01). From the results of exploratory factor analysis, we developed the following hypotheses.

H1: Contents Relevance is positively associated with User Satisfaction.

H1': Contents Relevance with higher Learning Ability for users contributes to higher level of User Satisfaction.

H2: Contents Assurance contributes to higher level of User satisfaction.

H2': Contents Assurance with higher Learning Ability for users contributes to higher level of User satisfaction.

H3: System Usability contributes to higher level of User satisfaction.

H3' System Usability with higher Learning Ability for users contributes to higher level of User satisfaction.

H4: System Assurance contributes to higher level of User satisfaction.

H4': System Assurance with higher Learning Ability for users contributes to higher level of User satisfaction.

H5: Service Commitment contributes to higher level of User satisfaction.

H5': Service Commitment with higher Learning Ability for users contributes to higher level of User satisfaction.

H6: Membership Community contributes to higher level of User satisfaction.

H6': Membership Community with higher Learning Ability for users contributes to higher level of User satisfaction.

To address the possibility that some of the constructs combined multiplicatively rather than additively in user satisfaction, we conducted stepwise regression analysis with p-in = 0.05, p-out = 0.10. As a result, a regression model, which contains all of the six factors, was proven to be significant. Rsquare value was 0.391 and adjusted R-square was 0.384. Standard Error of Estimation was 0.806.

According to Hair et al (15), the impact of moderators can be assessed via hierarchical regression analysis (HRA). In HRA, the independent and moderator variables are entered into the regression and simultaneously regressed on the dependent variable with the goal of improving the fit of the regression model. Table 3 shows results of the regression: while model 1 is without moderator, model2 is with moderator. Model 1 in Table 3 shows that all of the independent variables were significant with p-value = 0.000. Therefore, hypotheses Hl, H2, H3, H4, H5 and 116 were accepted. The most important construct which explains user satisfaction was Contents Relevance (ß=0.411). Model2 in Table 3 shows that the moderating effect was not statistically significant. That is, user's learning ability had no moderating effect on user satisfaction. Therefore, hypotheses H1', H2', H3', H4', H5', H6' were not supportive. The reason for no moderating effect could be attributed to the characteristics of respondents. That is, 66% of the sample belongs to upper middle class in student ranking and is considered to have positive attitude to technology adoption (9). Based on the results, we developed 'm-learning user satisfaction model' in Figure 1. The model in figure 1 includes six determinants of user satisfaction in m-learning: Contents Relevance, Contents Assurance, System Usability, System Assurance, Service Commitment and Membership Community.

CONCLUSION

As different computing environment requires the different criteria for quality measures, the previous research on IS effectiveness performed in the traditional data processing environment cannot be used directly in the newly formed environment, namely m-learning. Built upon previous concepts on IS Quality and Service Quality, this study developed 1Mlearning user satisfaction model.' This model includes the following factors that influence user satisfaction: Contents Relevance, Contents Assurance, System Usability, System Assurance, Service Commitment and Membership Community. This finding offers several contributions to the vendors and potential customers. This study will help m-learning vendors make decisions as to which aspect of m-learning system needs to be focused to better satisfy their customers. This study also helps the customer in selection of a m-learning vendor. The customer will seek the vendor who can provide as many mlearning success factors as possible.