Usability Analysis of Udayana University SIMAK System Interface
on
p-ISSN: 2301-5373
e-ISSN: 2654-5101
Jurnal Elektronik Ilmu Komputer Udayana
Volume 8, No 4. May 2020
Usability Analysis of Udayana University SIMAK System Interface
Made Harry Dananjaya Adiartika a1, Luh Gede Astuti.a2
aFaculty of Math and Science, Udayana University
Denpasar, Indonesia
Abstract
With the increasingly high level of exchange of information at this time, the demand for a system capable of managing all of that information is getting higher, where one of these systems is the Udayana University Student Management Information System (SIMAK) which has a vital role in the academic administration of Udayana University, which of course requires an interface that is easy to use so that it can speed up the process of academic administration, where it is one of the things that is often taken into consideration when designing an interface that is easy to use or user-friendly, to find out whether an interface from a user-friendly system can be done by testing the usability aspects of the system. Usability is one of the important aspects that must be fulfilled for a system where to find out whether the Usability aspect of a system is fulfilled, it can be tested using the Usability Testing method with questionnaire media, this study aims to examine the Usability aspects of the SIMAK system at Udayana University using the Usability method Testing with questionnaire media, where the results of this study indicate that the SIMAK system as a whole has met the Usability component with an average value of 3, which means that the SIMAK system of Udayana University already has a good Usability aspect value.
Keywords: User Interface, Usability, Usability Testing, SIMAK, Questionnaire
With the increasingly high level of exchange of information at this time, the demand for a system that is able to manage the amount of information is increasingly high, this can be seen by the rise of systems developed in various fields such as health, safety to education, where most of these systems website based, this is due to its nature that is not limited by the device, therefore there are various factors to consider when developing a website based system, where one of the things that is often taken into consideration in building a website based system is the design or interface design, interface design of a system that is designed must be user-friendly, in this case the design of the interface design must prioritize the ease of users in using the system, one aspect that can be used as an indicator in knowing whether an interface of a system user-friendly, namely by testing the Usability aspects of the system.
Usability is arguably the convenience and ability of users to use and learn an object, Nielsen [1] mentioned that for a website usability is a condition that must be met to survive, this is because if a website is difficult to use then, the user will leave, if a user is confused in using a website, the user will leave, if the information on a website is difficult to read or fail to answer the user's question, the user will leave, where Nielsen also mentioned that there is no name for a user reading the manual of a website let alone taking the time to study the interface, because there are many other websites available, thus it is important to have an interface from a website to be easily used by users.
There are many methods and approaches that can be used to test the usability aspects of a system where one of them is by doing Usability Testing. Usability Testing is not a new
method used to test the Usability aspects of a system, where Usability Testing is a method that involves the user to use the system followed by carrying out special tasks in using the system, there are various methods used as instruments in data collection, where one of the most frequently used methods is the use of questionnaires as data collection instruments, there are many forms of questionnaires used in usability analysis, for example Krisnayani & Arthana. in 2016 [2] using Heuristic Evaluation for Usability Analysis of UNDIKSHA Website, Dewi, et al. in 2018 [3], using the Heuristic Evaluation and Webuse methods in the case study questionnaire data evaluation Usability Analysis Mobile application Ordering Prime Taxi Services, Mustikaningtyas, et al. in 2016 [4], Using Heuristic Evaluation in Usability Analysis in Brawijaya University Website, Santari & Rahayuda in 2017 [5], using Heuristic Evaluation in Usability analysis on STIKOM Bali Almuni Website, Kusuma, et al. in 2016[6] using USE Questionnaire for usability analysis of Sistem KRS Online UMM, Marthasari & Hayatin. in 2017 [7], using USE Questionnaire in Usability analysis of Gegulang Lective System, Ricky in 2018 [8], used USE Questionnaire in Usability Testing on SIPOLIN application in West Java Province.
Udayana University is a state higher education institution that prints human resources in both academic and non-academic fields and is the first and oldest university in Bali, which is currently approximately 20658 [9] students are studying at Udayana University [2]. With a large number of students, of course, requires a good administration system in order to facilitate the management of the campus academic administration process. In Academic Administration, Udayana University has a SIMAK (Student Management Information System) system which is incorporated into a system called the IMISSU (Integrated Management Information System of Unud) which is a portal system used to access all systems used in the academic administration of Udayana University, SIMAK system has an important role in the academic administration process of Udayana University especially for Udayana University students.
With this description, in the case study of this research we aim to examine the Usability aspects of the Udayana University SIMAK system using the Usability Testing method with questionnaire media, so it is hoped that this research can provide a further insight or insight on the usability aspects of the Udayana University SIMAK system.
The interface is a space where humans and machines interact, with the aim of the interaction is to provide a control over the operation of these machines from the human side, the interface can also be said as a series of graphical displays that can be understood by computer users and programmed in such a way that it can legible by the computer's operating system and operating as it should.
SIMAK (Sistem Informasi Manajemen Mahasiswa) is an academic information system that was built to provide convenience to users in campus academic administration activities online [10], The SIMAK system is part of the IMISSU system (Integrated Management Information System of Unud) which is a Single Sign-On (SSO) system that provides access portals for all Udayana University systems, the SIMAK system is a system that has an important role in the academic administration process of Udayana University starting from the process of admission of new students (PMB), making curriculums, making class schedules, filling out study plan cards (KRS), charging grades, managing data of lecturers and students besides this system can also function as a support for data analysis in determining campus decisions.
Usability can be defined as the ease of use and the ability to study a man-made object, the word usability can also refer to the methods used to improve the ease of use in the Nielsen design process [1] states that usability can be defined by 5 components :
-
a. Learnability: shows the user's ease in learning the main system functions and the user's ability to achieve the skills needed to do a job.
-
b. Efficiency: How fast the user's ability to learn, use the system and carry out
their work using the system/application
-
c. Memorability: the ability of the user to recognize and remember the system so
that the user can use the system/application without having to re-learn.
-
d. Error: the number of errors that users can generate and how the user's ability to resolve these errors.
-
e. Satisfaction: user satisfaction or comfort when using the system/application.
Folmer in 2004 [11] mentions that there are several usability methods that focus on the user, these methods are Usability Testing, Usability Inspection, and Usability Inquiry, Barnum in 2010 [12] states that Usability testing can be defined simply as a process or activity that focuses on observing users who work with a product that are tangible and meaningful to them.
By using this definition [12] subdividing the testing into two type, depending on the point at which it is done and the goal for the study
-
a. Formative testing : done when the product is still in the development stage with the aim of diagnosing and correcting problems, usually based on small studies and repeated during development
-
b. Summative testing : done after the product is finished, with the aim of determining a metric baseline to validate that the product meets the requirements, usually requiring a greater amount of statistical validity
The questionnaire is an instrument in the form of a series of questions aimed at the interests of gathering information from a respondent, The questionnaire was prepared in a variety of ways used in many different situations using many different data collection media. the questionnaire contains a list of structured questions with alternative answers available so that the respondent simply chooses the answer according to aspirations, perceptions, attitudes, circumstances, or personal opinions[13], In taking data using the questionnaire media, of course, there will be bias, wherein an effort to avoid such bias, we need a form of testing of the questionnaire, one form of testing that is often used for questionnaires is testing or validity test used to test the validity of the data that was got from the questionnaire, as well as a reliability test where the questionnaire will be tested whether the same questionnaire can produce different data or results at different times.
Before the research is carried out in advance, a literature review is conducted on related matters needed to conduct a usability analysis of the online-based system from articles, journals, and other literature both nationally and internationally.
Primary data collection was carried out by questionnaire method through online survey media, where respondents were selected using the purposive random sampling method, this study took responses from respondents totaling 34 respondents from Udayana University students. This is based on research conducted by Neilsen in the "Quantitative Study: How Many Users to Test" in that study Nielsen revealed a result, where a quantitative study was recommended using at least 20 respondents, this was based on the results of his research, where Nielsen revealed that testing usability with respondents numbering 20 respondents produces a result that is not much different by using a larger number of respondents, so that with a smaller number of respondents can reduce costs and excess time, and based on the results found by Nielsen[3], This study took 34 respondents from various faculties of Udayana University as respondents in this usability analysis.
To obtain data, Likert Scale is used as a reference in providing assessment weight, where the linker scale is one type of scale commonly used in the research process using a questionnaire, Likert scale was devised in order to measure human attitude or individual traits, such as knowledge or attitudes in scientifically accepted and validated manner, Linkert Scale is a set of statements (items) offered for real or hypothetical situation under study, which participant are asked to show their level of agreement (from strongly disagree to strongly agree) with the given statement (items) on a metric scale[14]. which usually measured using a total score of several questions to measure individual behavior by responding to 5 to 7 choice points on each question.[15].
This study uses a scale of 1 to 5 which is used as a range of dimensions of the variables analyzed to obtain ordinal data where the items used will be adjusted to the questionnaire used, the following details and scores from the scale
Table 1. Likert Scale Score
|
Answers |
Score |
|
Not So Easy |
1 |
|
Not Easy |
2 |
|
Quite Easy |
3 |
|
Easy |
4 |
|
Very Easy |
5 |
Validity testing uses the Pearson Correlation method or also called the Pearson product correlation, Pearson correlation coefficients are usually used for jointly distributed data (data that follows the bivariate normal distribution). Pearson's correlation coefficient will be scaled in such a way that it ranges from -1 to +1, where 0 indicates that there is no linear or monotonic relationship, and the relationship gets stronger and finally approaches the straight line, when the coefficient approaches the absolute value 1 [16]. Pearson correlation can be formulated as follows :
r _ n ∑XY-(∑X)(∑Y)
(1)
xy √{n∑X 2 -(∑X)2}{n∑Y 2 -(∑Y)2}
Explanation :
rxy = Pearson correlation coefficient
-
n = Number of samples
-
x = First independent variable / variable
y = Dependent variable / second variable
Reliability testing uses the Cronbach's Alpha method, Cronbach’s Alpha describes the reliability of a sum (or average) of q measurement where the q measurement may represent q raters, occasions, alternative forms, or questionnaire/test items[17]. Cronbach's Alpha is the most commonly used method to measure reliability, where this method is most often used when wanting to test a number of Likert questions in the form of surveys or questionnaires that form a scale and want to know whether the scale is reliable, Cronbach’s Alpha can be formulated as follows:
N ∙C
(2)
V+(N-1)∙C
Explanation :
N = Number of Samples
_
= Average variance
= Covariance between items
The first stage in usability testing is to provide a number of tasks that have been prepared in advance for the user when accessing the system being tested, the tasks are given to 34 respondents who come from students of several faculties at Udayana University who already know and use the SIMAK system so respondents do not experience difficulties in carrying out these tasks:
Table 2. Task
|
No |
Task |
|
1 |
Open the SIMAK menu and pay attention to the menus |
|
2 |
Access all menus found on SIMAK |
|
3 |
Search for information on the Job Training menu, Student Profiles, Final Task List, KRS, Journal Publications |
|
4 |
Input information on the Job Training menu, Student Profiles, Final Task List, KRS, Journal Publications |
|
5 |
Editing personal information data on the Job Training menu, Student Profile, Final Task List, KRS, Journal Publication |
The following is an explanation of each of these tasks.
-
1. The user opens the SIMAK menu and pays attention to the menus on the system
-
2. The user accesses all menus found on the SIMAK system.
-
3. The user searches for information on all menus in the SIMAK system
-
4. Users enter information on the Job Training menu, Student Profiles, Final Task List, KRS and Journal Publications
-
5. Users Edit Personal Information on the Job Training menu, Student Profile, Final Task List, KRS, Journal Publication
Questionnaires are given to respondents who have performed the tasks outlined in table 2. with the aim that respondents gain experience of using the system interface that is tested both the things seen and felt by the respondent when performing the tasks given, from which the tasks are made a questionnaire consisting of 15 questions divided into five components of usability.
Table 3. Usability Questionnaire Form
|
No |
Question |
Answers | ||||
|
Not So Easy |
Not Easy |
Quite easy |
Easy |
Very easy | ||
|
Learnability | ||||||
|
1 |
Do you easily learn the functions of menus that are on the SIMAK system? | |||||
|
2 |
Are the interface display on each menu on the SIMAK system easy to understand? | |||||
|
3 |
Is the SIMAK system easy to use or operate? | |||||
|
Efficiency | ||||||
|
4 |
Can you get the information you want quickly and easily? | |||||
|
5 |
Can you access every menu on the system quickly? | |||||
|
6 |
Can you input information easily on the menus on the SIMAK system? | |||||
|
Memorabilit |
∕ | |||||
|
7 |
Can you easily use the SIMAK system after not using it for a long time? | |||||
|
8 |
Can you easily remember the functions of each menu on the SIMAK system? | |||||
|
9 |
Can you easily remember the location of each function on the SIMAK system? | |||||
|
Error | ||||||
|
10 |
Do you easily understand the errors that arise? | |||||
|
11 |
Can you easily find a solution to the errors that appear? | |||||
|
12 |
Do you easily encounter an error on the SIMAK system? | |||||
|
Satisfaction | ||||||
|
13 |
Is the color display on the SIMAK system comfortable and not boring? | |||||
|
14 |
Can the icons, colors, symbols, and images use on the SIMAK system be understood? | |||||
|
15 |
Do you feel comfortable with menu placement on the SIMAK system? | |||||
Of the 34 respondents who answered the questionnaire 102 data were obtained for each component of usability so that a total of 510 data was obtained,
Validity Testing uses the Pearson correlation method with degree of freedom (df) = n – 2 and significance value of 5% or 0.05, then the amount of data (n) is 32, obtained r table of 0.3388, after testing it was obtained in the form of a correlation variable Learnability to usability = 0.750, Efficiency = 0.647, Memorability = 0.795, Error = 0.550, and Satisfaction = 0.770, where after testing 2 sides with r count > r table (0.3388) it can be said that all component variables are declared valid.
Reliability test using Cronbach's Alpha method with degree of freedom (df) = n – 2 and significance value of 5% or 0.05, then the amount of data (n) is 32, obtained r table of 0.3388, after testing the Cronbach's Alpha value obtained from the questionnaire was 0.516 and the value was tested using the 2 sided r approach count = 0.775> r table = 0.3388, it can be concluded that the questionnaire can be declared reliable.
Next is to recap the results of the questionnaire that has been distributed to 34 respondents, where we get the average calculation results of the usability testing results above, then we get a recap of the usability values that can be seen in Table 4 below:
Table 4. Usability Score Recap Results
|
No I |
Question ∖ |
Score | |
|
Learnability | |||
|
1 |
Do you easily learn the functions of menus that are on the SIMAK system? |
3.2 | |
|
2 |
Are the interface display on each menu on the SIMAK system easy to understand? |
3.1 |
|
3 |
Is the SIMAK system easy to use or operate? |
3.3 |
|
Efficiency | ||
|
4 |
Can you get the information you want quickly and easily? |
2.9 |
|
5 |
Can you access every menu on the system quickly? |
3.1 |
|
6 |
Can you input information easily on the menus on the SIMAK system? |
3.2 |
|
Memorability | ||
|
7 |
Can you easily use the SIMAK system after not using it for a long time? |
3.2 |
|
8 |
Can you easily remember the functions of each menu on the SIMAK system? |
3.2 |
|
9 |
Can you easily remember the location of each function on the SIMAK system? |
2.9 |
|
Error | ||
|
10 |
Do you easily understand the errors that arise? |
3.2 |
|
11 |
Can you easily find a solution to the errors that appear? |
2.9 |
|
12 |
Do you easily encounter an error on the SIMAK system? |
2.3 |
|
Satisfaction | ||
|
13 |
Is the color display on the SIMAK system comfortable and not boring? |
3.3 |
|
14 |
Can the icons, colors, symbols, and images use on the SIMAK system be understood? |
3.3 |
|
15 |
Do you feel comfortable with menu placement on the SIMAK system? |
3.3 |
Based on the results of the usability score recap as shown in table 4, it can be seen that from the responses of 34 respondents to 15 questions, it was found that overall the SIMAK system received an average value of 3, where:
-
1. Learnability gets an average score of 3.2 this shows that the SIMAK system has been built so that users can easily learn and understand the functions of the menus contained in the system and can achieve the skills needed to work on or find information that students need.
-
2. Efficiency gets an average score of 3.1 this shows that the SIMAK system has been built so that students can access, obtain and input information quickly so that it can make it easier for students to access, obtain and input the information they need.
-
3. Memorability gets an average score of 3.1 this shows that the SIMAK system has been built so that students can recognize and recall the functions and features of the system without the need for students to learn again.
-
4. Errors get an average score of 2.8, this shows that the Udayana University SIMAK system has been built to minimize the possibility of users finding an error and has been built to help users understand and find solutions to errors that arise.
-
5. Satisfaction gets an average score of 3.3 this shows that the SIMAK system has been built by prioritizing student satisfaction or comfort when using the SIMAK system.
Based on the recap results of the Udayana University SIMAK system usability value taken from the responses of 34 student respondents obtained user usability values by an average of 3, which can be said that the Udayana University SIMAK system has a good usability value and overall the Udayana University SIMAK system has fulfilled aspects Reusability
Tests conducted in this study although using a small number of respondents, amounting to 34 people, the results obtained can be used to provide a good insight into the Usability aspects of the SIMAK system of Udayana University.
There are some suggestions that the author can provide:
-
1. Subsequent research can develop the results of testing the usability aspects of the Udayana University SIMAK system by conducting further studies by adding the number of respondents used and analyzing them quantitatively, so that they can provide a picture or further insight on the usability aspects of the Udayana
University SIMAK system.
-
2. Subsequent research can develop this research by adding a method to study the usability of the Udayana University SIMAK system by studying it through usability inspection so that it can improve the accuracy and results of the Udayana University SIMAK system usability analysis.
References
-
[1] J. Nielsen, “Usability 101: Introduction to usability. Nielsen Norman Group. 2003.” 2018.
-
[2] I. G. M. D. Putu Krisnayani, I Ketut Resika Arthana, “Analisa Usability Pada Website
Undiksha Dengan Menggunakan Metode Heuristic Evaluation,” KARMAPATI (Kumpulan Artik. Mhs. Pendidik. Tek. Inform. ISSN 2252-9063, vol. 5, no. 2, 2016.
-
[3] I. K. Dewi, Y. T. Mursityo, and R. R. P. Mardi, “Analisis Usability Aplikasi Mobile
Pemesanan Layanan Taksi Perdana Menggunakan Metode Webuse dan Heuristic Evaluation,” J. Pengemb. Teknol. Inf. dan Ilmu Komput. Univ. Brawijaya, vol. 2, no. 8, pp. 2909–2918, 2018.
-
[4] B. A. Mustikaningtyas, M. C. Saputra, and A. Pinandito, “Analisis usability pada website
universitas brawijaya dengan heuristic evaluation,” J. Teknol. Inf. dan Ilmu Komput., vol.
3, no. 3, pp. 188–192, 2016.
-
[5] N. P. L. Santiari and I. G. S. Rahayuda, “Analisis Usability Pada Website Alumni
STIKOM Bali Dengan Heuristic Evaluation,” E-Proceedings KNS&I STIKOM Bali, pp. 519–523, 2017.
-
[6] W. A. Kusuma, V. Noviasari, and G. I. Marthasari, “Analisis Usability dalam User
Experience pada Sistem KRS Online UMM menggunakan USE Questionnaire,” J. Nas. Tek. Elektro dan Teknol. Inf., vol. 5, no. 4, pp. 294–301, 2016.
-
[7] G. I. Marthasari and N. Hayatin, “Analisis Usability Terhadap Sistem Lective Gegulang
Berbasis USE Questionnaire,” in Prosiding SENTRA (Seminar Teknologi dan Rekayasa), 2017, no. 3.
-
[8] R. Firmansyah, “Usability Testing Dengan Use Questionnaire Pada Aplikasi Sipolin
Provinsi Jawa Barat,” Swabumi, vol. 6, no. 1, pp. 1–7, 2018.
-
[9] “Profile Perguruan Tinggi: Universitas Udayana.” [Online]. Available:
https://forlap.ristekdikti.go.id/perguruantinggi/detail/NEFGNzM0OTgtMzgwMy00RjQ5LU FBNjQtQ0FGODJDNTU0Q0NF. [Accessed: 02-Jul-2019].
-
[10] “SIMAK-NG | Universitas Udayana.” [Online]. Available: https://simak-ng.unud.ac.id/home. [Accessed: 02-Jul-2019].
-
[11] E. Folmer and J. Bosch, “Architecting for usability: A survey,” J. Syst. Softw., vol. 70, no. 1–2, pp. 61–78, 2004.
-
[12] C. M. Barnum, Usability testing essentials: ready, set... test! Elsevier, 2010.
-
[13] E. Nugroho, Prinsip-prinsip Menyusun Kuesioner. Universitas Brawijaya Press, 2018.
-
[14] A. Joshi, S. Kale, S. Chandel, and D. K. Pal, “Likert scale: Explored and explained,” Br.
J. Appl. Sci. Technol., vol. 7, no. 4, p. 396, 2015.
-
[15] W. Budiaji, “Skala Pengukuran dan Jumlah Respon Skala Likert (The Measurement Scale and The Number of Responses in Likert Scale),” J. Ilmu Pertan. dan Perikan. Desember, vol. 2, no. 2 Hal, pp. 127–133, 2013.
-
[16] P. Schober, C. Boer, and L. A. Schwarte, “Correlation coefficients: appropriate use and interpretation,” Anesth. Analg., vol. 126, no. 5, pp. 1763–1768, 2018.
-
[17] D. G. Bonett and T. A. Wright, “Cronbach’s alpha reliability: Interval estimation, hypothesis testing, and sample size planning,” J. Organ. Behav., vol. 36, no. 1, pp. 3–15, 2015.
495
Discussion and feedback