JSER: An Empirical Study of JavaScript Errors in the Wild
JSER stands for "JavaScript Evaluation of Reliability". This project involves the empirical study of JavaScript errors that occur in production Web 2.0 applications.
Conference Paper (ISSRE'11): JavaScript Errors in the Wild: An Empirical Study
Technical Report: JavaScript Errors in the Wild: An Empirical Study (Tech Report)
Motivation
The interactivity of modern web applications is achieved primarily through the use of a client-side language called JavaScript. JavaScript differs from languages such as C and Java in many ways. For example, JavaScript is a weakly typed language, which allows the creation and execution of code at runtime. These factors are widely believed to make JavaScript prone to programming errors. In addition, web browsers are often tolerant of errors in JavaScript code, and hence, do not stop executing a web application in the event of an error. This leads to subtle bugs that are difficult to find during testing. Unfortunately, very little attention has been paid towards understanding the nature of these JavaScript errors.
The main goal of our project is to perform an empirical study of JavaScript errors in Web 2.0 applications. By doing so, we were able to identify the common categories of errors that appear in web applications, study the effects of testing speed when testing the reliability of web applications, analyze the nature of non-deterministic JavaScript errors, and study the correlations of these errors with static and dynamic characteristics of the web application.
Methodology
In this project, we analyze the JavaScript errors that appear in fifty of the top 100 websites according to alexa.com. The steps are as follows:
- Create test cases for each web application. A test case represents an interaction with a web application. Fifteen test cases - collectively known as a test suite - are created for each web application
- Replay the test suites corresponding to each web application multiple times. The test suites are replayed in three testing modes - slow, medium, and fast. Each test suite is replayed three times in each testing mode; thus, each test suite is replayed a total of nine times. JavaScript error messages are collected while a test suite is being replayed. These error messages are redirected to an "error file"
- Parse the errror file to collect error statistics. We have created our own tool that performs this parsing, thereby helping us in the analysis of the errors collected.
Results
A spreadsheet of our results is available in the following link: JSER Results