在各種網頁應用程式與服務(以下簡稱網路服務)普及化的同時，網路服務的安全與穩定成為新的議題。網路服務經常被新增或修改功能，在這些修改過程中，可能會更動到之前的程式碼，造成新的安全性問題而未發覺。為了避免此類問題，在開發過程中導入回歸測試(Regression Testing)是常見的解決方案。
然而傳統式的網路服務回歸測試只會模擬使用者輸入，並檢查網頁上輸出的資訊；對於許多網路服務而言，僅檢查輸出資訊將可能遺漏許多有用的資訊，包括：SESSION、COOKIE記錄、POST信息，或其他不會輸出在網頁上的資訊。若在這些資訊中發生錯誤，將很可能被忽略，這對於網路服務的安全性將無法達到有效的改善。
為了解決上述網路服務安全性的問題，本研究開發了一套針對網路服務的回歸測試工具，利用自動化分析的方式來進行程式嵌入(Program Instrumentation)，減輕開發人員或測試人員的負擔。

While web services become more and more popular nowadays, the security issues and the stability of the web services have been taken more seriously. Since web services can be modified often, new bugs might be created in the changes and hence causing some new security problems. To avoid the issue stated above, the common way is to use regression testing to ensure that stable features have not been broken by the changes.
However, conventional regression testing can be ineffective in addressing the problems, since the technique mostly depends on simulating user input and checking the output data shown on the web pages. Lots of useful information such as browser sessions, cookies, or http post data which could not be outputted on the pages could not be verified. Therefore if there is an error occurs below the “surface”, it is likely to be ignored.
To solve the problem, we develop a regression testing tool called “AIR Tester” for web applications. Not only it analyzes the source files of the web applications automatically, but it is able to access all information in the web application through “program instrumentation”. The goal of this tool is to increase the power of test regression so that hidden errors can be revealed.

[1] Eric S. Raymond “ESR”, “How To Become A Hacker”, 2011
http://www.catb.org/~esr/faqs/hacker-howto.html
[2] Eric S. Raymond “ESR”, Ken LEE譯, “如何成為駭客”, 1999
http://www.angelfire.com/ok/leekawo/hacker.htm
[3] Alessandro Orso, Nanjuan Shi, and Mary Jean Harrold, “Scaling regression testing to large software systems” in SIGSOFT’04/FSE-12, Oct. 31-Nov. 6, 2004, Newport Beach, CA, USA
[4] Harry M. Sneed, “Testing a Web Application”, in Proc. Sixth IEEE international Workshop on Web Site Evolution (WSE’04)
[5] Nancy J. Wahl, “An overview of regression testing” in ACM SIGSOFT Software Engineering Notes vol 24 no 1, January 1999 Page 69
[6] Gerard Meszaros, “Agile Regression Testing Using Record & Playback” in OOPSLA 2003, Oct 26-30, Anaheim, California
[7] Atif M. Memon and Mary Lou Soffa, “Regression Testing of GUIs” in ESEC/FSE’03, Sep. 1-5, 2003, Helsinki, Finland
[8] Marnie L. Hutcheson, “Software Testing Fundamentals: Methods and Metrics”, John Wiley & Sons, Inc. New York, NY, USA, 2003, ISBN:047143020X
[9] Stephen R. Schach, “Testing: principles and practice” in Computing Surveys (CSUR), Volume 28 Issue 1, Mar. 1996
[10] Peter J. D. Matthews, “When to White Box Test” in ACM SIGSOFT Software Engineering Notes, Volume 17 Issue 1, Jan. 1992
[11] Frederick P Brooks Jr., “The Mythical Man-Month”, Addison-Wesley, 1975, ISBN: 0201006502
[12] Frederick P Brooks Jr., “The Mythical Man-Month”, Page 122
[13] nihitk, “Pesticide Paradox”, 2004
http://blogs.msdn.com/b/nihitk/archive/2004/07/16/185836.aspx
[14] Zhonglei Wang, Antonio Sanchez, and Andreas Herkersdorf, “SciSim: A Software Performance Estimation Framework using Source Code Instrumentation” in Proc. of the 7th international workshop on Software and performance (WOSP’08)
[15] Marina Biberstein, Vugranam C. Sreedahar, Bilha Mendelson, Daniel Citron, and Alberto Giammaria, “Instrumenting Annotated Programs” in Proc. VEE’05 1st ACM/USENIX international conference on Virtual execution environments
[16] Bruno Cabral, Paulo Marques, and Luís Silva, “RAIL: Code Instrumentation for .NET” in Proc. OOPSLA’04 Companion to the 19th annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications
[17] Heidi Pan, Krste Asanović, Robert Cohn, and Chi-Keung Luk, “Controlling Program Execution through Binary Instrumentation” in ACM SIGRACH Computer Architecture News - Special issue on the 2005 workshop on binary instrumentation and application, Volume 33 Issue 5, Dec. 2005
[18] Elena Machkasova, Kevin Archelger, and Fernando Trinciante, “The Observer Effect of Profiling on Dynamic Java Optimizations” in Proc. of the 24th ACM SIGPLAN conference companion on Object oriented programming systems languages and applications(OOPSLA’09)
[19] Lei Xu, Baowen Xu, Zhenqiang Chen, Jixiang Jiang, and Huowang Chen, “Regression Testing for Web Applications Based on Slicing” in Proc. 27th Annual International Computer Software and Applications Conference (COMPSAC’03)
[20] Abbas Tarhini, Hacène Fouchal, and Nashat Mansour, “Regression Testing Web Services-based Applications” in Proc. IEEE International Conference on Computer Systems and Applications (AICCSA’06)
[21] Abbas Tarhini, Zahi Ismail, and Nashat Mansour, “Regression Testing Web Applications” in ICACTE, pp.902-906, 2008 International Conference on Advanced Computer Theory and Engineering, 2008
[22] Lijun Mei, Zhenyu Zhang, W. K. Chan, and T. H. Tse, “Test Case Prioritization for Regression Testing of Service-Oriented Business Applications” in WWW’09, April 20-24, 2009, Madrid, Spain
[23] Sheng Huang, Jun Zhu, and Yuan Ni, “ORTS: A Tool for Optimized Regression Testing Selection” in Proc. OOPSLA’09, 24th ACM SIGPLAN conference companion on Object oriented programming systems languages and applications
[24] Wei Jin, Alessandro Orso, and Tao Xie, “BERT: A Tool for Behavioral Regression Testing” in Proc. FSE’10 the 18th ACM SIGSOFT international symposium on Foundations of software engineering
[25] Matthew H. Netkow, Dennis Brylow, “Xest: An Automated Framework for Regression Testing of Embedded Software” in Proc. of the 2010 Workshop on Embedded Systems Education (WESE’10)
[26] Ana Cavalli, Stephane Maag, and Gerardo Morales, “Regression and Performance Testing of an e-learning Web Application: dotLRN” in 3rd International IEEE Conference on Signal-Image Technologies and Internet-Based System
[27] Gregg Rothermel, Sebastian Elbaum, Alexey G. Malishevsky, Praveen Kallakuri, and Xuemei Qiu, “On Test Suite Composition and Cost-Effective Regresssion Testing” in ACM Transactions on Software Engineering and Methodology (TOSEM), Volume 13 Issue 3, July 2004
[28] Hyunsook Do, Siavash Mirarab, Ladan Tahvildari, and Gregg Rothermel, “An Empirical Study of The Effect of Time Constraints on The Cost-Benefits of Regression Testing” in Proc. of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering (SIGSOFT’08/FSE-16)
[29] Atif M. Memon, “Automatically Repairing Event Sequence-Based GUI Testing Suites for Regression Testing” in ACM Transactions on Software Engineering and Methodology (TOSEM), Volume 18 Issue 2, November 2008
[30] IBM Rational Functional Tester
http://www-01.ibm.com/software/awdtools/tester/functional/
[31] HP WinRunner
https://h10078.www1.hp.com/cda/hpms/display/main/hpms_content.jsp?zn=bto&cp=1-11-15-24^1322_4000_100__
[32] HP QuickTest
https://h10078.www1.hp.com/cda/hpms/display/main/hpms_content.jsp?zn=bto&cp=1-11-127-24^1352_4000_314__
[33] Automation Anywhere: Testing Anywhere
http://www.automationanywhere.com/Testing/
[34] phpautotest
http://phpautotest.sourceforge.net/
[35] Web Corder
http://www.crimsonsolutions.co.uk/
[36] David N. Gray, John Hotchkiss, Seth LaForge, Andrew Shalit, and Toby Weinberg, “Modern Languages and Microsoft’s Component Object Model” in magazine Communications of the ACM Volume 41 Issue 5, May 1998
[37] David Chappell, “Understanding ActiveX and OLE: A Guide for Developers and Managers”, Microsoft Press, 1996, ISBN: 1572312165
[38] SeleniumHQ: Selenium
http://seleniumhq.org/
[39] Alexander Sirotkin, “Web Application Testing with Selenium” in Linux Journal Volume 2010 Issue 192, April 2010
[40] Vidar Kongsli, “Security Testing with Selenium” in Proc. OOPSLA’07 Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion.
[41] PHPUnit
https://github.com/sebastianbergmann/phpunit/
[42] SeleniumHQ: Selenium RC
http://seleniumhq.org/docs/05_selenium_rc.html
[43] Quality First Software: web2test
http://www.qfs.de/
[44] VERISIUM: vTest
http://www.verisium.com/products/vTest/
[45] HttpUnit
http://httpunit.sourceforge.net/
[46] Symbio Team Lite
http://www.symbioware.com/
[47] Microsoft: Internet Information Services
http://www.iis.net/
[48] Apache Software Foundation: Apache Web Server
http://www.apache.org