在開發軟體的過程中，回歸測試是必不可少的程序，它可以確保開發者修改程式後，程式仍然能夠正常運行。圖形介面運用程式是最受歡迎的運用程式，而目前市面上也已經有許多的圖形介面測試工具。不幸的是，如果程式輸出是動態的圖像，大多的測試工具都很難地去驗證其正確性。在本論文中，我們提出一個測試自動化的解決方案，稱為侵入式測試自動化（Intrusive Test Automation），透過程序探測(Program Instrumentation)的技術方案來解決某些應用程序不能忠實地重放錄製/重播測試。 所謂“侵入式”，代表我們會在原本程式碼中，插入一些側錄程式行為的程式碼，藉此來完成擷取資訊以及驗證資訊。

此外，當自動化測試系統報告有數百個測試案例失敗時，我們透過集群技術找出一群失敗測試案例的代表，開發者只需要從此叢群中開始測試來減輕開發者的負擔。 本論文中提出藉由侵入式測試中所擷取到的資訊，來將所有失敗的測試案例叢集化的方法，進而選出代表測試案例。原則上，若修正了一個代表測試案例，也可以同時修正類似的測試案例。此外，我們以一個商業軟體為個案研究，並實施測試自動化與失敗測試案例群集化，分析我們所提出的方法是否有效。

Regression testing is an indispensable process in software development, which ensures stable features have not been adversely broken by new changes. When GUI plays an important role in an application, a popular choice to automate the tests is applying GUI Capture/Replay tools. Unfortunately, in many applications which render images as output, the correctness of a replay run can no longer be straightforwardly verified. In this paper, we propose a test automation solution, called Intrusive Test Automation, which uses program instrumentation technique to collect the runtime internal information of a program. As a result, the correctness of a test run can be verified by the runtime traces.

In addition, when large number of failed test cases are reported by the test automation system, recommending some representative test cases as a start for debugging can be helpful to programmers. This paper proposes a clustering technique based on the information collected from the instrumented code. In principle, fixing bugs in one representative test case can fix its related failed test cases as well. A case study is presented to demonstrate the effectiveness of the approach.

[1] M. Ancona, A. Clematis, G. Dodero, and V. Gianuzzi. An approach to application software fault tolerance. In Proceedings of the 4th workshop on ACM SIGOPS European workshop, EW 4, pages 1–4, New York, NY, USA, 1990. ACM.
[2] C. Bertolini and A. Mota. A framework for gui testing based on use case design. In Proceedings of the 2010 Third International Conference on Software Testing, Verification, and Validation Workshops, ICSTW ’10, pages 252–259, Washington, DC, USA, 2010. IEEE Computer Society.
[3] T. S. Chow. Testing software design modeled by finite-state machines. IEEE Trans. Softw. Eng., 4:178–187, May 1978.
[4] S. R. Dalal, A. Jain, N. Karunanithi, J. M. Leaton, C. M. Lott, G. C. Patton, and B. M. Horowitz. Model-based testing in practice. In Proceedings of the 21st international conference on Software engineering, ICSE ’99, pages 285–294, New York, NY, USA, 1999. ACM.
[5] A. C. Dias Neto, R. Subramanyan, M. Vieira, and G. H. Travassos. A survey on model-based testing approaches: a systematic review. In Proceedings of the 1st ACM international workshop on Empirical assessment of software engineering languages and technologies: held in conjunction with the 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE) 2007, pages 31–36. ACM, 2007.
[6] froglogic GmbH. Squish.
[7] M. J. Harrold, R. Gupta, and M. L. Soffa. A methodology for controlling the size of a test suite. ACM Trans. Softw. Eng. Methodol., 2:270–285, July 1993.
[8] IBM. Rational functional tester.
[9] J. A. Jones and M. J. Harrold. Empirical evaluation of the tarantula automatic faultlocalization technique. In Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering, ASE ’05, pages 273–282, New York, NY, USA, 2005. ACM.
[10] J.-M. Kim and A. Porter. A history-based test prioritization technique for regression testing in resource constrained environments. In Proceedings of the 24th International Conference on Software Engineering, ICSE ’02, pages 119–129, New York, NY, USA, 2002. ACM.
[11] D. Leon, W. Masri, and A. Podgurski. An empirical evaluation of test case filtering techniques based on exercising complex information flows. In Proceedings of the 27th international conference on Software engineering, ICSE ’05, pages 412–421, New York, NY, USA, 2005. ACM.
[12] S. Needleman and C. Wunsch. A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of Molecular Biology, 48(3):443–453, Mar. 1970.
[13] Nokia. Qt-a cross platform application and ui framework, 5 2011.
[14] G. Rothermel and M. J. Harrold. Analyzing regression test selection techniques. IEEE Trans. Softw. Eng., 22:529–551, August 1996.
[15] G. Rothermel, M. J. Harrold, J. Ostrin, and C. Hong. An empirical study of the effects of minimization on the fault detection capabilities of test suites. In Proceedings of the International Conference on Software Maintenance, ICSM ’98, pages–, Washington, DC, USA, 1998. IEEE Computer Society.
[16] RPMSolution. Winrunner - as a gui based load testing tool, 05 2011.
[17] P. G. Sapna and H. Mohanty. Clustering test cases to achieve effective test selection. In Proceedings of the 1st Amrita ACM-W Celebration on Women in Computing in India, A2CWiC ’10, pages 15:1–15:8, New York, NY, USA, 2010. ACM.
[18] SmartBear. Testcomplete 8.
[19] X. Wang, L. Guo, and H. Miao. An approach to transforming uml model to fsm model for automatic testing. In Proceedings of the 2008 International Conference on Computer Science and Software Engineering - Volume 02, pages 251–254,Washington, DC, USA, 2008. IEEE Computer Society.
[20] J. A. Whittaker and M. G. Thomason. A markov chain model for statistical software testing. IEEE Trans. Softw. Eng., 20:812–824, October 1994.
[21] Wikipedia. Levenshtein distance.
[22] W. E. Wong, J. R. Horgan, S. London, and H. A. Bellcore. A study of effective regression testing in practice. In Proceedings of the Eighth International Symposium on Software Reliability Engineering, ISSRE ’97, pages 264–,Washington, DC, USA, 1997. IEEE Computer Society.
[23] Y. Yu, J. A. Jones, and M. J. Harrold. An empirical study of the effects of test-suite reduction on fault localization. In Proceedings of the 30th international conference on Software engineering, ICSE ’08, pages 201–210, New York, NY, USA, 2008. ACM.
[24] C. Zhang, Z. Chen, Z. Zhao, S. Yan, J. Zhang, and B. Xu. An improved regression test selection technique by clustering execution profiles. In Proceedings of the 2010 10th International Conference on Quality Software, QSIC ’10, pages 171–179, Washington, DC, USA, 2010. IEEE Computer Society.