Teburin Abubuwan Ciki
1. Gabatarwa
Shekaru goma da suka wuce sun ga girma mai yawa a cikin APIs na yanar gizo, musamman RESTful APIs waɗanda ke bin tsarin gine-ginen REpresentational State Transfer. Sabis na yanar gizo na zamani suna ba da REST APIs akai-akai don abokan ciniki su sami damar yin amfani da ayyukansu, suna haifar da haɓaka dabarun gwaji na kwalliya da kayan aiki da yawa.
Wannan binciken yana magance kalubalen kwatanta kayan aikin gwaji na REST API waɗanda aka kimanta su a cikin saituna daban-daban tare da ma'auni daban-daban. Muna gabatar da cikakken bincike na farko na kwarai wanda ke gano kayan aikin masana da na masu aiki, yana nazarin halayen code da ke shafar aikin kayan aiki, gudanar da cikakken bincike na gazawa, da kuma gano takamaiman hanyoyin bincike na gaba.
Kayan Aiki 10 da aka kimanta
Gami da kayan aikin ilimi da masana'antu
Ayyuka na Duniya 20
RESTful APIs na buɗe tushe a matsayin ma'auni
Ma'auni Mafi Muhimmanci 2
ɗaukar hoto na code da gazawar musamman da aka gano
2. Hanyar Aiki
2.1 Zaɗin Kayan Aiki
Mun yi cikakken bincike na wallafe-wallafen da ya gano kayan aikin ilimi 8 da kayan aikin masu aiki 11. Bayan amfani da ma'aunin zaɓi ciki har da samuwa, takaddun bayanai, da matsayin kulawa, mun zaɓi kayan aiki 10 na zamani don cikakken kimantawa.
2.2 Ayyukan Benchmark
Ma'aunin mu ya ƙunshi ayyukan RESTful 20 da aka zaɓa daga ayyukan da ke da alaƙa da binciken GitHub. Ma'aunin zaɓi sun haɗa da:
- Aiwar buɗe tushe na Java/Kotlin
- Samuwar ƙayyadaddun OpenAPI
- Ƙaramin dogaro ga albarkatun waje
- Amfani da rikitarwa na duniya
2.3 Ma'aunin Kimantawa
Mun kimanta kayan aiki ta amfani da ma'auni na farko guda biyu:
- ɗaukar hoto na Code: ɗaukar hoto na layi, ɗaukar hoto na reshe, da ɗaukar hoto na hanya da aka auna ta amfani da JaCoCo
- Gano Gazawa: Gazawar musamman da aka haifar, an rarrabe su ta nau'i da tsanani
3. Sakamakon Gwaji
3.1 Nazarin ɗaukar hoto na Code
Sakamakonmu ya nuna bambanci mai yawa a cikin ɗaukar hoto na code da kayan aiki daban-daban suka samu. Mafi kyawun kayan aiki sun sami ɗaukar hoto na layi har zuwa 78%, yayin da wasu suka yi ƙoƙari su kai 30%. ɗaukar hoto ya kasance mai kalubale musamman ga code na sarrafa kuskure da rikitarwar dabarun kasuwanci.
Hoto 1: Kwatanta ɗaukar hoto na code a cikin kayan aikin gwaji 10. Kayan aikin da ke amfani da algorithms na juyin halitta da aiwar alama sun ci gaba da fifita hanyoyin gwaji bazuwar.
3.2 Gano Gazawa
Kayan aiki sun bayyana gazawar musamman 247 a cikin ayyukan benchmark. Nau'ikan gazawa sun haɗa da:
- Kurakuran Seva na Ciki HTTP 500 (42%)
- Buƙatar mara kyau HTTP 400 (28%)
- Bambance-bambancen maƙasudin Null (15%)
- Yoyon albarkatu (8%)
- Sauran keɓancewa (7%)
3.3 Kwatanta Kayan Aiki
Babu wani kayan aiki guda ɗaya da ya yi rinjaye a cikin duk ma'auni. Kayan aiki sun yi fice a fagage daban-daban:
- EvoMaster: Mafi kyawun ɗaukar hoto gabaɗaya
- RESTler: Mafi inganci don gwajin API mai ɗaukar hali
- Schemathesis: Kyakkyawa don tabbatar da tsari
4. Nazarin Fasaha
4.1 Tsarin Lissafi
Matsalar samarwa na gwaji za a iya tsara ta a matsayin matsala mai ingantawa. Bari $T$ ya zama saitin shari'o'in gwaji, $C$ ya zama ma'aunin ɗaukar hoto, kuma $F$ ya zama saitin gazawa. Manufar ita ce haɓaka:
$$\max_{T} \left( \alpha \cdot \text{cov}(T, C) + \beta \cdot \sum_{f \in F} \mathbb{1}_{f \text{ detected by } T} \right)$$
inda $\alpha$ da $\beta$ ma'auni ne, kuma $\text{cov}(T, C)$ yana auna yadda saitin gwaji $T$ ya gamsar da ma'aunin ɗaukar hoto $C$.
4.2 Aiwar Algorithm
Ga sauƙaƙan pseudocode don samar da gwajin REST API:
function generateTests(apiSpec, maxTime):
testSuite = initializeTestSuite()
population = initializePopulation(apiSpec)
while timeElapsed < maxTime:
for individual in population:
testCase = decodeIndividual(individual)
coverage, failures = executeTest(testCase, apiSpec)
fitness = calculateFitness(coverage, failures)
updateIndividualFitness(individual, fitness)
population = selectAndReproduce(population)
population = mutatePopulation(population, apiSpec)
testSuite.updateBestTests(population)
return testSuite.getBestTests()5. Hanyoyin Gaba
Dangane da bincikenmu, mun gano hanyoyin bincike masu ban sha'awa da yawa:
- Hanyoyin Haɗin kai: Haɗa dabarun gwaji da yawa
- Koyon Injin: Yin amfani da ML don hasashen shigarwar gwaji masu ban sha'awa
- Containerization: Mafi kyawun sarrafa dogaro na waje
- Gwajin Tsaro: Miƙa wa gano raunin tsaro na API
Bincike na Asali
Wannan binciken na kwarai yana wakiltar ci gaba mai mahimmanci a cikin binciken gwajin REST API ta hanyar ba da kwatanta cikakke na farko na kayan aikin ilimi da na masana'antu. Binciken ya bayyana cewa, duk da cewa an sami ci gaba mai yawa, har yanzu akwai ɗaki mai yawa don ingantawa, musamman a cikin samun daidaitaccen ɗaukar hoto mai girma a cikin aiwar API daban-daban.
Hanyar binciken ta dace da kafaffen ayyukan injiniyan software na kwarai, kama da ingantaccen tsarin kimantawa da ake gani a cikin ayyukan tushe kamar takardar CycleGAN (Zhu et al., 2017), wanda ya kwatanta samfuran samarwa da yawa a tsari. Koyaya, ba kamar mayar da hankali na CycleGAN akan fassarar hoto ba, wannan aikin yana magance ƙalubale na musamman na gwajin REST API, gami da hulɗar jiha da dogaro na bayanai masu rikitarwa.
Wani fahimta mai mahimmanci ita ce cinikin tsakanin dabarun gwaji daban-daban. Kayan aikin da suka dogara da algorithms na juyin halitta, kama da waɗanda aka yi amfani da su a cikin gwajin software na tushen bincike (Harman & Jones, 2001), sun nuna mafi girman ɗaukar hoto amma suna buƙatar ƙarin albarkatun lissafi. Wannan yana maimaita binciken daga IEEE Transactions on Software Engineering game da yawan albarkatun hanyoyin gwaji masu zurfi.
Binciken gazawa ya bayyana cewa kayan aiki na yanzu suna da tasiri musamman wajen gano kurakuran aiwarai kai tsaye amma suna fama da kurakuran dabarun kasuwanci masu rikitarwa. Wannan iyaka tana kama da ƙalubalen da aka gano a cikin binciken ACM Computing Surveys na iyakokin gwaji na kwalliya (Barr et al., 2015), inda fahimtar ma'ana ta kasance babban shamaki.
Idan aka duba gaba, haɗa manyan samfuran harshe don samar da gwaji, kamar yadda aka bincika a cikin aikin kwanan nan daga Google Research da Microsoft Research, zai iya magance wasu iyakoki na yanzu. Koyaya, kamar yadda aka lura a cikin bugu na arXiv na masu bincike daga Stanford da MIT, ana buƙatar ingantaccen tabbaci don tabbatar da irin waɗannan hanyoyin suna yaduwa a cikin tsarin API daban-daban.
Gudunmawar binciken don kafa ma'auni daidaitaccen ma'auni yana da mahimmanci musamman, kama da tasirin ImageNet a cikin hangen nesa na kwamfuta. Ta hanyar samar da tsarin kimantawa na gama gari, wannan aikin yana ba da damar kwatanta ma'ana da sauri a cikin fagen, yana yiwuwa yana rinjayar haɓaka kayan aiki na gaba a cikin saitunan ilimi da na masana'antu.
6. Bayanan da aka ambata
- Kim, M., Xin, Q., Sinha, S., & Orso, A. (2022). Automated Test Generation for REST APIs: No Time to Rest Yet. In Proceedings of ISSTA '22.
- Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. In Proceedings of ICCV.
- Harman, M., & Jones, B. F. (2001). Search-based software engineering. Information and Software Technology.
- Barr, E. T., Harman, M., McMinn, P., Shahbaz, M., & Yoo, S. (2015). The Oracle Problem in Software Testing: A Survey. IEEE Transactions on Software Engineering.
- Martin-Lopez, A., Segura, S., & Ruiz-Cortés, A. (2021). RESTest: Black-Box Testing of RESTful Web APIs. In Proceedings of ICSOC.
- Atlidakis, V., Godefroid, P., & Polishchuk, M. (2019). RESTler: Stateful REST API Fuzzing. In Proceedings of ICSE.
- Arcuri, A. (2019). RESTful API Automated Test Case Generation with EvoMaster. ACM Transactions on Software Engineering and Methodology.