ࡱ> cebq`bjbjqPqP>$::, ####%%%$%L)L)L)8))$%wt)))))++ +wwwwwww$zhk|4w%+++++4w##))Iw...+#)%)w.+w..:cx$^%Zj)) xL)]-g4v,_w0wgt|-j|hZjZj,|%j\ ++.+++++4w4wU.^+++w++++$%$%$%()$$%$%$%()$%$%$%###### Resumo: Ao realizar uma Reviso Sistemtica de Literatura (RSL), os pesquisadores geralmente enfrentam o desafio de projetar um protocolo que equilibre adequadamente a qualidade dos resultados e os esforos da reviso. Por um lado, usar buscas em bibliotecas digitais (ou bancos de dados) ou snowballing sozinhas no seria suficiente para alcanar resultados de alta qualidade. Por outro lado, o uso conjunto de buscas em bibliotecas digitais e snowballing pode aumentar o esforo geral da reviso. O objetivo desta pesquisa propor e avaliar estratgias de busca hbridas que combinam seletivamente buscas em bibliotecas digitais com snowballing. Propusemos quatro estratgias de busca hbridas combinando buscas em bibliotecas digitais com snowballing iterativo, paralelo, sequencial backward e forward, ou sequencial forward e backward. Simulamos as estratgias ao longo de trs RSL existentes na Engenharia de Software que adotaram busca em bibliotecas digitais seguidas de snowballing. Ns analisamos sete estratgias de busca: quarto estratgias hbridas, busca em bibliotecas digitais, snowballing e busca em bibliotecas digitais seguidas de snowballing. Ns comparamos o resultado de busca em bibliotecas digitais, snowballing, busca em bibliotecas digitais seguidas de snowballing e as estratgias hbridas por meio de precision, recall e F-measure para investigar a eficincia de cada estratgia. Nossas investigaes permitiram observar que, para as RSL analisadas, a combinao de buscas na biblioteca digital Scopus com snowballing paralelo ou sequencial alcanou o equilbrio mais adequado entre precision e recall. Dependendo das metas da RSL e dos recursos disponveis, usar uma estratgia hbrida, envolvendo busca em uma biblioteca digital representativa, seguida de snowballing paralelo ou sequencial, pode ser uma alternativa apropriada para balancear qualidade e esforo em RSL. Palavras-chave: reviso sistemtica da literatura, busca em biblioteca digital, busca em banco de dados, snowballing, engenharia de software, estratgia de busca hbrida. Abstract: When performing a Systematic Literature Review (SLR), researchers usually face the challenge of designing a protocol that appropriately balances result quality and review effort. On the one hand, using digital library (or database) searches or snowballing alone would not be enough to achieve high-quality results. On the other hand, using both digital library searches and snowballing together may increase the overall review effort. The goal of this research is to propose and evaluate hybrid search strategies that selectively combine digital library searches with snowballing. We proposed four hybrid search strategies combining searches in digital libraries with iterative, parallel, sequential backward and forward, or sequential forward and backward snowballing. We simulated the strategies over three existing SLRs in Software Engineering that adopted both digital library searches and snowballing. We analyzed seven search strategies: four hybrid strategies, database search, snowballing, and database searches in several digital libraries followed by snowballing. We compared the outcome of digital library searches, snowballing, database search followed by snowballing, and the hybrid strategies combining both by means of precision, recall, and F-measure to investigate the efficiency of each strategy. Our investigations allowed observing that, for the analyzed SLRs, combining searches in the Scopus digital library with parallel or sequential snowballing achieved the most appropriate balance of precision and recall. We put forward that, depending on the goals of the SLR and the available resources, using a hybrid search strategy involving a representative digital library and parallel or sequential snowballing tends to represent an appropriate alternative to balance quality and effort when searching for evidence in SLRs. Keywords: systematic literature review, digital library search, database search, snowballing, software engineering, hybrid search strategy.      PAGE \* MERGEFORMAT 1 $ / |    % 5 < ? G F Q ~ 1 C D N KTW]mn~ԵhIn9CJaJhIn9hIn96CJ\aJhIn9hIn9CJ\aJhIn9hIn95CJaJhIn9hIn9CJaJ'hIn9hIn96CJaJmHnH sHtH $hIn9hIn9CJaJmHnH sHtH hIn95;CJaJhIn9hIn95;CJaJ1mn&[\dgdWKdgdIn9$d`a$gdIn9 d`gdIn9 d`gdIn9 Ed`gdIn9$dh`a$gdIn9$%&OPZ[\f  оогumimimimia]aTa]i]uhIn9mHnHuhk5<jhk5<UhRUjhRUUhIn9hk5<CJaJmH sH hIn9hIn9CJ\aJmH sH hIn9hIn95CJaJmH sH hIn9hIn9CJaJmH sH hIn9hIn9CJaJ"hIn9hIn9h~CJaJnH tH hIn9hIn9CJaJnH tH "hIn9hIn95;CJaJmH sH hIn95;CJaJmH sH  d`gdIn9gd~ ;$`a$gd$Y ;$`a$gdwgdy5$a$gd^4gdDLgdRQ##$a$<01hP:pj. A!"n#$n% *2V 666666666666666 66666666666666666666666666 666666666 666666666666 666666666666666666666666666666666666666666666666666666666666666666\@\ \3Normal$dh`a$ CJOJQJ_HaJmHsHtH b@b K'0Ttulo 1$$$h@&`a$5;CJPJ\aJtHX@X 0Ttulo 2$$x@&`5;PJ\tHX@X K'0Ttulo 3$$x@&`5;PJ\tHX@X K'0Ttulo 4$$x@&`5;PJ\tHX@X K'0Ttulo 5$$x@&`5;PJ\tHX@X K'0Ttulo 6$$x@&`5;PJ\tHX@X K'0Ttulo 7$$x@&`5;PJ\tHX@X K'0Ttulo 8$$x@&`5;PJ\tHX @X K'0Ttulo 9 $$x@&`5;PJ\tH>A@> 0Fonte parg. padroTi@T 0 Tabela normal4 l4a ,k, 0 Sem lista ROR K'0 Char Char17!5;CJOJPJQJ\^JaJROR 0 Char Char16!5;CJOJPJQJ\^JaJROR K'0 Char Char15!5;CJOJPJQJ\^JaJRO!R K'0 Char Char14!5;CJOJPJQJ\^JaJRO1R K'0 Char Char13!5;CJOJPJQJ\^JaJROAR K'0 Char Char12!5;CJOJPJQJ\^JaJROQR K'0 Char Char11!5;CJOJPJQJ\^JaJROaR K'0 Char Char10!5;CJOJPJQJ\^JaJPOqP  K'0 Char Char9!5;CJOJPJQJ\^JaJbOb Ds0Ttulosd7$8$H$#5B*CJ\aJmHphsHtH^O^ g1H0 TextoSimplesdh#5B*CJOJQJ\_HaJphTOT Ds0 Ttulos Char#5B*CJOJQJ\^JaJphROR M0 FiguraEtabelaCJOJQJaJmHsHtH^O^ g1H0TextoSimples Char#5B*CJOJQJ\_HaJph`O` M0FiguraEtabela Char#5B*CJOJQJ\^JaJph:O: jI0 Book Title 5:@\BB jI0Placeholder Text B*phV@V !jI0Texto de balo dCJOJQJ^JaJtHBB  jI0 Char Char8CJOJQJ^JaJDO"D %k0Figuras"$a$CJaJmHsHtH@@2@ $mB0 Cabealho# 8!d.OA. #mB0 Char Char7TOQT "k0 Figuras Char#5B*CJOJQJ\^JaJph: @b: 'WK0Rodap& 8!d.Oq. &WK0 Char Char6NON *g1H0Seo($dha$;CJaJmHsHtHFOF +Ds0 Sub-seo)CJaJmHsHtHROR (g1H0 Seo Char&5;B*CJOJQJ\^JaJphZOZ )Ds0Sub-seo Char&5;B*CJOJQJ\^JaJph8O!8 .h0Tabela,( mHsHtH h0Tabela com grade<:V-0a- CJ^JaJROR ,h0 Tabela Char#5B*CJOJQJ\^JaJph2Oa2 0a0Rodap1 /^.Or. /a0 Rodap CharfYf 2D0Estrutura do documento1dCJOJQJ^JaJtHB!B 1D0 Char Char5CJOJQJ^JaJD'1D _R0Ref. de comentrioCJaJTBT 5_R0Texto de comentrio4d CJaJtHBQB 4_R0 Char Char4CJOJQJ^JaJLjABL 7_R0Assunto do comentrio65\4Rq4 6_R0 Char Char35\@O@ p0List Paragraph 8^ p0Medium Grid 3 - Accent 41:V90ajQ@ jQ jQ dj; djQ djQ d 449tCJ^JaJ56B*\]ph56B*\]ph56B*\]ph56B*\]ph p0Medium Grid 3 - Accent 21:V:0ajQ@ ߧjQ ߧjQ PMj; PMjQ PMjQ PM 44:tCJ^JaJ56B*\]ph56B*\]ph56B*\]ph56B*\]ph@"@@ \30Legenda;$da$5\6O6 gJ0 TOC Heading<@& >@> 9b0 Analtico 1= e# dR@R L0 Analtico 2!> e# d^`X@X E Q0 Analtico 3'? ne# d^`6U@6 B0 Hyperlink >*B*phF#@F  0ndice de ilustraesAO" CO0Referncias Bibliogrficas(Bdx1$7$8$H$^` mHsHtHjO1j BO0Referncias Bibliogrficas CharCJOJQJ^JaJNON B0 BibliographyD |d`fORf 0 Els-body-textE$d`a$OJPJQJ_HmH sH tHrObr 0Els-Abstract-textF$d$a$$CJOJPJQJ_HaJmH sH tHH&@qH EC0Ref. de nota de rodapH*v@v IEC0Texto de nota de rodapH$d`a$CJPJaJmH sH tHFF HEC0 Char Char2OJQJ^JmH sH tH|OR| I0Els-2ndorder-headJ$ & Fd*$"6OJPJQJ]_HmH sH tHxORx I0Els-3rdorder-headK$ & Fd*$"6OJPJQJ]_HmH sH tH~OR~ I0Els-4thorder-headL$$ & Fd*$a$"6OJPJQJ]_HmH sH tH|OR| I0Els-1storder-headM$ & Fd*$"5OJPJQJ\_HmH sH tH^B@^ Ov T0Corpo de textoN d` @CJPJaJmH sH JOJ Nv T0 Char Char1OJPJQJ^JmH sH tHtOt 0S0Els-reference-headP$d"5OJPJQJ\_HmH sH tHO(w0 Els-reference Q 88d8^8`+CJOJPJQJ_HaJmHnHsHtHue" S]0Pr-formatao HTMLKR$ 2( Px 4 #\'*.25@9d`a$CJOJPJQJ^JaJtHD1D R]0 Char CharOJQJ^JmHsHtHLBLp0RevisionT CJOJQJ_HaJmHsHtH B*QB I[0Ref. de nota de fimH* $mn&[\0E000000E000@0I09@0I09@0I09@0I09@#0@#0@0I0_I0 9I02I0,2I0@I0^I0^I0^I0^I0_I0_K009 +++++.  $&.!_b$Zd(p5eYb$y;%ث|b$A%ճyKvCb$&E,-Zgeb$E*~!T2jEܐ@0(  B S  ?_GoBack  $/|*4?GFQ~ 1:<BEN  KTW]X \ SWY_%,&%ZZF Xb&9 dH ] %"xu)A89(BHu#܆-W6$0ƨE9,2lG6ިO9\g_-}=jCK:R(L$tzUTX VMJKL\_Y(8? iU<}m Zoȍ0 x+wu)o+.5pUh^h`o(3.8^8`.L^`L. ^ `. ^ `.xL^x`L.H^H`.^`.L^`L. ^`5\o(5.^`.pL^p`L.@ ^@ `.^`.L^`L.^`.^`.PL^P`L.h^h`o(2.8^8`.L^`L. ^ `. ^ `.xL^x`L.H^H`.^`.L^`L.^`o(.80^8`0o(..80^8`0o(...^`o(.... ^`o( ..... `^``o( ...... `^``o(....... p^p`o(........ p^p`o(.........h^h`o(3.5.^`.pL^p`L.@ ^@ `.^`.L^`L.^`.^`.PL^P`L. 2P^`P567;<CJH*OJQJS*\]^JaJo( Captulo &@^@`567;<CJH*S*\]aJo(1.&0^`0567;<CJH*S*\]aJo(4.2`^``567;<CJH*OJQJS*\]^JaJo(... 2^`567;<CJH*OJQJS*\]^JaJo( .... 2^`567;<CJH*OJQJS*\]^JaJo( ..... 2^`567;<CJH*OJQJS*\]^JaJo( ...... 2`^``567;<CJH*OJQJS*\]^JaJo(....... 20^0`567;<CJH*OJQJS*\]^JaJo(........ 2P^`P567;<CJH*OJQJS*\]^JaJo( Captulo &@^@`567;<CJH*S*\]aJo(3.&0^`0567;<CJH*S*\]aJo(4.2`^``567;<CJH*OJQJS*\]^JaJo(... 2^`567;<CJH*OJQJS*\]^JaJo( .... 2^`567;<CJH*OJQJS*\]^JaJo( ..... 2^`567;<CJH*OJQJS*\]^JaJo( ...... 2`^``567;<CJH*OJQJS*\]^JaJo(....... 20^0`567;<CJH*OJQJS*\]^JaJo(........X^X`OJQJ^Jo((^(`OJQJ^Jo(o^`OJ QJ ^J o( ^ `OJQJ^Jo( ^ `OJQJ^Jo(oh^h`OJ QJ ^J o(8^8`OJQJ^Jo(^`OJQJ^Jo(o^`OJ QJ ^J o(h^h`o(1.8^8`.L^`L. ^ `. ^ `.xL^x`L.H^H`.^`.L^`L.h^h`o(4.^`.pL^p`L.@ ^@ `.^`.L^`L.^`.^`.PL^P`L.h^h`o(2.2.^`.pL^p`L.@ ^@ `.^`.L^`L.^`.^`.PL^P`L.h^h`o(3.5.^`.pL^p`L.@ ^@ `.^`.L^`L.^`.^`.PL^P`L.h^h`o(3.^`.pL^p`L.@ ^@ `.^`.L^`L.^`.^`.PL^P`L....o(3.....  .....  ...... ....... ........ .........h^h`o(5.^`.pL^p`L.@ ^@ `.^`.L^`L.^`.^`.PL^P`L.^`OJQJ^Jo(e^e`OJQJ^Jo(o5 ^5 `OJ QJ ^J o(^`OJQJ^Jo(^`OJQJ^Jo(o^`OJ QJ ^J o(u^u`OJQJ^Jo(E^E`OJQJ^Jo(o^`OJ QJ ^J o(..........  .....  ...... ....... ........ .........h^h`o(-^-`o(.Z0^Z`0o(.. 0^ `0o(... L^L`o( .... ^`o( ..... >`^>``o( ...... `^``o(....... 0^0`o(........^`o(4.e^e`.5 L^5 `L.^`.^`.L^`L.u^u`.E^E`.L^`L.^`OJQJ^Jo(^`OJQJ^Jo(op^p`OJ QJ ^J o(@ ^@ `OJQJ^Jo(^`OJQJ^Jo(o^`OJ QJ ^J o(^`OJQJ^Jo(^`OJQJ^Jo(oP^P`OJ QJ ^J o(X^X`o(3.5.^`.pL^p`L.@ ^@ `.^`.L^`L.^`.^`.PL^P`L.-^-`o()^`. L^ `L. ^ `.m^m`.=L^=`L. ^ `.^`.L^`L. 2P^`P567;<CJH*OJQJS*\]^JaJo( Captulo &@^@`567;<CJH*S*\]aJo(3.&0^`0567;<CJH*S*\]aJo(4.2`^``567;<CJH*OJQJS*\]^JaJo(... 2^`567;<CJH*OJQJS*\]^JaJo( .... 2^`567;<CJH*OJQJS*\]^JaJo( ..... 2^`567;<CJH*OJQJS*\]^JaJo( ...... 2`^``567;<CJH*OJQJS*\]^JaJo(....... 20^0`567;<CJH*OJQJS*\]^JaJo(........' u)<}mHu#X V\_YX VX VX VX VX VX VX Vu)u)? iu)u)99 u)%u):9 u)`:9 u):9 u):9 )o+&9 9(-}=E9,tzUZoG6O92] x+wW6$CK:RX1"a4(4n4v@^*.,4(LЁ        9m9m9m)<Q */ (>A@BX~@iHMnw;/+CPE~K]'~TB+j^8 N gz R h 8 $  1 ! { v~ Q )= [Z i {!FP)^c<"gnv"3AGg6 (4!Ns*gJB]M8[0(2<6^4]u^h{)19") Sj^.M"lXz7 +rFF^utE3YI\5 I M !/@! E!BQ!5v!&#Q9#NC#|S#)$G$X$m$+I%h%F&Ls&''K'|P'u'b(0(4(1z(() ))3)Z)s),*M%*8*x +Q+g+>,?,-J>-=J-R- .R>hU>3f>y?@] @@NA\eA5HBoNBmBqEp+E%KEeE-{F G].GHGTGVGHg1HuHIxIo6I>IOIUIjIGqIrIrI:J,J4/JUJpJK.0KiLQL+L&,L .L#CLlL?M@:MTM2YMx NmO PH/PWP~PTQMQE QX%QUQTR_R0S}6SoSpS~Sv Tp%T1T=T?T UH*U;URUf}U V2 V*=VP W.W&?W!X;X{,?{@{#U{]j{r| }B}l}*~0N^Px|0APoa-ejCEi4))4JRQQp',3KM Oimcoq@x+O% 2:"=\T%lIw^4 :;[``wh@:BAn}R5Xew2l]c+}oLDPXxz|$(PPZd848:]]a0qaX']Yl$MLL!+xCIKT$Y+ 8>p!?ECy1K6[L*[:kky)P';Il{pDkTRzL^fp==#SrY-\~eDP(w=N v"8+< )"..#`[ikT\3y5C]wL)ab2@hhb6zP|: gC7D|B ,.>CDYw!vl@|+w_4z.hH_o e>tfBB9 '.]o#@4f? P#[z&'zy#ZXh DLRhvxE1IwT%`I'6&N1~-^_ 3//gP3T-ny5/a4RMp}5 9vC%OtS =X xWWix~ L-MZ( E8Z1^t&86U9=sb1? QXz2$4S,lt3,to/WK]5:r]ya)o",TOm&,t&7R7AOikl 'y}**Y3<4]sf3Q ilu.xt}^T AhK-MUZx3p@jb*wN3bw= %@ ~ `@Unknown G: Times New Roman5Symbol3& : Arial7&@CalibriW5   jMS Gothic l r SVbN;& :Cx Helvetica5& >[`)Tahoma?(SimSun?5 :Cx Courier New;Wingdings"D w'zvg|l |l !x42$PU92'Template de Dissertaes e Teses da UFFEricaHelioh                  Oh+'0 $0 P \ h t(Template de Dissertaes e Teses da UFFEricaNormalHelio4Microsoft Office Word@0@B~,&@V&0@x|l ՜.+,D՜.+,T hp|   (Template de Dissertaes e Teses da UFF Ttulo(X`hZOTERO_PREF_1ZOTERO_PREF_2