azalea/fixminer_source
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

release cand. 2

Browse Source
This commit is contained in:
fixminer
2018-09-17 18:29:33 +02:00
parent 4a9d87974e
commit 2d12536b93
116 changed files with 650 additions and 118668 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Dataset/LiveStudy/Patches
-119
View File
@@ -1,119 +0,0 @@
BC_UNCONFIRMED_CAST
BC_UNCONFIRMED_CAST_OF_RETURN_VALUE Comments: no any useful common fix patterns.
EI_EXPOSE_REP
EI_EXPOSE_REP2
IS2_INCONSISTENT_SYNC
NM_METHOD_NAMING_CONVENTION
DE_MIGHT_IGNORE Compile failed.
DLS_DEAD_LOCAL_STORE Compile failed
NP_LOAD_OF_KNOWN_NULL_VALUE
DM_CONVERT_CASE To be improved
DM_BOXED_PRIMITIVE_FOR_PARSING
DM_DEFAULT_ENCODING
DM_FP_NUMBER_CTOR
DM_NUMBER_CTOR
ES_COMPARING_STRINGS_WITH_EQ Comments: ignored
MS_PKGPROTECT
MS_SHOULD_BE_FINAL Comments: failed to checkstyle.
NP_NONNULL_RETURN_VIOLATION
NP_NULL_ON_SOME_PATH
NP_NULL_ON_SOME_PATH_FROM_RETURN_VALUE
NP_PARAMETER_MUST_BE_NONNULL_BUT_MARKED_AS_NULLABLE
OBL_UNSATISFIED_OBLIGATION
OBL_UNSATISFIED_OBLIGATION_EXCEPTION_EDGE
ODR_OPEN_DATABASE_RESOURCE
OS_OPEN_STREAM
OS_OPEN_STREAM_EXCEPTION_PATH
PZLA_PREFER_ZERO_LENGTH_ARRAYS
RCN_REDUNDANT_NULLCHECK_OF_NONNULL_VALUE
RCN_REDUNDANT_NULLCHECK_WOULD_HAVE_BEEN_A_NPE
REC_CATCH_EXCEPTION
RI_REDUNDANT_INTERFACES
RV_RETURN_VALUE_IGNORED_BAD_PRACTICE
RV_RETURN_VALUE_IGNORED_NO_SIDE_EFFECT
SBSC_USE_STRINGBUFFER_CONCATENATION
SE_NO_SERIALVERSIONID
SF_SWITCH_NO_DEFAULT
SIC_INNER_SHOULD_BE_STATIC
SIC_INNER_SHOULD_BE_STATIC_ANON
SQL_PREPARED_STATEMENT_GENERATED_FROM_NONCONSTANT_STRING
ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD
UC_USELESS_CONDITION
UC_USELESS_OBJECT
UCF_USELESS_CONTROL_FLOW
UPM_UNCALLED_PRIVATE_METHOD
URF_UNREAD_FIELD
URF_UNREAD_PUBLIC_OR_PROTECTED_FIELD
UWF_FIELD_NOT_INITIALIZED_IN_CONSTRUCTOR
UWF_UNWRITTEN_FIELD
VA_FORMAT_STRING_USES_NEWLINE
WMI_WRONG_MAP_ITERATOR
rejected 1 https://github.com/apache/commons-io/pull/47
rejected 1 https://github.com/apache/commons-io/pull/45
rejected 1 https://github.com/apache/commons-lang/pull/298
5 https://github.com/apache/commons-lang/pull/297
improved 1 https://github.com/apache/commons-lang/pull/297 Comments: some patches failed to pass test cases, some patches led to compile failed, some patches led to checkstyle failed.
2 https://github.com/apache/commons-math/pull/69
1 https://github.com/apache/commons-math/pull/68 Comments: inner static class, there is no need to fix the violations in this kind of class.
2 https://github.com/apache/commons-math/pull/67 Comments: improve patches.
1 https://github.com/apache/commons-math/pull/66
rejected 1 https://github.com/apache/ant/pull/47
improved 1 https://github.com/apache/ant/pull/46
merged 2 https://github.com/apache/ant/pull/46
2 https://github.com/apache/ant/pull/45
rejected 3 https://github.com/apache/ant/pull/44
merged 1 https://github.com/apache/ant/pull/43
merged 5 https://github.com/apache/ant/pull/42 Comments: need to import new packages, unless the fixed patches will fail to compile.
merged 1 https://github.com/apache/ant/pull/41
1 https://github.com/apache/aries/pull/80
1 https://github.com/apache/aries/pull/79
1 https://github.com/apache/aries/pull/78
1 https://github.com/apache/aries/pull/77
1 https://github.com/apache/aries/pull/76
rejected 1 https://github.com/apache/camel/pull/2031 ShouldBeFinal
rejected 6 https://github.com/apache/camel/pull/2029 master5
merged 1 https://github.com/apache/camel/pull/2035 master7
merged 2 https://github.com/apache/camel/pull/2032 Useless
merged 2 https://github.com/apache/camel/pull/2025 master6
merged 1 https://github.com/apache/camel/pull/2024 master4
merged 1 https://github.com/apache/camel/pull/2023 master2
rejected 1 https://github.com/apache/camel/pull/2017 master3 Comments: compile failed, the return value of method invocation is not Boolean, it is a specific case.
merged 1 https://github.com/apache/camel/pull/2016 master
merged 6 https://github.com/apache/camel/pull/2038 boxedParsing
3 https://github.com/apache/cassandra/pull/165
4 https://github.com/apache/cassandra/pull/164
2 https://github.com/apache/cassandra/pull/163
Fixed 1 https://github.com/fangyidong/json-simple/pull/126 Comments: fixed in another version. https://github.com/RalleYTN/SimpleJSON
1 https://github.com/fangyidong/json-simple/pull/125
On hold 1 https://github.com/apache/mahout/pull/346
1 https://github.com/apache/mahout/pull/345
1 https://github.com/apache/mahout/pull/344
merged 2 https://github.com/apache/poi/pull/77 5
merged 4 https://github.com/apache/poi/pull/78 6
merged 2 https://github.com/apache/poi/pull/79 t
merged 1 https://github.com/apache/poi/pull/80 4
merged 4 https://github.com/apache/poi/pull/76 2
merged 30 https://github.com/apache/poi/pull/75 master
merged 1 https://github.com/apache/poi/pull/74 3
Some violations never happened in these projects:
Dataset/Patterns/Fix-Patterns
-131
View File
@@ -1,131 +0,0 @@
=====================================
=========Useful Fix Patterns=========
=====================================
==============Detailed===============
DM_CONVERT_CASE
(1) ADD a rule of Locale.ENGLISH into toLowerCase/toUpperCase.
RCN_REDUNDANT_NULLCHECK_OF_NONNULL_VALUE
(1) DEL Null-Check IfStatement,
(2) DEL Null-Check Conditional Expression.
REC_CATCH_EXCEPTION
(1) Change Exception to a specific exception class.
SE_NO_SERIALVERSIONID
(1) Insert a private static final field serialVersionUID.
VA_FORMAT_STRING_USES_NEWLINE
(1) DEL "$backslash$n",
(2) Replace "$backslash$n" with "n"
MS_SHOULD_BE_FINAL
(1) ADD a ``final'' modifier.
WMI_WRONG_MAP_ITERATOR
(1) Replace the KeySet iterating ForStatement with an EntrySet iterating ForStatement.
DM_NUMBER_CTOR
(1) Replace the NumberType ClassInstanceCreation with a NumberType.valueOf() method invocation.
SIC_INNER_SHOULD_BE_STATIC
(1) ADD a "static" modifier,
(2) DEL the buggy statements.
DE_MIGHT_IGNORE
(1) Change Exception to a specific exception class.
DM_BOXED_PRIMITIVE_FOR_PARSING
(1) Replace NumberType.valueOf() with NumberType.parseNumberType().
DM_FP_NUMBER_CTOR
(1) DEL the buggy statement,
(2) Replace Double or Float ClassInstanceCreation with (Double or Float).valueOf(),
(3) Remove the Double or Float ClassInstanceCreation from the buggy expression.
MS_PKGPROTECT
(1) DEL the buggy statement,
(2) DEL the ``public'' or ``protected'' modifier
(3) Replace the ``public'' or ``protected'' modifier with the ``private'' modifier.
(4) DEL the initialization expression.
ES_COMPARING_STRINGS_WITH_EQ
(1) Replace the ``=='' InfixExpression with a stringVar.equals() method invocation.
===============Abstract==============
NP_NULL_ON_SOME_PATH
(1) ADD a Null-check statement or expression.
RI_REDUNDANT_INTERFACES
(1) DEL the implemented Interface type.
NP_NULL_ON_SOME_PATH_FROM_RETURN_VALUE
(1) ADD a Null-check statement or expression,
(2) DEL the buggy statement.
UCF_USELESS_CONTROL_FLOW
(1) DEL the buggy statement,
(2) DEL the useless ConditionalExpression.
UC_USELESS_CONDITION
(1) UPD the operator of a ConditionalExpersion,
(2) UPD an sub-expression of a ConditionExpression.
NP_LOAD_OF_KNOWN_NULL_VALUE
(1) DEL Null-check expression or statement,
(2) Replace the null-known variable with NULL.
SBSC_USE_STRINGBUFFER_CONCATENATION
(1) Replace String type with StringBuilder, and replace plus operator of StringVarialbe with the append method of StringBuilder.
UC_USELESS_OBJECT
(1) DEL the buggy statement.
UPM_UNCALLED_PRIVATE_METHOD
(1) DEL the buggy statements.
NM_METHOD_NAMING_CONVENTION
(1) Update Method Name of MethodDeclaration.
URF_UNREAD_FIELD
(1) DEL the buggy statement.
=====================================
NP_NONNULL_RETURN_VIOLATION
(1) DEL the null-returned ReturnStatement.
(2) Replace NullLiteral with an Expression which returns an empty list.
PZLA_PREFER_ZERO_LENGTH_ARRAYS
(1) DEL the buggy statement,
√ (2) Replace NullLiteral with an empty array.
SIC_INNER_SHOULD_BE_STATIC_ANON
(1) DEL a parameter of a ClassInstanceCreation in an AnonymousClassDeclaration,
(2) DEL the statement contains an AnonymousClassDeclaration,
√ (3) Replace the AnonymousClassDeclaration with an defined Static Object of this class.
DLS_DEAD_LOCAL_STORE
(1) DEL the buggy statement.
DM_DEFAULT_ENCODING
√ (1) ADD a parameter (i.e., "UTF-8") into the ClassInstanceCreation of a Stream object.
(2) Replace ClassInstanceCreation with a MethodInvocation.
(3) DEL the buggy statement.
BC_UNCONFIRMED_CAST
(1) DEL the buggy statement,
(2) DEL the Cast Type,
(3) Replace CastExpression with NullLiteral.
URF_UNREAD_PUBLIC_OR_PROTECTED_FIELD
(1) ADD a ``final'' modifier,
√ (2) DEL the buggy statement.
RV_RETURN_VALUE_IGNORED_BAD_PRACTICE
(1) Replace boolean-returned statement with a a new MethodInvocation.
√ (2) ADD an IfStatement.
EI_EXPOSE_REP
(1) DEL the buggy statement,
* (2) Replace the being assigned Expression with a ConditionalExpression.
√ (3) Replace the Expression being assigned with other Expression
EI_EXPOSE_REP2
(1) DEL the buggy statement,
* (2) Replace the being assigned Expression with a ConditionalExpression.
√ (3) Replace the being assigned Expression with other Expression.
=====================================
==========Null Fix Patterns==========
=====================================
UWF_FIELD_NOT_INITIALIZED_IN_CONSTRUCTOR null.
OBL_UNSATISFIED_OBLIGATION null.
OS_OPEN_STREAM_EXCEPTION_PATH null.
SF_SWITCH_NO_DEFAULT null.
UWF_UNWRITTEN_FIELD null.
IS2_INCONSISTENT_SYNC null.
ODR_OPEN_DATABASE_RESOURCE null.
SQL_PREPARED_STATEMENT_GENERATED_FROM_NONCONSTANT_STRING null.
OBL_UNSATISFIED_OBLIGATION_EXCEPTION_EDGE null.
OS_OPEN_STREAM null.
NP_PARAMETER_MUST_BE_NONNULL_BUT_MARKED_AS_NULLABLE null.
=====================================
========Useless Fix Patterns=========
=====================================
BC_UNCONFIRMED_CAST_OF_RETURN_VALUE
(1) UPD T1 v1 = (T1) method_invocation() with T2 v1 = method_invocation(),
(2) DEL the buggy statement,
(3) Replace the buggy CastExpresion with a MethodInvocation.
ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD
(1) DEL the buggy statement.
RCN_REDUNDANT_NULLCHECK_WOULD_HAVE_BEEN_A_NPE
(1) ADD a Null-check statement or expression.
Dataset/Patterns/Source-Code-Patterns-Unfixed-Violations
-107
View File
@@ -1,107 +0,0 @@
================================
=========Single Pattern=========
================================
SIC_INNER_SHOULD_BE_STATIC_ANON: new Type() { override some methods(...){...} }. Anonymous inner class declaration. 100%
DE_MIGHT_IGNORE: try{...} catch (Exception e) {...}, Exception is not any specified.
VA_FORMAT_STRING_USES_NEWLINE: "\n", this new line symbol is in a StringLiteral of a format method.
UCF_USELESS_CONTROL_FLOW: if (conditional expression) {empty statements.}.
UC_USELESS_CONDITION: if (conditional expressions) {...}.
UC_USELESS_OBJECT: T1 v1 = new T1().
PZLA_PREFER_ZERO_LENGTH_ARRAYS: return null.
EI_EXPOSE_REP: return variable;
REC_CATCH_EXCEPTION: try{...} catch (Exception e) {...}, Exception is not any specific exception.
SF_SWITCH_NO_DEFAULT: switch(exp) {case1:...case2:...}, without default statement.
UWF_UNWRITTEN_FIELD: return exp / variable.
SBSC_USE_STRINGBUFFER_CONCATENATION: stringVariable += stringExp,
RV_RETURN_VALUE_IGNORED_NO_SIDE_EFFECT: return method_invocation_exp. Method invocation statement returns a value, but does not assign the value to any variable. (High false positives.)
NP_NONNULL_RETURN_VIOLATION: return null. (High false positives.)
================================
========Multiple Patterns=======
================================
NP_NULL_ON_SOME_PATH_FROM_RETURN_VALUE: (1) for (File file : filesArray) {...} (2) for (int i =0; i < filesArray.length; i ++) {...}
DM_CONVERT_CASE: (1) stringExp.toLowerCase(), 24,897
(2)stringExp.toUpperCase(). 10,406
MS_PKGPROTECT: (1) public static final T1 v1 = initializationExp, 2442
(2) protected static final T1 v1 = initializationExp. 175
(3) protected static T1 v1 = initializationExp. 158
(4) public static T1 v1 = initializationExp. 262, other 46
SIC_INNER_SHOULD_BE_STATIC: (1) this.v1 = v2; 590
(2) v1 = v2; 170
(3) super(...). 213
BC_UNCONFIRMED_CAST: (1) T1 v1 = (T(1) v2/exp, (2) v1 = (T(1) v2/exp, (3) ((T(1) v2).exp.
ES_COMPARING_STRINGS_WITH_EQ: (1) stringExp1 == stringExp2 IfStatement, ReturnStatement, WhileStatement.
(2) stringExp1 != stringExp2
RCN_REDUNDANT_NULLCHECK_OF_NONNULL_VALUE: (1) if (exp == null ...) {...},
(2) if (exp != null ...) {...},
(3) exp == null ? exp1 : exp2,
(4) exp != null ? exp1 : exp2.
DM_NUMBER_CTOR: (1) new Long(...), (2)new Integer(...), (3)new Short(...), (4)new Byte(...), (5)new Char(...).
DM_FP_NUMBER_CTOR: (1) new Double(exp), (2)new Float(exp).
DM_BOXED_PRIMITIVE_FOR_PARSING: (1) Integer.valueOf(str), (2)Long.valueOf(str). Comments: it contains fewer patterns than findbugs.
EI_EXPOSE_REP2: (1) this.v1 = v2,
(2) v1 = v2. 3700
BC_UNCONFIRMED_CAST_OF_RETURN_VALUE: (1) T1 v1 = (T) method_invocation(),
(2) v1 = (T) method_invocation(),
(3) ((T) method_invocation()).exp.
ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD: (1) static_v1 = instance_exp1.method_invocation(),
(2) v1 = instance_exp1.method_invocation(),
(3) v1 = exp1, other variable assignment statement.
RV_RETURN_VALUE_IGNORED_BAD_PRACTICE: (1) File.mkdirs(),
(2) File.mkdir(),
(3) File.delete(),
(4) File.createNewFile(),
(5) other exp.method_invoation() returns a value.
SQL_PREPARED_STATEMENT_GENERATED_FROM_NONCONSTANT_STRING: (1) sqlExecutionStatement = StringLiteral1 + stringV1 + ... + StringLiteral2 + stringV2.
(2) sqlRelatedVariable = sqlRelatedMethodInvocation();
OS_OPEN_STREAM: (1) StreamType v = new StreamType(...), 2) streamVariable = new StreamType(new StreamType2(...)), e.g., 1. FileOutputStream fos = new FileOutputStream(outFile);
DM_DEFAULT_ENCODING: (1) new BufferedReader(new InputStreamReader(input)), 99+
(2) new StreamWriter(output);
(3) stringExp.getBytes(),
(4) new String(byteExp),
(5) byteExp.toString().
ODR_OPEN_DATABASE_RESOURCE: (1) conn.createStatement(),
(2) conn.createStatement.execute(...),
(3) PreparedStatement st = conn.prepareStatement(),
(4) DataBaseConnection conn = exp.getConnection(...).
NP_PARAMETER_MUST_BE_NONNULL_BUT_MARKED_AS_NULLABLE: (1) return method_invocation(...), 2) return class_instance_creation(...).
========================================
==========Inefficient Patterns==========
========================================
OBL_UNSATISFIED_OBLIGATION: (1) StreamType v = new StreamType(...),
(2) streamVariable = new StreamType(new StreamType2(...)),
(3) sqlStatementExp.executeMethod(). e.g., 1. FileOutputStream fos = new FileOutputStream(outFile); 2. ResultSet rs = sqlStmt.executeQuery().
OS_OPEN_STREAM_EXCEPTION_PATH: (1) StreamType v = new StreamType(...),
(2) streamVariable = new StreamType(...).
IS2_INCONSISTENT_SYNC: (1) return v1,
(2) if(conditionalExp) {...},
(3) sqlStmt = stringLiteral1 + stringExp + stringLiteral2. not very consistent.
OBL_UNSATISFIED_OBLIGATION_EXCEPTION_EDGE: (1) StreamType v = new StreamType(...),
(2) streamVariable = new StreamType(new StreamType2(...)),
(3) sqlStatementExp.executeMethod(). e.g., 1. FileOutputStream fos = new FileOutputStream(outFile); 2. ResultSet rs = sqlStmt.executeQuery().
WMI_WRONG_MAP_ITERATOR: for statement { ... MapVariable.get(keyExp) ... }. or NULL. Lack of enough information.
========================================
=====Redundant Inconsistent Pattern=====
========================================
MS_SHOULD_BE_FINAL: (1) public (28,302) static T1 v1 = exp. e.g., public static int maxStringLength = 65535.
(2) protected (6,719) static T1 v1 = exp.
URF_UNREAD_FIELD: (1) this.v1 = v2, (2)this.v1 = new T(), (3)private Type v = exp. Comments: it contains more patterns than findbugs. only pattern 3 is consistent.
URF_UNREAD_PUBLIC_OR_PROTECTED_FIELD: (1) this.v1 = v2,
(2) public/protected (static) T1 v1 = exp; Comments: only 2 is consistent.
(3) v1 = exp1.
========================================
=========Inconsistent Patterns=========
========================================
RI_REDUNDANT_INTERFACES: super(...). Comments: not very consistent. Locate on the position of super classes.
========================================
==============Null Pattern==============
========================================
===========Inaccurate Position==========
NM_METHOD_NAMING_CONVENTION: null. Comments: violate on title, but position locates on body.
SE_NO_SERIALVERSIONID: null.
UPM_UNCALLED_PRIVATE_METHOD: null. (method body)
UWF_FIELD_NOT_INITIALIZED_IN_CONSTRUCTOR: null. inaccurate positions.
==========Specific Source Code==========
DLS_DEAD_LOCAL_STORE: null. Variable assignment statements. Comments: specific source code.
NP_NULL_ON_SOME_PATH: null.
NP_LOAD_OF_KNOWN_NULL_VALUE: null.
RCN_REDUNDANT_NULLCHECK_WOULD_HAVE_BEEN_A_NPE: null.
FixPatternMiner.iml
+10 -18
View File
@@ -11,15 +11,6 @@
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
<orderEntry type="library" name="Maven: org.javatuples:javatuples:1.2" level="project" />
<orderEntry type="module" module-name="simple-utils" />
<orderEntry type="library" name="Maven: org.apache.poi:poi:3.12" level="project" />
<orderEntry type="library" name="Maven: commons-codec:commons-codec:1.9" level="project" />
<orderEntry type="library" name="Maven: org.apache.poi:poi-ooxml:3.12" level="project" />
<orderEntry type="library" name="Maven: org.apache.poi:poi-ooxml-schemas:3.12" level="project" />
<orderEntry type="library" name="Maven: org.apache.xmlbeans:xmlbeans:2.6.0" level="project" />
<orderEntry type="library" name="Maven: stax:stax-api:1.0.1" level="project" />
<orderEntry type="library" name="Maven: net.sourceforge.jexcelapi:jxl:2.6.12" level="project" />
<orderEntry type="library" name="Maven: log4j:log4j:1.2.14" level="project" />
<orderEntry type="library" name="Maven: com.typesafe.akka:akka-actor_2.11:2.4.11" level="project" />
<orderEntry type="library" name="Maven: org.scala-lang:scala-library:2.11.8" level="project" />
<orderEntry type="library" name="Maven: com.typesafe:config:1.3.0" level="project" />
@@ -27,9 +18,18 @@
<orderEntry type="module" module-name="core" />
<orderEntry type="library" name="Maven: com.github.mpkorstanje:simmetrics-core:3.0.3" level="project" />
<orderEntry type="library" name="Maven: com.google.guava:guava:18.0" level="project" />
<orderEntry type="library" name="Maven: commons-codec:commons-codec:1.10" level="project" />
<orderEntry type="library" name="Maven: net.sf.trove4j:trove4j:3.0.3" level="project" />
<orderEntry type="library" name="Maven: com.google.code.gson:gson:2.3" level="project" />
<orderEntry type="module" module-name="gen.jdt" />
<orderEntry type="library" name="Maven: edu.lu.uni:simple-utils:0.0.1-SNAPSHOT" level="project" />
<orderEntry type="library" name="Maven: org.apache.poi:poi:3.12" level="project" />
<orderEntry type="library" name="Maven: org.apache.poi:poi-ooxml:3.12" level="project" />
<orderEntry type="library" name="Maven: org.apache.poi:poi-ooxml-schemas:3.12" level="project" />
<orderEntry type="library" name="Maven: org.apache.xmlbeans:xmlbeans:2.6.0" level="project" />
<orderEntry type="library" name="Maven: stax:stax-api:1.0.1" level="project" />
<orderEntry type="library" name="Maven: net.sourceforge.jexcelapi:jxl:2.6.12" level="project" />
<orderEntry type="library" name="Maven: log4j:log4j:1.2.14" level="project" />
<orderEntry type="library" name="Maven: org.eclipse.core:runtime:3.10.0-v20140318-2214" level="project" />
<orderEntry type="library" name="Maven: org.eclipse:osgi:3.10.0-v20140606-1445" level="project" />
<orderEntry type="library" name="Maven: org.eclipse.equinox:common:3.6.200-v20130402-1505" level="project" />
@@ -43,20 +43,12 @@
<orderEntry type="library" name="Maven: org.slf4j:slf4j-api:1.7.7" level="project" />
<orderEntry type="library" name="Maven: ch.qos.logback:logback-classic:1.1.2" level="project" />
<orderEntry type="library" name="Maven: ch.qos.logback:logback-core:1.1.2" level="project" />
<orderEntry type="module" module-name="GitTraveller" />
<orderEntry type="library" name="Maven: org.apache.commons:commons-lang3:3.7" level="project" />
<orderEntry type="library" name="Maven: org.jsoup:jsoup:1.11.2" level="project" />
<orderEntry type="library" name="Maven: org.eclipse.jgit:org.eclipse.jgit:4.7.0.201704051617-r" level="project" />
<orderEntry type="library" name="Maven: com.jcraft:jsch:0.1.54" level="project" />
<orderEntry type="library" name="Maven: com.googlecode.javaewah:JavaEWAH:1.1.6" level="project" />
<orderEntry type="library" name="Maven: org.apache.httpcomponents:httpclient:4.3.6" level="project" />
<orderEntry type="library" name="Maven: org.apache.httpcomponents:httpcore:4.3.3" level="project" />
<orderEntry type="library" name="Maven: commons-logging:commons-logging:1.1.3" level="project" />
<orderEntry type="library" name="Maven: junit:junit:4.11" level="project" />
<orderEntry type="library" name="Maven: org.hamcrest:hamcrest-core:1.3" level="project" />
<orderEntry type="library" name="Maven: redis.clients:jedis:2.8.1" level="project" />
<orderEntry type="library" name="Maven: org.apache.commons:commons-pool2:2.4.2" level="project" />
<orderEntry type="library" name="Maven: org.apache.commons:commons-text:1.3" level="project" />
<orderEntry type="library" name="Maven: org.apache.commons:commons-lang3:3.7" level="project" />
<orderEntry type="library" name="Maven: com.rabbitmq:amqp-client:4.0.0" level="project" />
</component>
</module>
inputData/1.txt
-730
View File
@@ -1,730 +0,0 @@
https://github.com/IDPF/epubcheck.git
https://github.com/treasure-data/td-import-java.git
https://github.com/Atmosphere/atmosphere-extensions.git
https://github.com/astefanutti/camel-cdi.git
https://github.com/imagej/imagej-ui-swing.git
https://github.com/komoot/photon.git
https://github.com/FIWARE-Middleware/KIARA.git
https://github.com/mbosecke/pebble.git
https://github.com/metamx/java-util.git
https://github.com/contextproject/discover.git
https://github.com/btrplace/scheduler.git
https://github.com/tuwiendsg/rSYBL.git
https://github.com/sker65/go-dmd-clock.git
https://github.com/liferay/liferay-maven-support.git
https://github.com/0xCopy/RelaxFactory.git
https://github.com/caskdata/hydrator-plugins.git
https://github.com/joelittlejohn/jsonschema2pojo.git
https://github.com/ArcBees/gwtquery.git
https://github.com/redpen-cc/redpen.git
https://github.com/sviperll/chicory.git
https://github.com/spotify/cassandra-reaper.git
https://github.com/sakai-mirror/calendar.git
https://github.com/notthebees/languagegames.git
https://github.com/imagej/ij1-patcher.git
https://github.com/cjdev/versions-maven-plugin.git
https://github.com/phax/as2-lib.git
https://github.com/RolecraftDev/RolecraftCore.git
https://github.com/dbs-leipzig/gradoop.git
https://github.com/dbsoftcombr/dbssdk.git
https://github.com/shabbies/clockwork.git
https://github.com/skillcoyne/IGCSA.git
https://github.com/GlobalTechnology/idm-user-management.git
https://github.com/instance01/MinigamesAPI.git
https://github.com/jaywarrick/JEX.git
https://github.com/mwcaisse/AndroidFT.git
https://github.com/figarocms/cucumber-jvm-fixtures.git
https://github.com/jillesvangurp/jsonj.git
https://github.com/SonarSource/sslr.git
https://github.com/OpenTreeOfLife/taxomachine.git
https://github.com/The-Dream-Team/Tardis.git
https://github.com/digipost/digipost-api-client-java.git
https://github.com/nuxeo/nuxeo-platform-video.git
https://github.com/INL/BlackLab.git
https://github.com/andrewgaul/s3proxy.git
https://github.com/OpenRock/OpenICF-java-framework.git
https://github.com/blurpy/kouchat-android.git
https://github.com/hypercube1024/firefly.git
https://github.com/Poweruser/MinetickMod.git
https://github.com/exoplatform/wiki.git
https://github.com/Bukkit/Bukkit.git
https://github.com/zanata/zanata-api.git
https://github.com/stripe/stripe-java.git
https://github.com/allanbank/mongodb-async-driver.git
https://github.com/AMOSTeam3/amos-ss15-proj3.git
https://github.com/groovy/GMavenPlus.git
https://github.com/SUSE/saltstack-netapi-client-java.git
https://github.com/nuxeo/nuxeo-chemistry.git
https://github.com/wattdepot/wattdepot.git
https://github.com/nuxeo/nuxeo-platform-forms-layout-demo.git
https://github.com/deeplearning4j/nd4j.git
https://github.com/NFSdb/nfsdb.git
https://github.com/cismet/cids-server.git
https://github.com/sarahtattersall/PIPE.git
https://github.com/apache/maven-shared.git
https://github.com/daisy/pipeline-mod-braille.git
https://github.com/jenkinsci/xunit-plugin.git
https://github.com/jclouds/jclouds-karaf.git
https://github.com/DICE-UNC/jargon.git
https://github.com/ancoron/glassfish-main.git
https://github.com/FallenMoonNetwork/CanaryLib.git
https://github.com/phillipsic/SeleniumDriverFramework.git
https://github.com/dekellum/iudex.git
https://github.com/apache/openjpa.git
https://github.com/Praqma/cool.git
https://github.com/SciGraph/SciGraph.git
https://github.com/TeamShadow/shadow.git
https://github.com/cowtowncoder/ClusterMate.git
https://github.com/shubhcollaborator/common-csvnew.git
https://github.com/albfan/jmeld.git
https://github.com/strongbox/strongbox.git
https://github.com/dropwizard/metrics.git
https://github.com/Studentmediene/Momus.git
https://github.com/apache/continuum.git
https://github.com/jenkinsci/junit-plugin.git
https://github.com/pcpratts/rootbeer1.git
https://github.com/nieshr/xjr.git
https://github.com/LiveRamp/hank.git
https://github.com/ralscha/extdirectspring.git
https://github.com/sjanaud/jensoft-core.git
https://github.com/marschraner/svm.git
https://github.com/LEDS/sincap-entities.git
https://github.com/AugGroup/hr-db.git
https://github.com/MineSworn/UltimateArena.git
https://github.com/benas/easy-batch.git
https://github.com/ocpsoft/rewrite.git
https://github.com/square/dagger.git
https://github.com/pgjdbc/pgjdbc.git
https://github.com/membrane/service-proxy.git
https://github.com/getsentry/raven-java.git
https://github.com/jboss-javassist/javassist.git
https://github.com/mybatis/mybatis-3.git
https://github.com/opendaylight/groupbasedpolicy.git
https://github.com/Bammerbom/UltimateCore.git
https://github.com/mrniko/netty-socketio.git
https://github.com/searchbox-io/Jest.git
https://github.com/jo-pol/DiBL.git
https://github.com/sukirtigupta/slf4j.git
https://github.com/immutables/immutables.git
https://github.com/dumptruckman/PluginBase.git
https://github.com/TridentSDK/TridentSDK.git
https://github.com/johan-martenson/settlers.git
https://github.com/lightblue-platform/lightblue-core.git
https://github.com/cismet/cismet-gui-commons.git
https://github.com/dakusui/jcunit.git
https://github.com/Pi4J/pi4j.git
https://github.com/lutece-secteur-public/espacepublic-plugin-dansmarue.git
https://github.com/lexicalscope/svm.git
https://github.com/ucam-cl-dtg/urop-2013-questions.git
https://github.com/grzegorz2047/OpenGuild2047.git
https://github.com/davidsoergel/ml.git
https://github.com/apache/sirona.git
https://github.com/Stratio/cassandra-lucene-index.git
https://github.com/jdr0887/MaPSeq-Pipeline-NCGenes-DX.git
https://github.com/Azure/azure-sdk-for-java.git
https://github.com/sakai-mirror/msgcntr.git
https://github.com/openmrs/openmrs-module-emrapi.git
https://github.com/openmrs/openmrs-module-mdrtb.git
https://github.com/iZettle/izettle-toolbox.git
https://github.com/jmxtrans/jmxtrans.git
https://github.com/effektif/effektif.git
https://github.com/DataTorrent/Apex.git
https://github.com/aeshell/aesh.git
https://github.com/SophieKoonin/JavaDecaf.git
https://github.com/jitsi/jitsi-videobridge.git
https://github.com/renepickhardt/generalized-language-modeling-toolkit.git
https://github.com/TeamExodus/external_gson.git
https://github.com/minnal/minnal.git
https://github.com/soi-toolkit/soi-toolkit-mule.git
https://github.com/davidmoten/rtree.git
https://github.com/zendesk/maxwell.git
https://github.com/apache/commons-compress.git
https://github.com/hector-client/hector.git
https://github.com/tubav/fiteagle.git
https://github.com/AtlasOfLivingAustralia/bie-service.git
https://github.com/sakai-mirror/basiclti.git
https://github.com/nuodb/migration-tools.git
https://github.com/jahlborn/jackcess.git
https://github.com/SebastianNiemann/ArmadilloJava.git
https://github.com/RuedigerMoeller/kontraktor.git
https://github.com/carewebframework/carewebframework-core.git
https://github.com/BattleNight/BattleNight-Core.git
https://github.com/apache/directory-studio.git
https://github.com/apache/creadur-rat.git
https://github.com/ARUP-NGS/Pipeline.git
https://github.com/zenbones/SmallMind.git
https://github.com/jayway/rest-assured.git
https://github.com/datanucleus/datanucleus-rdbms.git
https://github.com/TridentSDK/Trident.git
https://github.com/wildfly-security/wildfly-elytron.git
https://github.com/mathieu-bellange/ourses-plumes.git
https://github.com/apache/vxquery.git
https://github.com/openmrs/openmrs-distro-referenceapplication.git
https://github.com/apache/httpclient.git
https://github.com/costamojan/xbean.git
https://github.com/masatomix/repo.git
https://github.com/lviggiano/owner.git
https://github.com/apache/juddi.git
https://github.com/seulkikims/hashtable.git
https://github.com/davidkey/supertunnel.git
https://github.com/irstv/H2GIS.git
https://github.com/mindwind/craft-atom.git
https://github.com/relayrides/pushy.git
https://github.com/jdereg/json-io.git
https://github.com/Welchd1/resolve-lite.git
https://github.com/jmacglashan/burlap.git
https://github.com/fluxroot/hadaps.git
https://github.com/matthias-mueller/movingcode.git
https://github.com/aceleradora6-tw/RegistroLivre.git
https://github.com/st-js/st-js.git
https://github.com/OpenHFT/HugeCollections.git
https://github.com/zycgit/hasor.git
https://github.com/sesuncedu/htmlparser.git
https://github.com/lsds/SEEP.git
https://github.com/stephenostermiller/ostermillerutils.git
https://github.com/FamilySearch/gedcomx-java.git
https://github.com/apache/ddlutils.git
https://github.com/optimizationBenchmarking/optimizationBenchmarking.git
https://github.com/SomMeri/less4j.git
https://github.com/xwiki/xwiki-rendering.git
https://github.com/worm1k/OSS_Cauliflower.git
https://github.com/sakai-mirror/portal.git
https://github.com/aht-group/ofx.git
https://github.com/apache/struts.git
https://github.com/stapler/stapler.git
https://github.com/cismet/cids-navigator.git
https://github.com/virgo47/javasimon.git
https://github.com/xwic/appkit.git
https://github.com/tehbeard/BeardStat.git
https://github.com/nuxeo/nuxeo-dam.git
https://github.com/DavidAlphaFox/jetlang-core.git
https://github.com/openspim/SPIMAcquisition.git
https://github.com/dbsoftcombr/dbsfaces.git
https://github.com/sualeh/DaylightChart.git
https://github.com/mkovatsc/Californium.git
https://github.com/meltmedia/cadmium.git
https://github.com/diirt/graphene.git
https://github.com/martiner/gooddata-java.git
https://github.com/jdr0887/MaPSeq-Pipeline-CASAVA.git
https://github.com/egonw/cdk.git
https://github.com/AtlasOfLivingAustralia/biocache-service.git
https://github.com/ChiralBehaviors/Ultrastructure.git
https://github.com/lobid/lodmill.git
https://github.com/paul-hammant/qdox.git
https://github.com/SonarCommunity/sonar-javascript.git
https://github.com/jim618/multibit.git
https://github.com/dropwizard/dropwizard.git
https://github.com/tranzero/SoftServe_Java-105_Project.git
https://github.com/tinkerpop/blueprints.git
https://github.com/webbukkit/dynmap.git
https://github.com/bitcoinj/bitcoinj.git
https://github.com/AxonFramework/AxonFramework.git
https://github.com/hawkular/hawkular-metrics.git
https://github.com/picketlink/picketlink.git
https://github.com/opendaylight/bgpcep.git
https://github.com/rage/tmc-cli.git
https://github.com/openfigis/vme.git
https://github.com/apache/commons-codec.git
https://github.com/square/wire.git
https://github.com/Wikidata/Wikidata-Toolkit.git
https://github.com/itm/testbed-runtime.git
https://github.com/tomp2p/TomP2P.git
https://github.com/j256/ormlite-core.git
https://github.com/nishihatapalmer/byteseek.git
https://github.com/rapla/rapla.git
https://github.com/Bombe/Sone.git
https://github.com/yusuke/twitter4j.git
https://github.com/jhpoelen/eol-globi-data.git
https://github.com/google/error-prone.git
https://github.com/basho/riak-java-client.git
https://github.com/jamesagnew/hapi-fhir.git
https://github.com/brooklyncentral/clocker.git
https://github.com/spring-projects/spring-data-commons.git
https://github.com/SonarCommunity/sonar-php.git
https://github.com/imglib/imglib2.git
https://github.com/h2oai/h2o.git
https://github.com/lucmoreau/ProvToolbox.git
https://github.com/threerings/tripleplay.git
https://github.com/chapmajs/shiro.git
https://github.com/zalando-stups/fullstop.git
https://github.com/airlift/airlift.git
https://github.com/dcm4che/dcm4che.git
https://github.com/myui/hivemall.git
https://github.com/cismet/cismap-commons.git
https://github.com/suggitpe/java-sandbox.git
https://github.com/zanata/zanata-client.git
https://github.com/dalderliesten/Scrumbledore.git
https://github.com/cojen/Tupl.git
https://github.com/pendelhaven3/magic.git
https://github.com/jenkinsci/acceptance-test-harness.git
https://github.com/gbif/occurrence.git
https://github.com/OpenHFT/Java-Lang.git
https://github.com/doanduyhai/Achilles.git
https://github.com/shilad/wikibrain.git
https://github.com/xdoo/vaadin-demo.git
https://github.com/nuxeo/nuxeo-platform-document-routing.git
https://github.com/jitlogic/zorka.git
https://github.com/UniTime/unitime.git
https://github.com/anba/es6draft.git
https://github.com/yogendra12/Rescuefy.git
https://github.com/bcdev/ceres.git
https://github.com/OpenTreeOfLife/treemachine.git
https://github.com/Carboni/zebedee.git
https://github.com/Gigaspaces/mongo-datasource.git
https://github.com/mebigfatguy/fb-contrib.git
https://github.com/google/guava.git
https://github.com/mleduque/ide.git
https://github.com/finmath/finmath-lib.git
https://github.com/sbwhitecap/Phex-trunk.git
https://github.com/named-data/jndn.git
https://github.com/thinkaurelius/titan.git
https://github.com/apache/accumulo.git
https://github.com/Activiti/Activiti.git
https://github.com/apache/pdfbox.git
https://github.com/apache/servicemix4-bundles.git
https://github.com/deeplearning4j/deeplearning4j.git
https://github.com/checkstyle/checkstyle.git
https://github.com/IntroPV/vainilla.git
https://github.com/korpling/ANNIS.git
https://github.com/undertow-io/undertow.git
https://github.com/arjovanramshorst/bubble-bobble-sem.git
https://github.com/bcdev/coastcolour.git
https://github.com/nuxeo/nuxeo-platform-semantic-entities.git
https://github.com/protegeproject/swrlapi.git
https://github.com/jenkinsci/envinject-plugin.git
https://github.com/jensnerche/plantuml.git
https://github.com/teanutella/AppEmployee.git
https://github.com/SonarSource/sonar-python.git
https://github.com/Atmosphere/nettosphere.git
https://github.com/apache/maven-release.git
https://github.com/svn2github/commons-vfs2.git
https://github.com/guibin/Knowledge.git
https://github.com/kmbulebu/NickNack.git
https://github.com/alecgorge/jsonapi.git
https://github.com/aaron-santos/lanterna.git
https://github.com/Appendium/flatpack.git
https://github.com/google/closure-templates.git
https://github.com/XBigTK13X/Aigilas.git
https://github.com/greese/dasein-cloud-openstack.git
https://github.com/Graphity/graphity-client.git
https://github.com/mung3r/ecoCreature.git
https://github.com/harrifeng/java-in-action.git
https://github.com/no-hope/java-toolkit.git
https://github.com/tinman89/tinspx-utils.git
https://github.com/spring-cloud/spring-cloud-aws.git
https://github.com/DigitalPebble/storm-crawler.git
https://github.com/bitbar/testdroid-api.git
https://github.com/takari/takari-lifecycle.git
https://github.com/EHRI/ehri-rest.git
https://github.com/grisu/gricli.git
https://github.com/FasterXML/jackson-dataformat-xml.git
https://github.com/photon-infotech/commons.git
https://github.com/ClemsonRSRG/RESOLVE.git
https://github.com/Ensembl/ensj-healthcheck.git
https://github.com/haraldk/TwelveMonkeys.git
https://github.com/lkroesen/BubbleBobble.git
https://github.com/sugarcrm/candybean.git
https://github.com/stackmob/stackmob-java-client-sdk.git
https://github.com/undera/jmeter-plugins.git
https://github.com/ldbc/ldbc_driver.git
https://github.com/mojohaus/nbm-maven-plugin.git
https://github.com/openpnp/openpnp.git
https://github.com/paulhoule/infovore.git
https://github.com/simoc/csvjdbc.git
https://github.com/vkostyukov/la4j.git
https://github.com/valis/vclang.git
https://github.com/objectify/objectify.git
https://github.com/apache/maven-surefire.git
https://github.com/bpiwowar/experimaestro.git
https://github.com/visiriCEP/VISIRI.git
https://github.com/Incoding/apiTest.git
https://github.com/revapi/revapi.git
https://github.com/jdr0887/MaPSeq-Pipeline-RNASeq.git
https://github.com/jdr0887/MaPSeq-Pipeline-NCGenes-DepthOfCoverage.git
https://github.com/StopBadware/dsp-core.git
https://github.com/DerPavlov/Cannons.git
https://github.com/thinkofdeath/ThinkMap.git
https://github.com/nuxeo/nuxeo-diff.git
https://github.com/apache/mina-sshd.git
https://github.com/buddycloud/buddycloud-server-java.git
https://github.com/jenkinsci/remoting.git
https://github.com/PIH/mirebalais-smoke-tests.git
https://github.com/antoine-tran/Hedera.git
https://github.com/carrotsearch/randomizedtesting.git
https://github.com/keeps/db-preservation-toolkit.git
https://github.com/jbossas/jboss-ejb-client.git
https://github.com/hortonworks/knox.git
https://github.com/jbossws/jbossws-spi.git
https://github.com/VincSch/Photoplatform_Angular_Spring.git
https://github.com/eileenzheng/playground.git
https://github.com/selig/qea.git
https://github.com/apache/commons-bcel.git
https://github.com/apache/directory-shared.git
https://github.com/triceo/splitlog.git
https://github.com/ovgu-ccd/jchess.git
https://github.com/jsr107/jsr107tck.git
https://github.com/jenkinsci/matrix-project-plugin.git
https://github.com/datasalt/pangool.git
https://github.com/exoplatform/integration.git
https://github.com/kovertopz/Framework-GL.git
https://github.com/desht/ScrollingMenuSign.git
https://github.com/FluentLenium/FluentLenium.git
https://github.com/apache/maven-scm.git
https://github.com/KnisterPeter/Smaller.git
https://github.com/oboformat/oboformat.git
https://github.com/FoxDev/FoxBot.git
https://github.com/sonatype/nexus-maven-plugins.git
https://github.com/bridgedb/BridgeDb.git
https://github.com/locationtech/spatial4j.git
https://github.com/jenkinsci/sauce-ondemand-plugin.git
https://github.com/kohsuke/github-api.git
https://github.com/niyue/coding.git
https://github.com/roundrop/facebook4j.git
https://github.com/sk89q/CommandBook.git
https://github.com/cylong1016/NJULily.git
https://github.com/vivantech/kc_fixes.git
https://github.com/Weltraumschaf/JUberblog.git
https://github.com/marytts/marytts.git
https://github.com/richardwilly98/elasticsearch-river-mongodb.git
https://github.com/andsel/moquette.git
https://github.com/diorcety/maven-dependency-plugin.git
https://github.com/gbif/checklistbank.git
https://github.com/huangyingw/blablacode.git
https://github.com/reinert/requestor.git
https://github.com/tntim96/JSCover.git
https://github.com/xenomorpheus/heisenberg.git
https://github.com/zyong2004/mybatis-spring.git
https://github.com/roberth/pitest.git
https://github.com/JensBee/QueryClarity.git
https://github.com/Weltraumschaf/commons.git
https://github.com/greese/dasein-cloud-aws.git
https://github.com/bguerout/jongo.git
https://github.com/thomasjungblut/thomasjungblut-common.git
https://github.com/goalhub/runtime.git
https://github.com/apache/directory-server.git
https://github.com/clojure/clojure.git
https://github.com/wnameless/rubycollect4j.git
https://github.com/omnifaces/omnifaces.git
https://github.com/Razz0991/Minigames.git
https://github.com/apache/commons-fileupload.git
https://github.com/ChandraCXC/iris.git
https://github.com/zyyettie/LaaS.git
https://github.com/alibaba/druid.git
https://github.com/thothbot/parallax.git
https://github.com/msokolov/lux.git
https://github.com/jenkinsci/git-client-plugin.git
https://github.com/statsbiblioteket/newspaper-batch-event-framework.git
https://github.com/apache/curator.git
https://github.com/mikesname/ehri-rest.git
https://github.com/apache/chemistry-opencmis.git
https://github.com/Dandelion/dandelion.git
https://github.com/yuchaosydney/kouchat.git
https://github.com/xEssentials/xEssentials.git
https://github.com/couchbase/couchbase-java-client.git
https://github.com/sitewhere/sitewhere.git
https://github.com/scriptella/scriptella-etl.git
https://github.com/medusa-project/cantaloupe.git
https://github.com/charite/jannovar.git
https://github.com/code4craft/webmagic.git
https://github.com/tavlima/fosstrak-epcis.git
https://github.com/IMExConsortium/dip-proxy.git
https://github.com/junit-team/junit.git
https://github.com/twilio/twilio-java.git
https://github.com/sannies/mp4parser.git
https://github.com/xerial/sqlite-jdbc.git
https://github.com/seges/sesam.git
https://github.com/moravianlibrary/RecordManager2.git
https://github.com/Omertron/api-themoviedb.git
https://github.com/KittehOrg/KittehIRCClientLib.git
https://github.com/taverna/taverna-engine-core.git
https://github.com/dankurka/mgwt.git
https://github.com/FITeagle/adapters.git
https://github.com/drnoa/schemaspy.git
https://github.com/carrotsearch/hppc.git
https://github.com/neo4j/neo4j-ogm.git
https://github.com/calrissian/accumulo-recipes.git
https://github.com/apache/maven.git
https://github.com/GluuFederation/oxTrust.git
https://github.com/sanity/quickml.git
https://github.com/davidmoten/rxjava-extras.git
https://github.com/elex-bigdata/lldaMahout.git
https://github.com/tengstrand/Laja.git
https://github.com/spring-cloud/spring-cloud-config.git
https://github.com/trajano/doxdb.git
https://github.com/yegor256/s3auth.git
https://github.com/greese/dasein-cloud-core.git
https://github.com/jdr0887/MaPSeq-Pipeline-NCGenes.git
https://github.com/sesuncedu/elk-reasoner.git
https://github.com/ColoradoSchoolOfMines/interface_sdk.git
https://github.com/rhuss/docker-maven-plugin.git
https://github.com/k-hotta/SCAnalyzer.git
https://github.com/tastybento/askyblock.git
https://github.com/osiam/connector4java.git
https://github.com/juzu/juzu.git
https://github.com/OpenHFT/Chronicle-Queue.git
https://github.com/junkdog/artemis-odb.git
https://github.com/apache/archiva.git
https://github.com/ESAPI/esapi-java-legacy.git
https://github.com/motech/modules.git
https://github.com/Benoker/DockingFrames.git
https://github.com/barchart/barchart-feed.git
https://github.com/swagger-api/swagger-core.git
https://github.com/droolsjbpm/droolsjbpm-knowledge.git
https://github.com/soul2zimate/resteasy2.git
https://github.com/konsoletyper/teavm.git
https://github.com/veraPDF/veraPDF-library.git
https://github.com/VictorBac/LO23.git
https://github.com/europeana/corelib.git
https://github.com/GoogleCloudPlatform/gcloud-java.git
https://github.com/mojohaus/jaxws-maven-plugin.git
https://github.com/threerings/clyde.git
https://github.com/maxymania/jxta-jxse.git
https://github.com/xetorthio/jedis.git
https://github.com/thervh70/SEM_Team9.git
https://github.com/PerfCake/PerfCake.git
https://github.com/plutext/docx4j.git
https://github.com/slipcor/pvparena.git
https://github.com/barchart/barchart-feed-ddf.git
https://github.com/SpigotMC/BungeeCord.git
https://github.com/forcedotcom/phoenix.git
https://github.com/eFaps/eFaps-WebApp.git
https://github.com/AtlasOfLivingAustralia/spatial-portal.git
https://github.com/eugen-eugen/eugensjbehave.git
https://github.com/GenomicParisCentre/aozan.git
https://github.com/ikasanEIP/ikasan.git
https://github.com/WorldCretornica/PlotMe-Core.git
https://github.com/marklogic/java-client-api.git
https://github.com/apache/jackrabbit.git
https://github.com/tfredrich/docussandra.git
https://github.com/codeine-cd/codeine.git
https://github.com/Mobicents/sip-servlets.git
https://github.com/movsim/movsim.git
https://github.com/threerings/playn.git
https://github.com/jcabi/jcabi-github.git
https://github.com/apache/wss4j.git
https://github.com/galenframework/galen.git
https://github.com/marylinh/ESAPI_JAVA_ALL.git
https://github.com/wso2/carbon-mediation.git
https://github.com/SINTEF-9012/ThingML.git
https://github.com/OhmData/hbase-public.git
https://github.com/nodebox/nodebox.git
https://github.com/jayway/powermock.git
https://github.com/julianhyde/optiq.git
https://github.com/ImmobilienScout24/deadcode4j.git
https://github.com/rhuss/jolokia.git
https://github.com/taverna/taverna-ui-components.git
https://github.com/cloudera/cdk.git
https://github.com/qcadoo/qcadoo.git
https://github.com/ebean-orm/avaje-ebeanorm.git
https://github.com/simon-eastwood/DependencyCheckCM.git
https://github.com/spring-projects/spring-data-rest.git
https://github.com/hudec/sql-processor.git
https://github.com/aherbert/GDSC-SMLM.git
https://github.com/rovo89/public-transport-enabler-fork.git
https://github.com/Hidendra/LWC.git
https://github.com/twizmwazin/CardinalPGM.git
https://github.com/mrniko/redisson.git
https://github.com/bigdataops/bgpcep.git
https://github.com/GrizzlyNIO/grizzly-mirror.git
https://github.com/adangel/pmd.git
https://github.com/lrozenblyum/chess.git
https://github.com/nuxeo/nuxeo-drive-server.git
https://github.com/imagej/imagej-legacy.git
https://github.com/tyrus-project/tyrus.git
https://github.com/elBukkit/MagicLib.git
https://github.com/libetl/soundtransform.git
https://github.com/yegor256/takes.git
https://github.com/graphaware/neo4j-framework.git
https://github.com/sboesebeck/morphium.git
https://github.com/OpenRock/OpenIG.git
https://github.com/DeveloperLiberationFront/social-screencasting-core.git
https://github.com/geenen124/Team1BubbleTrouble.git
https://github.com/kaazing/gateway.git
https://github.com/sakai-mirror/sam.git
https://github.com/apache/commons-configuration.git
https://github.com/fiji/TrackMate3.git
https://github.com/yegor256/thindeck.git
https://github.com/Eluinhost/ultrahardcore.git
https://github.com/caprica/vlcj.git
https://github.com/threerings/depot.git
https://github.com/ysc/word.git
https://github.com/slowenthal/classproject.git
https://github.com/apache/directory-fortress-core.git
https://github.com/greese/dasein-cloud-google.git
https://github.com/insideo/randomcoder-website.git
https://github.com/lemire/RoaringBitmap.git
https://github.com/telefonicaid/fiware-cygnus.git
https://github.com/hk2-project/hk2.git
https://github.com/ninjaframework/ninja.git
https://github.com/Xephi/AuthMeReloaded.git
https://github.com/droolsjbpm/jbpm-form-modeler.git
https://github.com/gwtbootstrap/gwt-bootstrap.git
https://github.com/objectos/way.git
https://github.com/maxapryg/UITest.git
https://github.com/ghedlund/phon.git
https://github.com/aadnk/ProtocolLib.git
https://github.com/karamelchef/karamel.git
https://github.com/Hansschouten/context_health_informatics.git
https://github.com/apache/incubator-streams.git
https://github.com/kuujo/vertigo.git
https://github.com/vmi/selenese-runner-java.git
https://github.com/spotify/docker-client.git
https://github.com/JWebUnit/jwebunit.git
https://github.com/johncarl81/transfuse.git
https://github.com/sakai-mirror/kernel.git
https://github.com/CloudSlang/cloud-slang.git
https://github.com/datanucleus/datanucleus-core.git
https://github.com/eXistence/TeeTime.git
https://github.com/jitsi/libjitsi.git
https://github.com/FasterXML/jackson-core.git
https://github.com/mizdebsk/xmvn.git
https://github.com/ATLauncher/ATLauncher.git
https://github.com/maartentbm/ContextPL1.git
https://github.com/apache/incubator-ambari.git
https://github.com/spring-projects/spring-data-neo4j.git
https://github.com/apache/uima-uimaj.git
https://github.com/mongodb/morphia.git
https://github.com/minusone13/InvoicingSystem.git
https://github.com/jqno/equalsverifier.git
https://github.com/sdl/Testy.git
https://github.com/fcrepo4/fcrepo4.git
https://github.com/GenomicParisCentre/eoulsan.git
https://github.com/te-con/ehour.git
https://github.com/ansell/owlapi.git
https://github.com/opendaylight/yangtools.git
https://github.com/OpenRock/OpenDJ.git
https://github.com/JolantaWojcik/biojavaOwn.git
https://github.com/DirtyUnicorns/android_external_spongycastle.git
https://github.com/owlcs/owlapi.git
https://github.com/stormpath/stormpath-sdk-java.git
https://github.com/apache/clerezza.git
https://github.com/aht-group/utils.git
https://github.com/webanno/webanno.git
https://github.com/apache/mahout.git
https://github.com/hal/core.git
https://github.com/playernodie/weupnp.git
https://github.com/apache/servicemix-bundles.git
https://github.com/robovm/robovm.git
https://github.com/graphhopper/graphhopper.git
https://github.com/perfectsense/dari.git
https://github.com/svn2github/SAT4J.git
https://github.com/jeremylong/DependencyCheck.git
https://github.com/apache/qpid-proton.git
https://github.com/svn2github/forge.git
https://github.com/droolsjbpm/optaplanner.git
https://github.com/soluvas/soluvas-framework.git
https://github.com/droolsjbpm/droolsjbpm-integration.git
https://github.com/swagger-api/swagger-codegen.git
https://github.com/druid-io/druid.git
https://github.com/FasterXML/jackson-databind.git
https://github.com/biojava/biojava.git
https://github.com/telefonicaid/fiware-cosmos-ambari.git
https://github.com/kyoken74/gwt-angular.git
https://github.com/apache/commons-lang.git
https://github.com/scifio/scifio.git
https://github.com/alibaba/RocketMQ.git
https://github.com/kuujo/copycat.git
https://github.com/jankotek/MapDB.git
https://github.com/droolsjbpm/kie-wb-distributions.git
https://github.com/buschmais/jqassistant.git
https://github.com/PLOS/ambra.git
https://github.com/spring-cloud/spring-cloud-netflix.git
https://github.com/apache/commons-math.git
https://github.com/OpenHFT/Chronicle-Network.git
https://github.com/chocoteam/choco3.git
https://github.com/apache/flume.git
https://github.com/rinde/RinSim.git
https://github.com/kotcrab/VisEditor.git
https://github.com/void256/nifty-gui.git
https://github.com/apache/uima-ducc.git
https://github.com/windup/windup.git
https://github.com/heuermh/dishevelled.git
https://github.com/salyh/javamailspec.git
https://github.com/SpoutDev/Spout.git
https://github.com/antlr/antlr4.git
https://github.com/Flipkart/quartz.git
https://github.com/rasto/lcmc.git
https://github.com/mafagafogigante/dungeon.git
https://github.com/PerfGeeks/pinpoint.git
https://github.com/opendaylight/openflowplugin.git
https://github.com/EngineHub/CommandHelper.git
https://github.com/tananaev/traccar.git
https://github.com/BroadleafCommerce/BroadleafCommerce.git
https://github.com/apache/hadoop-common.git
https://github.com/spring-projects/spring-boot.git
https://github.com/bcdev/beam.git
https://github.com/weld/core.git
https://github.com/elki-project/elki.git
https://github.com/belaban/JGroups.git
https://github.com/molgenis/molgenis.git
https://github.com/lennartj/jaxb2-maven-plugin.git
https://github.com/bugcy013/opennms-tmp-tools.git
https://github.com/Talend/tcommon-studio-se.git
https://github.com/netty/netty.git
https://github.com/openmicroscopy/bioformats.git
https://github.com/droolsjbpm/guvnor.git
https://github.com/jetty-project/codehaus-jetty6.git
https://github.com/apache/jackrabbit-oak.git
https://github.com/deegree/deegree3.git
https://github.com/apache/sling.git
https://github.com/EriclLee/ActiveMQ-On-Azure.git
https://github.com/amplab/tachyon.git
https://github.com/neo4j/neo4j.git
https://github.com/darranl/directory-server.git
https://github.com/hazelcast/hazelcast.git
https://github.com/infinispan/infinispan.git
https://github.com/DSpace/DSpace.git
https://github.com/mulesoft/mule.git
https://github.com/imCodePartnerAB/imcms.git
https://github.com/backslash47/webstart-maven-plugin.git
https://github.com/apache/hbase.git
https://github.com/languagetool-org/languagetool.git
https://github.com/droolsjbpm/drools.git
https://github.com/SpigotMC/Spigot-API.git
https://github.com/irstv/orbisgis.git
https://github.com/jgheerardyn/yields.git
https://github.com/ning/killbill.git
https://github.com/jline/jline2.git
https://github.com/altran/Whydah-UserIdentityBackend.git
https://github.com/encog/encog-java-core.git
https://github.com/highsource/jaxb2-basics.git
https://github.com/msoute/vertx-deploy-tools.git
https://github.com/apache/portals-pluto.git
https://github.com/antlr/stringtemplate4.git
https://github.com/kongchen/swagger-maven-plugin.git
https://github.com/gertvv/addis.git
https://github.com/ProjectCCNx/ccnx.git
https://github.com/mulesoft/mule-common.git
https://github.com/cismet/cismap-plugin.git
https://github.com/greese/dasein-cloud-azure.git
https://github.com/Requinard/TeamTab.git
https://github.com/SonarSource/jenkins-sonar-plugin.git
https://github.com/fbacchella/jrds.git
https://github.com/cdi-spec/cdi-tck.git
https://github.com/gwtbootstrap3/gwtbootstrap3.git
https://github.com/apache/commons-dbcp.git
https://github.com/timmolter/XChart.git
https://github.com/aranega/testgen.git
https://github.com/mysticfall/pivot4j.git
https://github.com/datasift/datasift-java.git
https://github.com/viktor-z/fb2pdf.git
https://github.com/Cas-B/Group17-BubbleBobble.git
https://github.com/TheCoder4eu/BootsFaces-OSP.git
https://github.com/UnifiedViews/Plugin-DevEnv.git
https://github.com/jidesoft/jide-oss.git
https://github.com/codescape/bitvunit.git
https://github.com/grandwazir/BanHammer.git
https://github.com/wildfly/wildfly-arquillian.git
https://github.com/Eluinhost/pluginframework.git
https://github.com/mtedone/podam.git
https://github.com/Mobicents/jss7.git
https://github.com/basis-technology-corp/rosette-common-java-api.git
https://github.com/webbit/webbit.git
https://github.com/GWASpi/GWASpi.git
https://github.com/jenkinsci/warnings-plugin.git
https://github.com/google/auto.git
https://github.com/TechnicPack/LauncherV3.git
https://github.com/ArtificerRepo/artificer.git
https://github.com/Jahia/external-provider.git
https://github.com/ICGC-TCGA-PanCancer/SeqWare-CGP-SomaticCore.git
https://github.com/PEXPlugins/PermissionsEx.git
logs/FixedViolationCodeParseResults.log
-3521
View File
File diff suppressed because it is too large Load Diff
logs/HaveSourceCode.txt
-87622
View File
File diff suppressed because it is too large Load Diff
logs/MultiMethodsOrFieldOrInnerClasses.txt
-3867
View File
File diff suppressed because it is too large Load Diff
logs/MultiV.txt
-4327
View File
File diff suppressed because it is too large Load Diff
logs/Null.txt
-124
View File
@@ -1,124 +0,0 @@
1
BC_VACUOUS_INSTANCEOF Ignore it. no source code, no source code change
CN_IDIOM_NO_SUPER_CALL Ignore it. no source code, no source code change
CN_IMPLEMENTS_CLONE_BUT_NOT_CLONEABLE Ignore it. no source code, no source code change
DLS_DEAD_LOCAL_STORE Ignore it. no source code, no source code change
DM_DEFAULT_ENCODING
DM_NUMBER_CTOR
DM_STRING_CTOR
DM_STRING_VOID_CTOR
DP_DO_INSIDE_DO_PRIVILEGED
EI_EXPOSE_REP
EQ_COMPARETO_USE_OBJECT_EQUALS
EQ_DOESNT_OVERRIDE_EQUALS
EQ_SELF_USE_OBJECT
HE_EQUALS_USE_HASHCODE
HE_HASHCODE_USE_OBJECT_EQUALS
HE_INHERITS_EQUALS_USE_HASHCODE
IS2_INCONSISTENT_SYNC
MS_CANNOT_BE_FINAL
MS_MUTABLE_ARRAY
MS_PKGPROTECT
MS_SHOULD_BE_FINAL
NM_CLASS_NAMING_CONVENTION
NM_CLASS_NOT_EXCEPTION
NM_SAME_SIMPLE_NAME_AS_SUPERCLASS
OBL_UNSATISFIED_OBLIGATION_EXCEPTION_EDGE
OS_OPEN_STREAM
OS_OPEN_STREAM_EXCEPTION_PATH
RI_REDUNDANT_INTERFACES
SA_FIELD_DOUBLE_ASSIGNMENT
SE_COMPARATOR_SHOULD_BE_SERIALIZABLE
SE_NO_SERIALVERSIONID
SIO_SUPERFLUOUS_INSTANCEOF
UCF_USELESS_CONTROL_FLOW
UG_SYNC_SET_UNSYNC_GET
UPM_UNCALLED_PRIVATE_METHOD
URF_UNREAD_FIELD
URF_UNREAD_PUBLIC_OR_PROTECTED_FIELD
UWF_FIELD_NOT_INITIALIZED_IN_CONSTRUCTOR
UWF_UNWRITTEN_FIELD
UWF_UNWRITTEN_PUBLIC_OR_PROTECTED_FIELD
WMI_WRONG_MAP_ITERATOR
-1
CI_CONFUSED_INHERITANCE
CO_ABSTRACT_SELF
DMI_COLLECTION_OF_URLS
EQ_ABSTRACT_SELF
HE_SIGNATURE_DECLARES_HASHING_OF_UNHASHABLE_CLASS
JCIP_FIELD_ISNT_FINAL_IN_IMMUTABLE_CLASS
ME_MUTABLE_ENUM_FIELD
MF_CLASS_MASKS_FIELD
MS_CANNOT_BE_FINAL
MS_PKGPROTECT
MS_SHOULD_BE_FINAL
NM_FIELD_NAMING_CONVENTION
NM_SAME_SIMPLE_NAME_AS_INTERFACE
SE_BAD_FIELD
SE_TRANSIENT_FIELD_NOT_RESTORED
STCAL_STATIC_SIMPLE_DATE_FORMAT_INSTANCE
UPM_UNCALLED_PRIVATE_METHOD
UUF_UNUSED_FIELD
UUF_UNUSED_PUBLIC_OR_PROTECTED_FIELD
UWF_NULL_FIELD
UWF_UNWRITTEN_FIELD
UWF_UNWRITTEN_PUBLIC_OR_PROTECTED_FIELD
VO_VOLATILE_REFERENCE_TO_ARRAY
CI_CONFUSED_INHERITANCE change modifier of fields from protected to private or public, or remove final of class.
CO_ABSTRACT_SELF Interface, remove compareTo() method.
SE_NO_SERIALVERSIONID add a field: serialVersionUID
EQ_ABSTRACT_SELF Interface, remove compareTo() method.
EQ_COMPARETO_USE_OBJECT_EQUALS: Update or Delete compareTo(), Add equals()
EQ_DOESNT_OVERRIDE_EQUALS override equals()
HE_SIGNATURE_DECLARES_HASHING_OF_UNHASHABLE_CLASS remove equals() only one instance
ME_MUTABLE_ENUM_FIELD enum, field add final.
MF_CLASS_MASKS_FIELD, change super class or delete the field with a same name in super class.
MS_SHOULD_BE_FINAL Add final key word
STCAL_STATIC_SIMPLE_DATE_FORMAT_INSTANCE remove public static final DateFormat DATE_FORMAT.... or SimpleDateFormat
UUF_UNUSED_FIELD remove unused fields. not sure
UUF_UNUSED_PUBLIC_OR_PROTECTED_FIELD field
UWF_NULL_FIELD update field, remove field
UWF_UNWRITTEN_FIELD field
UWF_UNWRITTEN_PUBLIC_OR_PROTECTED_FIELD remove field
VO_VOLATILE_REFERENCE_TO_ARRAY field
all GumTree results. class, field, method, //statements.
NM_SAME_SIMPLE_NAME_AS_INTERFACE rename class name or remove super class
NM_SAME_SIMPLE_NAME_AS_SUPERCLASS, class name
NM_METHOD_NAMING_CONVENTION
NM_CLASS_NAMING_CONVENTION class name
NM_CLASS_NOT_EXCEPTION class name
RI_REDUNDANT_INTERFACES class implemented interface
SE_NO_SUITABLE_CONSTRUCTOR Serializable Class level remove Serializable interface and serialVersionUID
SIC_INNER_SHOULD_BE_STATIC add static to inner class, not sure, delete it, 如果是匿名内部类，该如何处理？ ，其他的直接内部类的范围。
SIC_INNER_SHOULD_BE_STATIC_NEEDS_THIS 同上
===
CO_ABSTRACT_SELF inner class
SE_INNER_CLASS inner class
SE_COMPARATOR_SHOULD_BE_SERIALIZABLE, inner class, 读definition
SE_BAD_FIELD_INNER_CLASS inner class
SIC_INNER_SHOULD_BE_STATIC_ANON inner class
HE_INHERITS_EQUALS_USE_HASHCODE inner class or class
CN_IDIOM add clone method. or update clone method
DM_NUMBER_CTOR multi v multi patches
EQ_COMPARETO_USE_OBJECT_EQUALS, the same as before
IC_INIT_CIRCULARITY, multi V, multi patches>
SE_NO_SUITABLE_CONSTRUCTOR_FOR_EXTERNALIZATION, add a constructor or add public to old constructor,
NP_NONNULL_FIELD_NOT_INITIALIZED_IN_CONSTRUCTOR
SE_NO_SERIALVERSIONID change superclass or interface, add field or remove @SuppressWarnings("serial"), some are inner class
logs/NullSourceCode.txt
-1305
View File
File diff suppressed because it is too large Load Diff
logs/NullV.txt
-3130
View File
File diff suppressed because it is too large Load Diff
logs/fixedTypes.txt
-339
View File
@@ -1,339 +0,0 @@
SIC_INNER_SHOULD_BE_STATIC_ANON
DLS_DEAD_LOCAL_STORE
DM_CONVERT_CASE
DM_DEFAULT_ENCODING
UWF_FIELD_NOT_INITIALIZED_IN_CONSTRUCTOR
RCN_REDUNDANT_NULLCHECK_OF_NONNULL_VALUE
NM_METHOD_NAMING_CONVENTION
URF_UNREAD_FIELD
BC_UNCONFIRMED_CAST
REC_CATCH_EXCEPTION
BC_UNCONFIRMED_CAST_OF_RETURN_VALUE
SE_NO_SERIALVERSIONID
UPM_UNCALLED_PRIVATE_METHOD
VA_FORMAT_STRING_USES_NEWLINE
MS_SHOULD_BE_FINAL
RV_RETURN_VALUE_IGNORED_BAD_PRACTICE
ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD
EI_EXPOSE_REP2
URF_UNREAD_PUBLIC_OR_PROTECTED_FIELD
WMI_WRONG_MAP_ITERATOR
OBL_UNSATISFIED_OBLIGATION
EI_EXPOSE_REP
NP_LOAD_OF_KNOWN_NULL_VALUE
DM_NUMBER_CTOR
SIC_INNER_SHOULD_BE_STATIC
SBSC_USE_STRINGBUFFER_CONCATENATION
OS_OPEN_STREAM_EXCEPTION_PATH
NP_NONNULL_RETURN_VIOLATION
SF_SWITCH_NO_DEFAULT
UWF_UNWRITTEN_FIELD
DE_MIGHT_IGNORE
IS2_INCONSISTENT_SYNC
DM_BOXED_PRIMITIVE_FOR_PARSING
RV_RETURN_VALUE_IGNORED_NO_SIDE_EFFECT
ODR_OPEN_DATABASE_RESOURCE
PZLA_PREFER_ZERO_LENGTH_ARRAYS
RI_REDUNDANT_INTERFACES
NP_NULL_ON_SOME_PATH_FROM_RETURN_VALUE
UCF_USELESS_CONTROL_FLOW
UC_USELESS_CONDITION
NP_NULL_ON_SOME_PATH
UC_USELESS_OBJECT
DM_FP_NUMBER_CTOR
MS_PKGPROTECT
SQL_PREPARED_STATEMENT_GENERATED_FROM_NONCONSTANT_STRING
OBL_UNSATISFIED_OBLIGATION_EXCEPTION_EDGE
ES_COMPARING_STRINGS_WITH_EQ
OS_OPEN_STREAM
RCN_REDUNDANT_NULLCHECK_WOULD_HAVE_BEEN_A_NPE
NP_PARAMETER_MUST_BE_NONNULL_BUT_MARKED_AS_NULLABLE
SS_SHOULD_BE_STATIC
FE_FLOATING_POINT_EQUALITY
HE_EQUALS_USE_HASHCODE
DM_EXIT
NM_CONFUSING
LI_LAZY_INIT_STATIC
BX_UNBOXING_IMMEDIATELY_REBOXED
EQ_COMPARETO_USE_OBJECT_EQUALS
SQL_NONCONSTANT_STRING_PASSED_TO_EXECUTE
NP_NULL_ON_SOME_PATH_EXCEPTION
NP_UNWRITTEN_FIELD
MS_MUTABLE_COLLECTION_PKGPROTECT
UWF_UNWRITTEN_PUBLIC_OR_PROTECTED_FIELD
CN_IMPLEMENTS_CLONE_BUT_NOT_CLONEABLE
DMI_HARDCODED_ABSOLUTE_FILENAME
SE_BAD_FIELD
DM_STRING_TOSTRING
UUF_UNUSED_FIELD
ICAST_INTEGER_MULTIPLY_CAST_TO_LONG
EC_UNRELATED_TYPES
BC_VACUOUS_INSTANCEOF
ICAST_IDIV_CAST_TO_DOUBLE
RCN_REDUNDANT_NULLCHECK_OF_NULL_VALUE
ES_COMPARING_PARAMETER_STRING_WITH_EQ
RV_RETURN_VALUE_IGNORED
CI_CONFUSED_INHERITANCE
DB_DUPLICATE_SWITCH_CLAUSES
ODR_OPEN_DATABASE_RESOURCE_EXCEPTION_PATH
NP_ALWAYS_NULL
RpC_REPEATED_CONDITIONAL_TEST
SE_BAD_FIELD_STORE
NP_NULL_PARAM_DEREF
GC_UNRELATED_TYPES
DLS_DEAD_LOCAL_STORE_OF_NULL
RC_REF_COMPARISON_BAD_PRACTICE
MS_MUTABLE_ARRAY
SE_INNER_CLASS
UL_UNRELEASED_LOCK_EXCEPTION_PATH
SE_COMPARATOR_SHOULD_BE_SERIALIZABLE
CN_IDIOM_NO_SUPER_CALL
NP_UNWRITTEN_PUBLIC_OR_PROTECTED_FIELD
UC_USELESS_VOID_METHOD
NP_NONNULL_PARAM_VIOLATION
DMI_INVOKING_TOSTRING_ON_ARRAY
DP_DO_INSIDE_DO_PRIVILEGED
SF_SWITCH_FALLTHROUGH
NP_BOOLEAN_RETURN_NULL
NM_CLASS_NOT_EXCEPTION
SIO_SUPERFLUOUS_INSTANCEOF
NP_GUARANTEED_DEREF
UL_UNRELEASED_LOCK
IP_PARAMETER_IS_DEAD_BUT_OVERWRITTEN
HE_INHERITS_EQUALS_USE_HASHCODE
MS_CANNOT_BE_FINAL
DB_DUPLICATE_BRANCHES
STCAL_INVOKE_ON_STATIC_DATE_FORMAT_INSTANCE
MS_MUTABLE_COLLECTION
IL_INFINITE_RECURSIVE_LOOP
VO_VOLATILE_INCREMENT
JLM_JSR166_UTILCONCURRENT_MONITORENTER
NP_NULL_PARAM_DEREF_NONVIRTUAL
DM_STRING_CTOR
UR_UNINIT_READ
XFB_XML_FACTORY_BYPASS
RV_EXCEPTION_NOT_THROWN
UI_INHERITANCE_UNSAFE_GETRESOURCE
VA_FORMAT_STRING_EXTRA_ARGUMENTS_PASSED
IT_NO_SUCH_ELEMENT
NM_CLASS_NAMING_CONVENTION
RC_REF_COMPARISON
RC_REF_COMPARISON_BAD_PRACTICE_BOOLEAN
DLS_DEAD_LOCAL_STORE_SHADOWS_FIELD
HE_EQUALS_NO_HASHCODE
NS_NON_SHORT_CIRCUIT
NP_DEREFERENCE_OF_READLINE_VALUE
NP_GUARANTEED_DEREF_ON_EXCEPTION_PATH
EQ_DOESNT_OVERRIDE_EQUALS
DM_STRING_VOID_CTOR
DP_CREATE_CLASSLOADER_INSIDE_DO_PRIVILEGED
BIT_SIGNED_CHECK
ME_ENUM_FIELD_SETTER
BC_EQUALS_METHOD_SHOULD_WORK_FOR_ALL_OBJECTS
NS_DANGEROUS_NON_SHORT_CIRCUIT
NP_EQUALS_SHOULD_HANDLE_NULL_ARGUMENT
SR_NOT_CHECKED
RR_NOT_CHECKED
UUF_UNUSED_PUBLIC_OR_PROTECTED_FIELD
VA_FORMAT_STRING_MISSING_ARGUMENT
EC_BAD_ARRAY_COMPARE
NP_NONNULL_FIELD_NOT_INITIALIZED_IN_CONSTRUCTOR
DM_BOXED_PRIMITIVE_FOR_COMPARE
NP_NULL_ON_SOME_PATH_MIGHT_BE_INFEASIBLE
UMAC_UNCALLABLE_METHOD_OF_ANONYMOUS_CLASS
UWF_NULL_FIELD
CO_COMPARETO_INCORRECT_FLOATING
NP_NULL_PARAM_DEREF_ALL_TARGETS_DANGEROUS
NO_NOTIFY_NOT_NOTIFYALL
SC_START_IN_CTOR
RV_RETURN_VALUE_IGNORED_INFERRED
UM_UNNECESSARY_MATH
SA_FIELD_SELF_ASSIGNMENT
BC_IMPOSSIBLE_CAST
LI_LAZY_INIT_UPDATE_STATIC
NP_TOSTRING_COULD_RETURN_NULL
SIC_INNER_SHOULD_BE_STATIC_NEEDS_THIS
AT_OPERATION_SEQUENCE_ON_CONCURRENT_ABSTRACTION
EQ_UNUSUAL
SE_NO_SUITABLE_CONSTRUCTOR
UW_UNCOND_WAIT
MF_CLASS_MASKS_FIELD
FE_TEST_IF_EQUAL_TO_NOT_A_NUMBER
IA_AMBIGUOUS_INVOCATION_OF_INHERITED_OR_OUTER_METHOD
SE_BAD_FIELD_INNER_CLASS
RV_NEGATING_RESULT_OF_COMPARETO
ISC_INSTANTIATE_STATIC_CLASS
HE_HASHCODE_USE_OBJECT_EQUALS
DM_BOOLEAN_CTOR
DMI_RANDOM_USED_ONLY_ONCE
EQ_CHECK_FOR_OPERAND_NOT_COMPATIBLE_WITH_THIS
DM_GC
NM_SAME_SIMPLE_NAME_AS_SUPERCLASS
EC_NULL_ARG
EQ_SELF_USE_OBJECT
RV_ABSOLUTE_VALUE_OF_RANDOM_INT
SE_NO_SUITABLE_CONSTRUCTOR_FOR_EXTERNALIZATION
MS_EXPOSE_REP
SE_TRANSIENT_FIELD_NOT_RESTORED
IM_AVERAGE_COMPUTATION_COULD_OVERFLOW
NP_METHOD_PARAMETER_TIGHTENS_ANNOTATION
NP_METHOD_RETURN_RELAXING_ANNOTATION
UG_SYNC_SET_UNSYNC_GET
NM_FIELD_NAMING_CONVENTION
MS_OOI_PKGPROTECT
UCF_USELESS_CONTROL_FLOW_NEXT_LINE
VA_FORMAT_STRING_BAD_CONVERSION
BX_BOXING_IMMEDIATELY_UNBOXED_TO_PERFORM_COERCION
SA_FIELD_DOUBLE_ASSIGNMENT
BC_BAD_CAST_TO_CONCRETE_COLLECTION
SF_DEAD_STORE_DUE_TO_SWITCH_FALLTHROUGH
MS_FINAL_PKGPROTECT
DMI_COLLECTION_OF_URLS
INT_BAD_COMPARISON_WITH_SIGNED_BYTE
RV_CHECK_FOR_POSITIVE_INDEXOF
RE_CANT_USE_FILE_SEPARATOR_AS_REGULAR_EXPRESSION
RU_INVOKE_RUN
BC_IMPOSSIBLE_DOWNCAST_OF_TOARRAY
ICAST_BAD_SHIFT_AMOUNT
ICAST_INT_CAST_TO_FLOAT_PASSED_TO_ROUND
DM_BOXED_PRIMITIVE_TOSTRING
UR_UNINIT_READ_CALLED_FROM_SUPER_CONSTRUCTOR
WA_NOT_IN_LOOP
CN_IDIOM
DMI_BLOCKING_METHODS_ON_URL
EC_UNRELATED_CLASS_AND_INTERFACE
SWL_SLEEP_WITH_LOCK_HELD
IM_BAD_CHECK_FOR_ODD
RE_POSSIBLE_UNINTENDED_PATTERN
INT_BAD_COMPARISON_WITH_NONNEGATIVE_VALUE
ICAST_INT_CAST_TO_DOUBLE_PASSED_TO_CEIL
EQ_OVERRIDING_EQUALS_NOT_SYMMETRIC
LG_LOST_LOGGER_DUE_TO_WEAK_REFERENCE
DLS_DEAD_LOCAL_STORE_IN_RETURN
J2EE_STORE_OF_NON_SERIALIZABLE_OBJECT_INTO_SESSION
NP_STORE_INTO_NONNULL_FIELD
RANGE_ARRAY_INDEX
IL_INFINITE_LOOP
DC_DOUBLECHECK
NN_NAKED_NOTIFY
EC_UNRELATED_TYPES_USING_POINTER_EQUALITY
SE_PRIVATE_READ_RESOLVE_NOT_INHERITED
INT_VACUOUS_BIT_OPERATION
DM_NEXTINT_VIA_NEXTDOUBLE
VA_FORMAT_STRING_ILLEGAL
DMI_INVOKING_HASHCODE_ON_ARRAY
RV_DONT_JUST_NULL_CHECK_READLINE
RE_BAD_SYNTAX_FOR_REGULAR_EXPRESSION
SA_LOCAL_SELF_ASSIGNMENT_INSTEAD_OF_FIELD
NM_VERY_CONFUSING
MSF_MUTABLE_SERVLET_FIELD
BC_IMPOSSIBLE_INSTANCEOF
SA_LOCAL_SELF_COMPARISON
SA_LOCAL_SELF_ASSIGNMENT
IC_SUPERCLASS_USES_SUBCLASS_DURING_INITIALIZATION
CNT_ROUGH_CONSTANT_VALUE
SI_INSTANCE_BEFORE_FINALS_ASSIGNED
DLS_DEAD_STORE_OF_CLASS_LITERAL
RV_ABSOLUTE_VALUE_OF_HASHCODE
NM_SAME_SIMPLE_NAME_AS_INTERFACE
HE_HASHCODE_NO_EQUALS
DM_NEW_FOR_GETCLASS
SA_FIELD_SELF_COMPARISON
NM_METHOD_CONSTRUCTOR_CONFUSION
EQ_COMPARING_CLASS_NAMES
UC_USELESS_CONDITION_TYPE
RV_01_TO_INT
SA_LOCAL_DOUBLE_ASSIGNMENT
SQL_BAD_RESULTSET_ACCESS
EQ_GETCLASS_AND_CLASS_CONSTANT
BSHIFT_WRONG_ADD_PRIORITY
HRS_REQUEST_PARAMETER_TO_HTTP_HEADER
DMI_USING_REMOVEALL_TO_CLEAR_COLLECTION
STCAL_STATIC_SIMPLE_DATE_FORMAT_INSTANCE
FI_FINALIZER_NULLS_FIELDS
ML_SYNC_ON_FIELD_TO_GUARD_CHANGING_THAT_FIELD
JCIP_FIELD_ISNT_FINAL_IN_IMMUTABLE_CLASS
CO_COMPARETO_RESULTS_MIN_VALUE
IMSE_DONT_CATCH_IMSE
QF_QUESTIONABLE_FOR_LOOP
MWN_MISMATCHED_NOTIFY
RV_RETURN_VALUE_OF_PUTIFABSENT_IGNORED
DMI_NONSERIALIZABLE_OBJECT_WRITTEN
BC_BAD_CAST_TO_ABSTRACT_COLLECTION
SW_SWING_METHODS_INVOKED_IN_SWING_THREAD
EC_ARRAY_AND_NONARRAY
NP_NULL_INSTANCEOF
MTIA_SUSPECT_SERVLET_INSTANCE_FIELD
EQ_ALWAYS_FALSE
INT_BAD_COMPARISON_WITH_INT_VALUE
NP_OPTIONAL_RETURN_NULL
BX_BOXING_IMMEDIATELY_UNBOXED
VA_FORMAT_STRING_EXPECTED_MESSAGE_FORMAT_SUPPLIED
BIT_IOR
FI_MISSING_SUPER_CALL
VO_VOLATILE_REFERENCE_TO_ARRAY
SF_DEAD_STORE_DUE_TO_SWITCH_FALLTHROUGH_TO_THROW
DMI_CONSTANT_DB_PASSWORD
NP_IMMEDIATE_DEREFERENCE_OF_READLINE
ML_SYNC_ON_UPDATED_FIELD
HE_USE_OF_UNHASHABLE_CLASS
FI_PUBLIC_SHOULD_BE_PROTECTED
DMI_BIGDECIMAL_CONSTRUCTED_FROM_DOUBLE
TQ_NEVER_VALUE_USED_WHERE_ALWAYS_REQUIRED
XSS_REQUEST_PARAMETER_TO_SERVLET_WRITER
NP_ALWAYS_NULL_EXCEPTION
DM_INVALID_MIN_MAX
NM_WRONG_PACKAGE
MS_SHOULD_BE_REFACTORED_TO_BE_FINAL
DC_PARTIALLY_CONSTRUCTED
WA_AWAIT_NOT_IN_LOOP
INT_VACUOUS_COMPARISON
RV_CHECK_COMPARETO_FOR_SPECIFIC_RETURN_VALUE
RCN_REDUNDANT_COMPARISON_OF_NULL_AND_NONNULL_VALUE
DL_SYNCHRONIZATION_ON_SHARED_CONSTANT
EQ_ALWAYS_TRUE
DMI_EMPTY_DB_PASSWORD
DMI_ENTRY_SETS_MAY_REUSE_ENTRY_OBJECTS
WL_USING_GETCLASS_RATHER_THAN_CLASS_LITERAL
FI_USELESS
VA_FORMAT_STRING_BAD_CONVERSION_FROM_ARRAY
SP_SPIN_ON_FIELD
BOA_BADLY_OVERRIDDEN_ADAPTER
MWN_MISMATCHED_WAIT
NP_CLONE_COULD_RETURN_NULL
CO_ABSTRACT_SELF
DMI_COLLECTIONS_SHOULD_NOT_CONTAIN_THEMSELVES
QBA_QUESTIONABLE_BOOLEAN_ASSIGNMENT
IC_INIT_CIRCULARITY
ICAST_INT_2_LONG_AS_INSTANT
FI_EXPLICIT_INVOCATION
ME_MUTABLE_ENUM_FIELD
RCN_REDUNDANT_COMPARISON_TWO_NULL_VALUES
HE_SIGNATURE_DECLARES_HASHING_OF_UNHASHABLE_CLASS
RS_READOBJECT_SYNC
SQL_BAD_PREPARED_STATEMENT_ACCESS
VA_FORMAT_STRING_BAD_CONVERSION_TO_BOOLEAN
EQ_ABSTRACT_SELF
NP_CLOSING_NULL
RV_REM_OF_HASHCODE
PT_RELATIVE_PATH_TRAVERSAL
TLW_TWO_LOCK_WAIT
INT_BAD_REM_BY_1
ICAST_QUESTIONABLE_UNSIGNED_RIGHT_SHIFT
VA_PRIMITIVE_ARRAY_PASSED_TO_OBJECT_VARARG
DM_USELESS_THREAD
DMI_DOH
IJU_ASSERT_METHOD_INVOKED_FROM_RUN_METHOD
BIT_IOR_OF_SIGNED_BYTE
DMI_THREAD_PASSED_WHERE_RUNNABLE_EXPECTED
SA_LOCAL_SELF_COMPUTATION
This method performs a nonsensical computation of a local variable with another reference to the same variable (e.g., x&x or x-x).
Because of the nature of the computation, this operation doesn't seem to make sense, and may indicate a typo or a logic error. Double check the computation.
IM_MULTIPLYING_RESULT_OF_IREM
The code multiplies the result of an integer remaining by an integer constant.
Be sure you don't have your operator precedence confused. For example i % 60 * 1000 is (i % 60) * 1000, not i % (60 * 1000).
MS_MUTABLE_HASHTABLE
Field is a mutable Hashtable (MS_MUTABLE_HASHTABLE)
A final static field references a Hashtable and can be accessed by malicious code or by accident from another package.
This code can freely modify the contents of the Hashtable.
logs/myLog.log
View File
pom.xml
+7 -12
View File
@@ -25,11 +25,11 @@
<version>1.2</version>
</dependency>
<dependency>
<groupId>edu.lu.uni</groupId>
<artifactId>simple-utils</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
<!--<dependency>-->
<!--<groupId>edu.lu.uni</groupId>-->
<!--<artifactId>simple-utils</artifactId>-->
<!--<version>0.0.1-SNAPSHOT</version>-->
<!--</dependency>-->
<!-- akka -->
<dependency>
@@ -69,18 +69,13 @@
<version>${logback.version}</version>
</dependency>
<!--<dependency>-->
<!--<groupId>edu.lu.uni.serval</groupId>-->
<!--<artifactId>SourceCodeParser</artifactId>-->
<!--<artifactId>GitTraveller</artifactId>-->
<!--<version>0.0.1-SNAPSHOT</version>-->
<!--</dependency>-->
<dependency>
<groupId>edu.lu.uni.serval</groupId>
<artifactId>GitTraveller</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
src/main/java/edu/lu/uni/serval/FixPattern/utils/CNodeMap.java
-101
View File
@@ -1,101 +0,0 @@
package edu.lu.uni.serval.FixPattern.utils;
import java.util.HashMap;
import java.util.Map;
public class CNodeMap {
public static Map<Integer, String> map;
static {
map = new HashMap<Integer, String>();
map.put(20100,"Left");
map.put(30100,"ActMisc");
map.put(40000,"FullType");
map.put(50000,"TypeQualifier");
map.put(60100,"BaseType");
map.put(60200,"Pointer");
map.put(60800,"EnumName");
map.put(60900,"StructUnionName");
map.put(61000,"TypeName");
map.put(70002,"SizeType");
map.put(70100,"IntType");
map.put(80001,"CChar");
map.put(80100,"Si");
map.put(90002,"UnSigned");
map.put(100003,"CInt");
map.put(200000,"ParamList");
map.put(210000,"DotsParameter");
map.put(220100,"ParameterType");
map.put(240100,"Ident");
map.put(240200,"Constant");
map.put(240400,"FunCall");
map.put(240500,"CondExpr");
map.put(240600,"Sequence");
map.put(240700,"Assignment");
map.put(240800,"Postfix");
map.put(240900,"Infix");
map.put(241000,"Unary");
map.put(241100,"Binary");
map.put(241200,"ArrayAccess");
map.put(241300,"RecordAccess");
map.put(241400,"RecordPtAccess");
map.put(241500,"SizeOfExpr");
map.put(241600,"SizeOfType");
map.put(241700,"Cast");
map.put(241900,"Constructor");
map.put(242000,"ParenExpr");
map.put(260300,"ExprStatement");
map.put(260800,"Asm");
map.put(270100,"Label");
map.put(270200,"Case");
map.put(270300,"CaseRange");
map.put(270400,"Default");
map.put(280001,"Continue");
map.put(280002,"Break");
map.put(280003,"Return");
map.put(280100,"Goto");
map.put(280200,"ReturnExpr");
map.put(290001,"None");
map.put(290100,"Some");
map.put(300100,"If");
map.put(300200,"Switch");
map.put(310100,"While");
map.put(310200,"DoWhile");
map.put(310300,"For");
map.put(310400,"MacroIteration");
map.put(330000,"Compound");
map.put(340000,"Storage");
map.put(350100,"DeclList");
map.put(350200,"MacroDecl");
map.put(360100,"InitExpr");
map.put(360200,"InitList");
map.put(360300,"InitDesignators");
map.put(370100,"DesignatorField");
map.put(370200,"DesignatorIndex");
map.put(370300,"DesignatorRange");
map.put(380000,"Definition");
map.put(400100,"Define");
map.put(400200,"Include");
map.put(400400,"OtherDirective");
map.put(410001,"DefineVar");
map.put(410002,"Undef");
map.put(410100,"DefineFunc");
map.put(420001,"DefineEmpty");
map.put(420100,"DefineExpr");
map.put(420200,"DefineStmt");
map.put(420400,"DefineDoWhileZero");
map.put(420600,"DefineInit");
map.put(440100,"IfdefDirective");
map.put(450100,"Declaration");
map.put(450200,"Definition");
map.put(450300,"CppTop");
map.put(450400,"IfdefTop");
map.put(450700,"NotParsedCorrectly");
map.put(450800,"FinalDef");
map.put(460000,"Program");
map.put(470000,"GenericList");
map.put(480000,"GenericString");
map.put(490100,"IfToken");
}
}
src/main/java/edu/lu/uni/serval/FixPattern/utils/EDiff.java
+98
View File
@@ -0,0 +1,98 @@
package edu.lu.uni.serval.FixPattern.utils;
import com.github.gumtreediff.actions.model.*;
import com.github.gumtreediff.tree.ITree;
import com.github.gumtreediff.tree.TreeContext;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import java.io.IOException;
import java.util.List;
/**
* Created by anilkoyuncu on 17/09/2018.
*/
public class EDiff {
private static Logger log = LoggerFactory.getLogger(EDiff.class);
public static ITree getSimpliedTree(String fn, JedisPool outerPool) {
ITree tree = null;
Jedis inner = null;
try {
inner = outerPool.getResource();
while (!inner.ping().equals("PONG")){
log.info("wait");
}
inner.select(1);
String dist2load = inner.get(fn);
inner.select(0);
String s = inner.get(dist2load);
HierarchicalActionSet actionSet = (HierarchicalActionSet) EDiffHelper.fromString(s);
ITree parent = null;
ITree children =null;
TreeContext tc = new TreeContext();
tree = EDiffHelper.getASTTree(actionSet, parent, children,tc);
tree.setParent(null);
tc.validate();
} catch (IOException e) {
e.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
}finally {
if (inner != null) {
inner.close();
}
}
return tree;
}
public static ITree getActionTree(HierarchicalActionSet actionSet, ITree parent, ITree children,TreeContext tc){
int newType = 0;
Action action = actionSet.getAction();
if (action instanceof Update){
newType = 101;
}else if(action instanceof Insert){
newType =100;
}else if(action instanceof Move){
newType = 102;
}else if(action instanceof Delete){
newType=103;
}else{
new Exception("unknow action");
}
if(actionSet.getParent() == null){
//root
parent = tc.createTree(newType, "", null);
tc.setRoot(parent);
// parent = new Tree(newType,"");
}else{
children = tc.createTree(newType, "", null);
children.setParentAndUpdateChildren(parent);
// children = new Tree(newType,"");
// parent.addChild(children);
}
List<HierarchicalActionSet> subActions = actionSet.getSubActions();
if (subActions.size() != 0){
for (HierarchicalActionSet subAction : subActions) {
if(actionSet.getParent() == null){
children = parent;
}
getActionTree(subAction,children,null,tc);
}
}
return parent;
}
}
src/main/java/edu/lu/uni/serval/FixPattern/utils/EDiffHelper.java
+88
View File
@@ -0,0 +1,88 @@
package edu.lu.uni.serval.FixPattern.utils;
import com.github.gumtreediff.tree.ITree;
import com.github.gumtreediff.tree.TreeContext;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
import java.util.Base64;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.stream.Collectors;
/**
* Created by anilkoyuncu on 17/09/2018.
*/
public class EDiffHelper {
private static Logger log = LoggerFactory.getLogger(EDiffHelper.class);
/** Read the object from Base64 string. */
public static Object fromString( String s ) throws IOException,
ClassNotFoundException {
byte [] data = Base64.getDecoder().decode( s );
ObjectInputStream ois = new ObjectInputStream(
new ByteArrayInputStream( data ) );
Object o = ois.readObject();
ois.close();
return o;
}
/** Write the object to a Base64 string. */
public static String toString( Serializable o ) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream( baos );
oos.writeObject( o );
oos.close();
return Base64.getEncoder().encodeToString(baos.toByteArray());
}
public static ITree getASTTree(HierarchicalActionSet actionSet, ITree parent, ITree children, TreeContext tc){
int newType = 0;
String astNodeType = actionSet.getAstNodeType();
List<Integer> keysByValue = getKeysByValue(ASTNodeMap.map, astNodeType);
if(keysByValue.size() != 1){
log.error("More than 1");
}
newType = keysByValue.get(0);
if(actionSet.getParent() == null){
//root
// parent = new Tree(newType,"");
parent = tc.createTree(newType, "", null);
tc.setRoot(parent);
}else{
// children = new Tree(newType,"");
// parent.addChild(children);
children = tc.createTree(newType, "", null);
children.setParentAndUpdateChildren(parent);
}
List<HierarchicalActionSet> subActions = actionSet.getSubActions();
if (subActions.size() != 0){
for (HierarchicalActionSet subAction : subActions) {
if(actionSet.getParent() == null){
children = parent;
}
getASTTree(subAction,children,null,tc);
}
}
return parent;
}
public static <T, E> List<T> getKeysByValue(Map<T, E> map, E value) {
return map.entrySet()
.stream()
.filter(entry -> Objects.equals(entry.getValue(), value))
.map(Map.Entry::getKey)
.collect(Collectors.toList());
}
}
src/main/java/edu/lu/uni/serval/FixPattern/utils/PoolBuilder.java
+33
View File
@@ -0,0 +1,33 @@
package edu.lu.uni.serval.FixPattern.utils;
import redis.clients.jedis.JedisPoolConfig;
import java.time.Duration;
/**
* Created by anilkoyuncu on 17/09/2018.
*/
public class PoolBuilder {
public static JedisPoolConfig getPoolConfig() {
return poolConfig;
}
static final JedisPoolConfig poolConfig = buildPoolConfig();
private static JedisPoolConfig buildPoolConfig() {
final JedisPoolConfig poolConfig = new JedisPoolConfig();
poolConfig.setMaxTotal(128);
poolConfig.setMaxIdle(128);
poolConfig.setMinIdle(16);
poolConfig.setTestOnBorrow(true);
poolConfig.setTestOnReturn(true);
poolConfig.setTestWhileIdle(true);
poolConfig.setMinEvictableIdleTimeMillis(Duration.ofMinutes(60).toMillis());
poolConfig.setTimeBetweenEvictionRunsMillis(Duration.ofHours(30).toMillis());
poolConfig.setNumTestsPerEvictionRun(3);
poolConfig.setBlockWhenExhausted(true);
return poolConfig;
}
}
src/main/java/edu/lu/uni/serval/FixPatternParser/CUCreator.java
+2 -6
View File
@@ -1,15 +1,11 @@
package edu.lu.uni.serval.FixPatternParser;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import org.eclipse.jdt.core.dom.AST;
import org.eclipse.jdt.core.dom.ASTParser;
import org.eclipse.jdt.core.dom.CompilationUnit;
import java.io.*;
/**
* Creator of a CompilationUnit.
*
src/main/java/edu/lu/uni/serval/FixPatternParser/Parser.java
+13 -454
View File
@@ -1,19 +1,6 @@
package edu.lu.uni.serval.FixPatternParser;
import java.io.File;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import com.github.gumtreediff.actions.model.Action;
import com.github.gumtreediff.actions.model.Move;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.gumtree.GumTreeComparer;
import edu.lu.uni.serval.gumtree.regroup.ActionFilter;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalRegrouper;
import edu.lu.uni.serval.gumtree.regroup.SimplifyTree;
/**
* Parse fix patterns with GumTree.
@@ -33,434 +20,6 @@ public abstract class Parser implements ParserInterface {
public abstract void parseFixPatterns(File prevFile, File revFile, File diffEntryFile,String project);
protected List<HierarchicalActionSet> parseChangedSourceCodeWithGumTree(File prevFile, File revFile) {
List<HierarchicalActionSet> actionSets = new ArrayList<>();
// GumTree results
List<Action> gumTreeResults = new GumTreeComparer().compareTwoFilesWithGumTree(prevFile, revFile);
// List<Action> gumTreeResults = new GumTreeComparer().compareTwoFilesWithGumTreeForCCode(prevFile, revFile);
if (gumTreeResults != null && gumTreeResults.size() > 0) {
// Regroup GumTre results.
List<HierarchicalActionSet> allActionSets = new HierarchicalRegrouper().regroupGumTreeResults(gumTreeResults);
// Filter out modified actions of changing method names, method parameters, variable names and field names in declaration part.
// TODO: variable effects range, sub-actions are these kinds of modification?
actionSets.addAll(new ActionFilter().filterOutUselessActions(allActionSets));
}
return actionSets;
}
protected List<Move> getFirstAndLastMoveAction(HierarchicalActionSet gumTreeResult) {
List<Move> firstAndLastMoveActions = new ArrayList<>();
List<HierarchicalActionSet> actions = gumTreeResult.getSubActions();
if (actions.size() == 0) {
return null;
}
Move firstMoveAction = null;
Move lastMoveAction = null;
while (actions.size() > 0) {
List<HierarchicalActionSet> subActions = new ArrayList<>();
for (HierarchicalActionSet action : actions) {
subActions.addAll(action.getSubActions());
if (action.toString().startsWith("MOV")) {
if (firstMoveAction == null) {
firstMoveAction = (Move) action.getAction();
lastMoveAction = (Move) action.getAction();
} else {
int startPosition = action.getStartPosition();
int length = action.getLength();
int startPositionFirst = firstMoveAction.getPosition();
int startPositionLast = lastMoveAction.getPosition();
int lengthLast = lastMoveAction.getNode().getLength();
if (startPosition < startPositionFirst || (startPosition == startPositionFirst && length > firstMoveAction.getLength())) {
firstMoveAction = (Move) action.getAction();
}
if ((startPosition + length) > (startPositionLast + lengthLast)) {
lastMoveAction = (Move) action.getAction();
}
}
}
}
actions.clear();
actions.addAll(subActions);
}
if (firstMoveAction == null) {
return null;
}
firstAndLastMoveActions.add(firstMoveAction);
firstAndLastMoveActions.add(lastMoveAction);
return firstAndLastMoveActions;
}
protected String getSemiSourceCodeEditScripts(HierarchicalActionSet actionSet) {
// TODO Auto-generated method stub
return null;
}
protected String getAbstractIdentifiersEditScripts(HierarchicalActionSet actionSet) {
// TODO Auto-generated method stub
return null;
}
protected String getRawTokenEditScripts(HierarchicalActionSet actionSet) {
// TODO Auto-generated method stub
return null;
}
protected int getEndPosition(List<ITree> children) {
int endPosition = 0;
for (ITree child : children) {
if (child.getLabel().endsWith("Body")) {
endPosition = child.getPos() - 1;
break;
}
}
return endPosition;
}
/**
* Get the AST node based edit script of patches in terms of breadth first.
*
* @param actionSet
* @return
*/
protected String getASTEditScriptsBreadthFirst(HierarchicalActionSet actionSet) {
String editScript = "";
List<HierarchicalActionSet> actionSets = new ArrayList<>();
actionSets.add(actionSet);
while (actionSets.size() != 0) {
List<HierarchicalActionSet> subSets = new ArrayList<>();
for (HierarchicalActionSet set : actionSets) {
subSets.addAll(set.getSubActions());
String actionStr = set.getActionString();
int index = actionStr.indexOf("@@");
String singleEdit = actionStr.substring(0, index).replace(" ", "");
singleEdit = handleSimpleNameNode(singleEdit, actionStr, index);
editScript += singleEdit + " ";
}
actionSets.clear();
actionSets.addAll(subSets);
}
return editScript;
}
private String handleSimpleNameNode(String singleEdit, String actionStr, int index) {
// TODO
if (singleEdit.endsWith("SimpleName")) {
actionStr = actionStr.substring(index + 2);
if (actionStr.startsWith("MethodName")) {
singleEdit = singleEdit.replace("SimpleName", "MethodName");
} else if (actionStr.startsWith("ClassName")) {
singleEdit = singleEdit.replace("SimpleName", "ClassName");
} else {
if (actionStr.startsWith("Name")) {
char c = actionStr.charAt(5);
if (Character.isUpperCase(c)) {
singleEdit = singleEdit.replace("SimpleName", "Name");
} else {
singleEdit = singleEdit.replace("SimpleName", "Variable");
}
} else {
singleEdit = singleEdit.replace("SimpleName", "Variable");
}
}
}
return singleEdit;
}
/**
* Get the AST node based edit script of patches in terms of deep first.
*
* @param actionSet
* @return
*/
protected String getASTEditScriptsDeepFirst(HierarchicalActionSet actionSet) {
String editScript = "";
String actionStr = actionSet.getActionString();
int index = actionStr.indexOf("@@");
String singleEdit = actionStr.substring(0, index).replace(" ", "");
singleEdit = handleSimpleNameNode(singleEdit, actionStr, index);
editScript = singleEdit + " ";
List<HierarchicalActionSet> subActionSets = actionSet.getSubActions();;
for (HierarchicalActionSet subActionSet : subActionSets) {
editScript += getASTEditScriptsDeepFirst(subActionSet);
}
return editScript;
}
/**
*
* @param hunkActionSets
* @param bugStartLine
* @param bugEndLine
* @param fixStartLine
* @param fixEndLine
* @return
*/
protected String getASTEditScriptsBreadthFirst(List<HierarchicalActionSet> hunkActionSets, int bugEndPosition, int fixEndPosition) {
String editScript = "";
for (HierarchicalActionSet hunkActionSet : hunkActionSets) {
editScript += getASTEditScriptsBreadthFirst(hunkActionSet, bugEndPosition, fixEndPosition);
}
return editScript;
}
private String getASTEditScriptsBreadthFirst(HierarchicalActionSet hunkActionSet, int bugEndPosition, int fixEndPosition) {
String editScript = "";
List<HierarchicalActionSet> actionSets = new ArrayList<>();
actionSets.add(hunkActionSet);
while (actionSets.size() != 0) {
List<HierarchicalActionSet> subSets = new ArrayList<>();
for (HierarchicalActionSet set : actionSets) {
int position = set.getAction().getPosition();
String actionStr = set.getActionString();
if (isOutofPosition(actionStr, position, bugEndPosition, fixEndPosition)) {
continue;
}
subSets.addAll(set.getSubActions());
int index = actionStr.indexOf("@@");
String singleEdit = actionStr.substring(0, index).replace(" ", "");
singleEdit = handleSimpleNameNode(singleEdit, actionStr, index);
editScript += singleEdit + " ";
}
actionSets.clear();
actionSets.addAll(subSets);
}
return editScript;
}
private boolean isOutofPosition(String actionStr, int actionPosition, int bugEndPosition, int fixEndPosition) {
if (actionStr.startsWith("INS")) {
if (actionPosition > fixEndPosition) {
return true;
}
} else if (!actionStr.startsWith("MOV") && actionPosition > bugEndPosition) {
return true;
}
return false;
}
protected String getASTEditScriptsDeepFirst(List<HierarchicalActionSet> hunkActionSets, int bugEndPosition, int fixEndPosition) {
StringBuilder editScript = new StringBuilder();
for (HierarchicalActionSet hunkActionSet : hunkActionSets) {
editScript.append(getASTEditScriptsDeepFirst2(hunkActionSet, bugEndPosition, fixEndPosition));
}
return editScript.toString();
}
private String getASTEditScriptsDeepFirst(HierarchicalActionSet actionSet, int bugEndPosition, int fixEndPosition) {
String editScripts = "";
String actionStr = actionSet.getActionString();
int index = actionStr.indexOf("@@");
String singleEdit = actionStr.substring(0, index).replace(" ", "");
singleEdit = handleSimpleNameNode(singleEdit, actionStr, index);
editScripts += singleEdit + " ";
for (HierarchicalActionSet subActionSet : actionSet.getSubActions()) {
int position = subActionSet.getAction().getPosition();
actionStr = subActionSet.getActionString();
if (isOutofPosition(actionStr, position, bugEndPosition, fixEndPosition)) {
continue;
}
editScripts += getASTEditScriptsDeepFirst(subActionSet, bugEndPosition, fixEndPosition);
}
return editScripts;
}
private String getASTEditScriptsDeepFirst2(HierarchicalActionSet actionSet, int bugEndPosition, int fixEndPosition) {
StringBuilder editScripts = new StringBuilder();
String actionStr = actionSet.getActionString();
int index = actionStr.indexOf("@@");
String singleEdit = actionStr.substring(0, index);
if (singleEdit.endsWith("Statement")) {
singleEdit = singleEdit + " " + singleEdit.substring(4, singleEdit.indexOf("Statement")).toLowerCase(Locale.ENGLISH);
} else {
singleEdit = handleSimpleNameNode2(singleEdit, actionStr, index, actionSet);
}
editScripts.append(singleEdit).append(" ");
for (HierarchicalActionSet subActionSet : actionSet.getSubActions()) {
int position = subActionSet.getAction().getPosition();
actionStr = subActionSet.getActionString();
if (isOutofPosition(actionStr, position, bugEndPosition, fixEndPosition)) {
continue;
}
editScripts.append(getASTEditScriptsDeepFirst2(subActionSet, bugEndPosition, fixEndPosition));
}
return editScripts.toString();
}
private String handleSimpleNameNode2(String singleEdit, String actionStr, int index, HierarchicalActionSet actionSet) {
if (singleEdit.endsWith("SimpleName")) {
actionStr = actionStr.substring(index + 2);
if (actionStr.startsWith("MethodName")) {
singleEdit = singleEdit.replace("SimpleName", "MethodName");
String methodName = actionStr.substring(actionStr.indexOf("MethodName:") + 11);
int index1 = methodName.indexOf(":");
int index2 = methodName.indexOf(" ");
index = (index1 < 0 || index1 > index2) ? index2 : index1;
methodName = methodName.substring(0, index);
singleEdit += " " + methodName;
} else if (actionStr.startsWith("ClassName")) {
singleEdit = singleEdit.replace("SimpleName", "ClassName");
String className = actionStr.substring(actionStr.indexOf("ClassName:") + 10);
int index1 = className.indexOf(" ");
index = index1 < 0 ? className.length() : index1;
className = className.substring(0, index);
singleEdit += " " + className;
} else {
if (actionStr.startsWith("Name")) {
char c = actionStr.charAt(5);
if (Character.isUpperCase(c)) {
singleEdit = singleEdit.replace("SimpleName", "Name");
String name = actionStr.substring(actionStr.indexOf("Name:") + 5);
int index1 = name.indexOf(" ");
index = index1 < 0 ? name.length() : index1;
name = name.substring(0, index);
singleEdit += " " + name;
} else {
singleEdit = singleEdit.replace("SimpleName", "Variable");
int index1 = actionStr.indexOf(" ");
index = index1 < 0 ? actionStr.length() : index1;
String var = actionStr.substring(0, index);
var = new SimplifyTree().canonicalVariableName(var, actionSet.getAction().getNode());
singleEdit += " " + var.replaceAll(" ", "");
}
} else {
singleEdit = singleEdit.replace("SimpleName", "Variable");
String var = actionStr.substring(0, (actionStr.indexOf(" ") < 0 ? actionStr.length() : actionStr.indexOf(" ")));
var = new SimplifyTree().canonicalVariableName(var, actionSet.getAction().getNode());
singleEdit += " " + var.replaceAll(" ", "");
}
}
} else {
if (actionSet.getSubActions() != null && actionSet.getSubActions().size() > 0) {
singleEdit += " " + actionSet.getAstNodeType()+ "exp";
} else {
if (singleEdit.endsWith("CharacterLiteral")) {
singleEdit += " charLiteral";
} else if (singleEdit.endsWith("NumberLiteral")) {
singleEdit += " numLiteral";
} else if (singleEdit.endsWith("StringLiteral")) {
singleEdit += " strLiteral";
} else if (singleEdit.endsWith("BooleanLiteral") || singleEdit.endsWith("NullLiteral") || singleEdit.endsWith("Operator") ||
singleEdit.endsWith("ThisExpression") || singleEdit.endsWith("TypeLiteral") || singleEdit.endsWith("Instanceof") ||
singleEdit.endsWith("New") || singleEdit.endsWith("WildcardType") || singleEdit.endsWith("SimpleType") ||
singleEdit.endsWith("QualifiedType") || singleEdit.endsWith("PrimitiveType") || singleEdit.endsWith("NameQualifiedType")) {
singleEdit += " " + actionSet.getNode().getLabel();
} else if (singleEdit.endsWith("SuperConstructorInvocation")) {
singleEdit += " super";
} else if (singleEdit.endsWith("ConstructorInvocation")) {
singleEdit += " this";
} else if (singleEdit.endsWith("SimpleName")) {
actionStr = actionStr.substring(index + 2);
if (actionStr.startsWith("MethodName")) {
singleEdit = singleEdit.replace("SimpleName", "MethodName");
String methodName = actionStr.substring(actionStr.indexOf("MethodName:") + 11);
int index1 = methodName.indexOf(":");
int index2 = methodName.indexOf(" ");
index = (index1 < 0 || index1 > index2) ? index2 : index1;
methodName = methodName.substring(0, index);
singleEdit += " " + methodName;
} else if (actionStr.startsWith("ClassName")) {
singleEdit = singleEdit.replace("SimpleName", "ClassName");
String className = actionStr.substring(actionStr.indexOf("ClassName:") + 10);
int index1 = className.indexOf(" ");
index = index1 < 0 ? className.length() : index1;
className = className.substring(0, index);
singleEdit += " " + className;
} else {
if (actionStr.startsWith("Name")) {
char c = actionStr.charAt(5);
if (Character.isUpperCase(c)) {
singleEdit = singleEdit.replace("SimpleName", "Name");
String name = actionStr.substring(actionStr.indexOf("Name:") + 5);
int index1 = name.indexOf(" ");
index = index1 < 0 ? name.length() : index1;
name = name.substring(0, index);
singleEdit += " " + name;
} else {
singleEdit = singleEdit.replace("SimpleName", "Variable");
int index1 = actionStr.indexOf(" ");
index = index1 < 0 ? actionStr.length() : index1;
String var = actionStr.substring(0, index);
var = new SimplifyTree().canonicalVariableName(var, actionSet.getAction().getNode());
singleEdit += " " + var.replaceAll(" ", "");
}
} else {
singleEdit = singleEdit.replace("SimpleName", "Variable");
String var = actionStr.substring(0, (actionStr.indexOf(" ") < 0 ? actionStr.length() : actionStr.indexOf(" ")));
var = new SimplifyTree().canonicalVariableName(var, actionSet.getAction().getNode());
singleEdit += " " + var.replaceAll(" ", "");
}
}
} else {
ITree nodeTree = actionSet.getNode();
if (nodeTree.getChildren().size() > 0) {
singleEdit += " " + actionSet.getAstNodeType()+ "exp";
} else {
singleEdit += " " + nodeTree.getLabel().replaceAll(" ", "");
}
}
}
}
return singleEdit;
}
protected String getAstEditScripts(List<HierarchicalActionSet> actionSets, int bugEndLine, int fixEndLine) {
String editScripts = "";
for (HierarchicalActionSet actionSet : actionSets) {
editScripts += getActionString(actionSet, bugEndLine, fixEndLine, "");
}
return editScripts;
}
private String getActionString(HierarchicalActionSet actionSet, int bugEndPosition, int fixEndPosition, String hierarchicalLabel) {
String editScripts = "";
String actionStr = actionSet.getActionString();
int index = actionStr.indexOf("@@");
String singleEdit = actionStr.substring(0, index);
singleEdit = handleSimpleNameNode(singleEdit, actionStr, index);
editScripts += hierarchicalLabel + singleEdit + "\n";
for (HierarchicalActionSet subActionSet : actionSet.getSubActions()) {
int position = subActionSet.getAction().getPosition();
actionStr = subActionSet.getActionString();
if (isOutofPosition(actionStr, position, bugEndPosition, fixEndPosition)) {
continue;
}
editScripts += getActionString(subActionSet, bugEndPosition, fixEndPosition, hierarchicalLabel + "---");
}
return editScripts;
}
protected void clearITree(HierarchicalActionSet actionSet) {
actionSet.getAction().setNode(null);
for (HierarchicalActionSet subActionSet : actionSet.getSubActions()) {
clearITree(subActionSet);
}
}
@Override
public String getAstEditScripts() {
@@ -472,10 +31,10 @@ public abstract class Parser implements ParserInterface {
return patchesSourceCode;
}
@Override
public String getBuggyTrees() {
return buggyTrees;
}
// @Override
// public String getBuggyTrees() {
// return buggyTrees;
// }
@Override
public String getSizes() {
@@ -487,13 +46,13 @@ public abstract class Parser implements ParserInterface {
return tokensOfSourceCode;
}
@Override
public String getOriginalTree() {
return originalTree;
}
@Override
public String getActionSets() {
return actionSets;
}
// @Override
// public String getOriginalTree() {
// return originalTree;
// }
//
// @Override
// public String getActionSets() {
// return actionSets;
// }
}
src/main/java/edu/lu/uni/serval/FixPatternParser/ParserInterface.java
+4 -6
View File
@@ -1,23 +1,21 @@
package edu.lu.uni.serval.FixPatternParser;
import java.io.File;
public interface ParserInterface {
public void parseFixPatterns(File prevFile, File revFile, File diffEntryFile);
// public void parseFixPatterns(File prevFile, File revFile, File diffEntryFile);
public String getAstEditScripts();
public String getPatchesSourceCode();
public String getBuggyTrees();
// public String getBuggyTrees();
public String getSizes();
public String getTokensOfSourceCode();
public String getOriginalTree();
// public String getOriginalTree();
public String getActionSets();
// public String getActionSets();
}
src/main/java/edu/lu/uni/serval/FixPatternParser/Tokenizer.java
-75
View File
@@ -1,75 +0,0 @@
package edu.lu.uni.serval.FixPatternParser;
import java.util.List;
import edu.lu.uni.serval.gumtree.regroup.SimpleTree;
public class Tokenizer {
public static String getTokensDeepFirst(SimpleTree simpleTree) {
String tokens = "";
List<SimpleTree> children = simpleTree.getChildren();
String astNodeType = simpleTree.getNodeType();
if (children.isEmpty()) { // BreakStatement, ContinueStatement, ReturnStatement, TryStatement
if (astNodeType.endsWith("Statement")) {
String label = astNodeType;
label = label.substring(0, label.lastIndexOf("S")).toLowerCase();
tokens += astNodeType + " " + label + " ";
} else if ("SuperConstructorInvocation".equals(astNodeType)) {
tokens += astNodeType + " super ";
} else if ("ConstructorInvocation".equals(astNodeType)) {
tokens += astNodeType + " this ";
} else if ("StringLiteral".equals(astNodeType)) {
tokens += astNodeType + " stringLiteral ";
} else if ("CharacterLiteral".equals(astNodeType)) {
tokens += astNodeType + " charLiteral ";
} else if ("ArrayInitializer".equals(astNodeType)) {
tokens += astNodeType + " arrayInitializer ";
} else if ("LambdaExpression".equals(astNodeType)) {
tokens += astNodeType + " lambda ";
} else if ("NumberLiteral".equals(astNodeType)) {
tokens += astNodeType + " numberLiteral ";
} else {
tokens += astNodeType + " " + simpleTree.getLabel() + " ";
}
} else {
if ("AssertStatement".equals(astNodeType) || "DoStatement".equals(astNodeType)
|| "ForStatement".equals(astNodeType) || "IfStatement".equals(astNodeType)
|| "ReturnStatement".equals(astNodeType) || "SwitchStatement".equals(astNodeType)
|| "SynchronizedStatement".equals(astNodeType) || "ThrowStatement".equals(astNodeType)
|| "TryStatement".equals(astNodeType) || "WhileStatement".equals(astNodeType)) {
String label = astNodeType;
label = label.substring(0, label.lastIndexOf("S")).toLowerCase();
tokens += astNodeType + " " + label + " ";
} else if ("EnhancedForStatement".equals(astNodeType)) {
tokens += astNodeType + " " + "for ";
} else if ("CatchClause".equals(astNodeType)) {
tokens += astNodeType + " " + "catch ";
} else if ("SwitchCase".equals(astNodeType)) {
tokens += astNodeType + " case ";
} else if ("SuperConstructorInvocation".equals(astNodeType)) {
tokens += astNodeType + " super ";
} else if ("ConstructorInvocation".equals(astNodeType)) {
tokens += astNodeType + " this ";
} else if ("FinallyBody".equals(astNodeType)) {
tokens += astNodeType + " finally ";
} else if ("LabeledStatement".equals(astNodeType)) {
tokens += "LabeledStatement " + simpleTree.getLabel();
} else if ("SuperMethodInvocation".equals(astNodeType)) {
tokens += astNodeType + " super ";
} else if ("MethodName".equals(astNodeType)) {
tokens += "MethodName " + simpleTree.getLabel() + " ";
}
if ("ArrayInitializer".equals(astNodeType)) {
tokens += astNodeType + " arrayInitializer ";
} else {
for (SimpleTree child : children) {
tokens += getTokensDeepFirst(child);
}
}
}
return tokens;
}
}
src/main/java/edu/lu/uni/serval/FixPatternParser/patch/CommitPatchHunkParser.java
-113
View File
@@ -1,113 +0,0 @@
package edu.lu.uni.serval.FixPatternParser.patch;
import java.io.File;
import java.util.ArrayList;
import java.util.List;
import edu.lu.uni.serval.FixPatternParser.Tokenizer;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.diffentry.DiffEntryHunk;
import edu.lu.uni.serval.diffentry.DiffEntryReader;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
import edu.lu.uni.serval.gumtree.regroup.HunkActionFilter;
import edu.lu.uni.serval.gumtree.regroup.HunkFixPattern;
import edu.lu.uni.serval.gumtree.regroup.SimpleTree;
import edu.lu.uni.serval.gumtree.regroup.SimplifyTree;
/**
* Parse fix patterns with GumTree.
*
* Multiple statements bugs.
*
* @author kui.liu
*
*/
public class CommitPatchHunkParser extends CommitPatchParser {
@Override
public void parseFixPatterns(File prevFile, File revFile, File diffEntryFile) {
// GumTree results
List<HierarchicalActionSet> actionSets = parseChangedSourceCodeWithGumTree(prevFile, revFile);
if (actionSets.size() > 0) {
// DiffEntry size: filter out big hunks.
List<DiffEntryHunk> diffentryHunks = new DiffEntryReader().readHunks(diffEntryFile);
//Filter out the modify actions, which are not in the DiffEntry hunks.
HunkActionFilter hunkFilter = new HunkActionFilter();
List<HunkFixPattern> allHunkFixPatterns = hunkFilter.filterActionsByDiffEntryHunk2(diffentryHunks, actionSets, revFile, prevFile);
for (HunkFixPattern hunkFixPattern : allHunkFixPatterns) {
// Range of buggy source code
int startLine = 0;
int endLine = 0;
// Range of fixing source code
int startLine2 = 0;
int endLine2 = 0;
/*
* Convert the ITree of buggy code to a simple tree.
* It will be used to compute the similarity.
*/
List<HierarchicalActionSet> hunkActionSets = hunkFixPattern.getHunkActionSets();
SimpleTree simpleTree = new SimpleTree();
simpleTree.setLabel("Block");
simpleTree.setNodeType("Block");
List<SimpleTree> children = new ArrayList<>();
String astEditScripts = "";
for (HierarchicalActionSet hunkActionSet : hunkActionSets) {
SimplifyTree abstractIdentifier = new SimplifyTree();
abstractIdentifier.abstractTree(hunkActionSet);
SimpleTree simpleT = hunkActionSet.getSimpleTree();
if (simpleT == null) { // Failed to get the simple tree for INS actions.
continue;
}
children.add(simpleT);
/**
* Select edit scripts for deep learning.
* Edit scripts will be used to mine common fix patterns.
*/
// 1. First level: AST node type.
astEditScripts += getASTEditScriptsBreadthFirst(hunkActionSet);
// 2. source code: raw tokens
// 3. abstract identifiers:
// 4. semi-source code:
if (startLine == 0) {
startLine = hunkActionSet.getBugStartLineNum();
endLine = hunkActionSet.getBugEndLineNum();
startLine2 = hunkActionSet.getFixStartLineNum();
endLine2 = hunkActionSet.getFixEndLineNum();
} else {
if (startLine > hunkActionSet.getBugStartLineNum()) startLine = hunkActionSet.getBugStartLineNum();
if (startLine2 > hunkActionSet.getFixStartLineNum()) startLine2 = hunkActionSet.getFixStartLineNum();
if (endLine < hunkActionSet.getBugEndLineNum()) endLine = hunkActionSet.getBugEndLineNum();
if (endLine2 < hunkActionSet.getFixEndLineNum()) endLine2 = hunkActionSet.getFixEndLineNum();
}
}
if (children.size() == 0) continue;
if (endLine - startLine >= Configuration.HUNK_SIZE - 2 || endLine2 - startLine2 >= Configuration.HUNK_SIZE - 2 ) continue;
simpleTree.setChildren(children);
simpleTree.setParent(null);
// Source Code of patches.
String patchSourceCode = getPatchSourceCode(hunkFixPattern.getHunk(), startLine, endLine, startLine2, endLine2);
if ("".equals(patchSourceCode)) continue;
this.patchesSourceCode += "PATCH###\n" + patchSourceCode + "\n";
int size = astEditScripts.split(" ").length;
this.sizes += size + "\n";
this.astEditScripts += astEditScripts + "\n";
// this.buggyTrees += Configuration.BUGGY_TREE_TOKEN + "\n" + simpleTree.toString() + "\n";
this.tokensOfSourceCode += Tokenizer.getTokensDeepFirst(simpleTree).trim() + "\n";
// this.actionSets += Configuration.BUGGY_TREE_TOKEN + "\n" + readActionSet(actionSet, "") + "\n";
// this.originalTree += Configuration.BUGGY_TREE_TOKEN + "\n" + actionSet.getOriginalTree().toString() + "\n";
}
}
}
}
src/main/java/edu/lu/uni/serval/FixPatternParser/patch/CommitPatchParser.java
-105
View File
@@ -1,105 +0,0 @@
package edu.lu.uni.serval.FixPatternParser.patch;
import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.StringReader;
import java.util.List;
import edu.lu.uni.serval.FixPatternParser.Parser;
import edu.lu.uni.serval.diffentry.DiffEntryHunk;
/**
* Parse fix patterns with GumTree.
*
* @author kui.liu
*
*/
public class CommitPatchParser extends Parser{
@Override
public void parseFixPatterns(File prevFile, File revFile, File diffEntryFile) {
}
protected DiffEntryHunk matchHunk(int startLine, int endLine, int startLine2, int endLine2, String actionStr, List<DiffEntryHunk> hunks) {
for (DiffEntryHunk hunk : hunks) {
int bugStartLine = hunk.getBugLineStartNum();
int bugRange = hunk.getBugRange();
int fixStartLine = hunk.getFixLineStartNum();
int fixRange = hunk.getFixRange();
if (actionStr.startsWith("INS")) {
if (fixStartLine + fixRange < startLine2) {
continue;
}
if (endLine2 < fixStartLine ) {
return null;
}
return hunk;
} else {
if (bugStartLine + bugRange < startLine) {
continue;
}
if (endLine < bugStartLine ) {
return null;
}
return hunk;
}
}
return null;
}
protected String getPatchSourceCode(DiffEntryHunk hunk, int startLineNum, int endLineNum, int startLineNum2, int endLineNum2) {
String sourceCode = hunk.getHunk();
int bugStartLine = hunk.getBugLineStartNum();
int fixStartLine = hunk.getFixLineStartNum();
String buggyStatements = "";
String fixedStatements = "";
BufferedReader reader = null;
try {
reader = new BufferedReader(new StringReader(sourceCode));
String line = null;
int bugLines = 0;
int fixLines = 0;
int contextLines = 0; // counter of non-buggy code line.
while ((line = reader.readLine()) != null) {
int bugLineIndex = bugLines + contextLines;
int fixLineIndex = fixLines + contextLines;
if (line.startsWith("-")) {
if (bugStartLine + bugLineIndex >= startLineNum && bugStartLine + bugLineIndex <= endLineNum) {
buggyStatements += line + "\n";
}
bugLines ++;
} else if (line.startsWith("+")) {
if (fixStartLine + fixLineIndex >= startLineNum2 && fixStartLine + fixLineIndex <= endLineNum2) {
fixedStatements += line + "\n";
}
fixLines ++;
} else {
contextLines ++;
}
if (bugStartLine + bugLineIndex > endLineNum && fixStartLine + fixLineIndex > endLineNum2) {
break;
}
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (reader != null) {
reader.close();
reader = null;
}
} catch (IOException e) {
e.printStackTrace();
}
}
return buggyStatements + fixedStatements;
}
@Override
public void parseFixPatterns(File prevFile, File revFile, File diffEntryFile, String project) {
}
}
src/main/java/edu/lu/uni/serval/FixPatternParser/patch/CommitPatchSingleStatementParser.java
-158
View File
@@ -1,158 +0,0 @@
package edu.lu.uni.serval.FixPatternParser.patch;
import java.io.File;
import java.util.List;
import org.eclipse.jdt.core.dom.CompilationUnit;
import com.github.gumtreediff.actions.model.Move;
import com.github.gumtreediff.actions.model.Update;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.FixPattern.utils.Checker;
import edu.lu.uni.serval.FixPatternParser.CUCreator;
import edu.lu.uni.serval.FixPatternParser.Tokenizer;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.diffentry.DiffEntryHunk;
import edu.lu.uni.serval.diffentry.DiffEntryReader;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
import edu.lu.uni.serval.gumtree.regroup.SimpleTree;
import edu.lu.uni.serval.gumtree.regroup.SimplifyTree;
/**
* Parse fix patterns with GumTree.
*
* Single Statement bugs.
*
* @author kui.liu
*
*/
public class CommitPatchSingleStatementParser extends CommitPatchParser {
@Override
public void parseFixPatterns(File prevFile, File revFile, File diffEntryFile) {
// GumTree results
List<HierarchicalActionSet> actionSets = parseChangedSourceCodeWithGumTree(prevFile, revFile);
if (actionSets.size() > 0) {
// DiffEntry Hunks: filter out big hunks.
List<DiffEntryHunk> diffentryHunks = new DiffEntryReader().readHunks(diffEntryFile);
CUCreator cuCreator = new CUCreator();
CompilationUnit prevUnit = cuCreator.createCompilationUnit(prevFile);
CompilationUnit revUnit = cuCreator.createCompilationUnit(revFile);
if (prevUnit == null || revUnit == null) {
return;
}
for (HierarchicalActionSet actionSet : actionSets) {
// position of buggy statements
int startPosition = 0;
int endPosition = 0;
// position of fixed statements
int startPosition2 = 0;
int endPosition2 = 0;
String actionStr = actionSet.getActionString();
String astNodeType = actionSet.getAstNodeType();
if (actionStr.startsWith("INS")) {
startPosition2 = actionSet.getStartPosition();
endPosition2 = startPosition2 + actionSet.getLength();
List<Move> firstAndLastMov = getFirstAndLastMoveAction(actionSet);
if (firstAndLastMov != null) {
startPosition = firstAndLastMov.get(0).getNode().getPos();
ITree lastTree = firstAndLastMov.get(1).getNode();
endPosition = lastTree.getPos() + lastTree.getLength();
} else { // Ignore the pure insert actions without any move actions.
continue;
}
} else if (actionStr.startsWith("UPD")) {
startPosition = actionSet.getStartPosition();
endPosition = startPosition + actionSet.getLength();
Update update = (Update) actionSet.getAction();
ITree newNode = update.getNewNode();
startPosition2 = newNode.getPos();
endPosition2 = startPosition2 + newNode.getLength();
if (Checker.containsBodyBlock(astNodeType)) {
List<ITree> children = update.getNode().getChildren();
endPosition = getEndPosition(children);
List<ITree> newChildren = newNode.getChildren();
endPosition2 = getEndPosition(newChildren);
if (endPosition == 0) {
endPosition = startPosition + actionSet.getLength();
}
if (endPosition2 == 0) {
endPosition2 = startPosition2 + newNode.getLength();
}
}
} else {// DEL actions and MOV actions: we don't need these actions, as for now.
continue;
}
if (startPosition == 0 || startPosition2 == 0) {
continue;
}
// Get line numbers.
int startLine = prevUnit.getLineNumber(startPosition);
int endLine = prevUnit.getLineNumber(endPosition);
int startLine2 = revUnit.getLineNumber(startPosition2);
int endLine2 = revUnit.getLineNumber(endPosition2);
if (endLine - startLine >= Configuration.HUNK_SIZE - 2 || endLine2 - startLine2 >= Configuration.HUNK_SIZE - 2)
continue;
// Filter out the modify actions, which are not in the DiffEntry hunks.
DiffEntryHunk hunk = matchHunk(startLine, endLine, startLine2, endLine2, actionStr, diffentryHunks);
if (hunk == null) {
continue;
}
/*
* Convert the ITree of buggy code to a simple tree. It will be
* used to compute the similarity.
*/
SimplifyTree abstractIdentifier = new SimplifyTree();
abstractIdentifier.abstractTree(actionSet);
SimpleTree simpleTree = actionSet.getSimpleTree();
if (simpleTree == null) { // Failed to get the simple tree for INS actions.
continue;
}
/**
* Select edit scripts for deep learning. Edit scripts will be
* used to mine common fix patterns.
*/
// 1. First level: AST node type.
String astEditScripts = getASTEditScriptsBreadthFirst(actionSet);
int size = astEditScripts.split(" ").length;
if (size < 2) {
// System.out.println(actionSet);
continue;
}
// Source Code of patches.
String patchSourceCode = getPatchSourceCode(hunk, startLine, endLine, startLine2, endLine2);
if ("".equals(patchSourceCode))
continue;
this.patchesSourceCode += Configuration.PATCH_SIGNAL + "\n" + patchSourceCode + "\n";
this.sizes += size + "\n";
this.astEditScripts += astEditScripts + "\n";
// // 2. source code: raw tokens
// String rawTokenEditScripts = getRawTokenEditScripts(actionSet);
// // 3. abstract identifiers:
// String abstractIdentifiersEditScripts = getAbstractIdentifiersEditScripts(actionSet);
// // 4. semi-source code:
// String semiSourceCodeEditScripts = getSemiSourceCodeEditScripts(actionSet);
// this.buggyTrees += Configuration.BUGGY_TREE_TOKEN + "\n" +
// simpleTree.toString() + "\n";
this.tokensOfSourceCode += Tokenizer.getTokensDeepFirst(simpleTree).trim() + "\n";
// this.actionSets += Configuration.BUGGY_TREE_TOKEN + "\n" +
// readActionSet(actionSet, "") + "\n";
// this.originalTree += Configuration.BUGGY_TREE_TOKEN + "\n" +
// actionSet.getOriginalTree().toString() + "\n";
}
actionSets.clear();
}
}
}
src/main/java/edu/lu/uni/serval/FixPatternParser/violations/CSVUtils.java
-62
View File
@@ -1,62 +0,0 @@
package edu.lu.uni.serval.FixPatternParser.violations;
import java.io.IOException;
import java.io.Writer;
import java.util.List;
/**
* Created by anilkoyuncu on 19/03/2018.
*/
public class CSVUtils {
private static final char DEFAULT_SEPARATOR = ',';
public static void writeLine(Writer w, List<String> values) throws IOException {
writeLine(w, values, DEFAULT_SEPARATOR, ' ');
}
public static void writeLine(Writer w, List<String> values, char separators) throws IOException {
writeLine(w, values, separators, ' ');
}
//https://tools.ietf.org/html/rfc4180
private static String followCVSformat(String value) {
String result = value;
if (result.contains("\"")) {
result = result.replace("\"", "\"\"");
}
return result;
}
public static void writeLine(Writer w, List<String> values, char separators, char customQuote) throws IOException {
boolean first = true;
//default customQuote is empty
if (separators == ' ') {
separators = DEFAULT_SEPARATOR;
}
StringBuilder sb = new StringBuilder();
for (String value : values) {
if (!first) {
sb.append(separators);
}
if (customQuote == ' ') {
sb.append(followCVSformat(value));
} else {
sb.append(customQuote).append(followCVSformat(value)).append(customQuote);
}
first = false;
}
sb.append("\n");
w.append(sb.toString());
}
}
src/main/java/edu/lu/uni/serval/FixPatternParser/violations/Consumer.java
-226
View File
@@ -1,226 +0,0 @@
package edu.lu.uni.serval.FixPatternParser.violations;
import com.github.gumtreediff.actions.ActionGenerator;
import com.github.gumtreediff.actions.model.Action;
import com.github.gumtreediff.matchers.Matcher;
import com.github.gumtreediff.matchers.Matchers;
import com.github.gumtreediff.tree.ITree;
import com.github.gumtreediff.tree.Tree;
import com.rabbitmq.client.*;
import edu.lu.uni.serval.FixPattern.utils.ASTNodeMap;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;
import java.io.*;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.concurrent.TimeoutException;
import java.util.stream.Collectors;
/**
* Created by anilkoyuncu on 03/04/2018.
*/
public class Consumer {
private static Logger log = LoggerFactory.getLogger(Consumer.class);
private static String inputPath = "/Users/anilkoyuncu/bugStudy/dataset/GumTreeOutput2";
public static void main(String[] args) {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("10.240.5.10");
// factory.setVirtualHost("treeCompare");
factory.setPort(5672);
factory.setUsername("anil");
factory.setPassword("changeme2018");
factory.setConnectionTimeout(1000);
Connection connection = null;
Channel channel = null;
try {
connection = factory.newConnection();
channel = connection.createChannel();
channel.queueDeclare("tree_queue", false, false, false, null);
boolean autoAck = false;
Channel finalChannel = channel;
channel.basicConsume("tree_queue", autoAck,"myConsumerTag",
new DefaultConsumer(finalChannel) {
@Override
public void handleDelivery(
String consumerTag,
Envelope envelope,
AMQP.BasicProperties properties,
byte[] body) throws IOException {
String message = new String(body, "UTF-8");
String[] split = message.split(",");
String key = split[0];
String firstV = split[1];
String secondV = split[2];
try{
calculateSimi(firstV,secondV,key);
}catch (Exception e){
finalChannel.basicNack(envelope.getDeliveryTag(),false,false);
}
finalChannel.basicAck(envelope.getDeliveryTag(),false);
// process the message
}
});
// channel.basicConsume("tree_queue", true, consumer);
} catch (IOException e) {
e.printStackTrace();
} catch (TimeoutException e) {
e.printStackTrace();
}
}
public static void calculateSimi(String firstVal,String secondVal, String key){
String[] split = key.split("_");
String i = split[1];
String j = split[2];
String[] firstValueSplit = firstVal.split("GumTreeOutput2");
String[] secondValueSplit = secondVal.split("GumTreeOutput2");
String firstValue;
if (firstValueSplit.length == 1) {
firstValue = inputPath + firstValueSplit[0];
} else {
firstValue = inputPath + firstValueSplit[1];
}
String secondValue;
if (secondValueSplit.length == 1) {
secondValue = inputPath + secondValueSplit[0];
} else {
secondValue = inputPath + secondValueSplit[1];
}
ITree oldTree = getSimpliedTree(firstValue);
ITree newTree = getSimpliedTree(secondValue);
Matcher m = Matchers.getInstance().getMatcher(oldTree, newTree);
m.match();
ActionGenerator ag = new ActionGenerator(oldTree, newTree, m.getMappings());
ag.generate();
List<Action> actions = ag.getActions();
double chawatheSimilarity1 = m.chawatheSimilarity(oldTree, newTree);
String chawatheSimilarity = String.format("%1.2f", chawatheSimilarity1);
double diceSimilarity1 = m.diceSimilarity(oldTree, newTree);
String diceSimilarity = String.format("%1.2f", diceSimilarity1);
double jaccardSimilarity1 = m.jaccardSimilarity(oldTree, newTree);
String jaccardSimilarity = String.format("%1.2f", jaccardSimilarity1);
String editDistance = String.valueOf(actions.size());
String result = firstVal + "," + secondVal + "," + chawatheSimilarity + "," + diceSimilarity + "," + jaccardSimilarity + "," + editDistance;
if (((Double) chawatheSimilarity1).equals(1.0) || ((Double) diceSimilarity1).equals(1.0)
|| ((Double) jaccardSimilarity1).equals(1.0) || actions.size() == 0) {
String matchKey = "match_" + (String.valueOf(i)) + "_" + String.valueOf(j);
log.info(matchKey);
// JedisPool jedisPool = new JedisPool(new JedisPoolConfig(), "127.0.0.1", Integer.valueOf(port), 20000000);
// try (Jedis outer = jedisPool.getResource()) {
// outer.select(1);
// outer.set(matchKey, result);
// }
}
// try (Jedis jedis = pool.getResource()) {
// jedis.del("pair_"+ (String.valueOf(i)) + "_" + String.valueOf(j));
// }
}
public static ITree getSimpliedTree(String fn) {
HierarchicalActionSet actionSet = null;
try {
FileInputStream fi = new FileInputStream(new File(fn));
ObjectInputStream oi = new ObjectInputStream(fi);
actionSet = (HierarchicalActionSet) oi.readObject();
oi.close();
fi.close();
} catch (FileNotFoundException e) {
log.error("File not found");
e.printStackTrace();
} catch (IOException e) {
log.error("Error initializing stream");
e.printStackTrace();
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
ITree parent = null;
ITree children =null;
ITree tree = getASTTree(actionSet, parent, children);
tree.setParent(null);
return tree;
}
public static <T, E> List<T> getKeysByValue(Map<T, E> map, E value) {
return map.entrySet()
.stream()
.filter(entry -> Objects.equals(entry.getValue(), value))
.map(Map.Entry::getKey)
.collect(Collectors.toList());
}
public static ITree getASTTree(HierarchicalActionSet actionSet, ITree parent, ITree children){
int newType = 0;
String astNodeType = actionSet.getAstNodeType();
List<Integer> keysByValue = getKeysByValue(ASTNodeMap.map, astNodeType);
if(keysByValue.size() != 1){
log.error("Birden cok astnodemapmapping");
}
newType = keysByValue.get(0);
if(actionSet.getParent() == null){
//root
parent = new Tree(newType,"");
}else{
children = new Tree(newType,"");
parent.addChild(children);
}
List<HierarchicalActionSet> subActions = actionSet.getSubActions();
if (subActions.size() != 0){
for (HierarchicalActionSet subAction : subActions) {
if(actionSet.getParent() == null){
children = parent;
}
getASTTree(subAction,children,null);
}
}
return parent;
}
}
src/main/java/edu/lu/uni/serval/FixPatternParser/violations/FieldViolation.java
-121
View File
@@ -1,121 +0,0 @@
package edu.lu.uni.serval.FixPatternParser.violations;
public enum FieldViolation {
UWF_FIELD_NOT_INITIALIZED_IN_CONSTRUCTOR,
ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD,
LI_LAZY_INIT_STATIC,
URF_UNREAD_FIELD,
MS_SHOULD_BE_FINAL,
URF_UNREAD_PUBLIC_OR_PROTECTED_FIELD,
UWF_UNWRITTEN_FIELD,
MS_PKGPROTECT,
SS_SHOULD_BE_STATIC,
NP_UNWRITTEN_FIELD,
UWF_UNWRITTEN_PUBLIC_OR_PROTECTED_FIELD,
SE_BAD_FIELD,
UUF_UNUSED_FIELD,
SE_BAD_FIELD_STORE,
MS_MUTABLE_ARRAY,
NP_UNWRITTEN_PUBLIC_OR_PROTECTED_FIELD,
}
enum ClassViolation{
SIC_INNER_SHOULD_BE_STATIC_ANON, //inner class. ClassInstanceCreation by overriding some methods.
SIC_INNER_SHOULD_BE_STATIC,
RI_REDUNDANT_INTERFACES,
SE_COMPARATOR_SHOULD_BE_SERIALIZABLE,
SE_NO_SERIALVERSIONID,
CI_CONFUSED_INHERITANCE,
SE_INNER_CLASS,
NM_CLASS_NOT_EXCEPTION,
}
enum MethodViolation {
NM_METHOD_NAMING_CONVENTION,
UPM_UNCALLED_PRIVATE_METHOD,
DE_MIGHT_IGNORE,
NM_CONFUSING,
CN_IMPLEMENTS_CLONE_BUT_NOT_CLONEABLE,
UC_USELESS_VOID_METHOD,
}
enum StmtViolation {
DLS_DEAD_LOCAL_STORE, //Assignment statement. false positives may be high.
//This instruction assigns a value to a local variable, but the value is not read or used in any subsequent instruction.
DM_BOXED_PRIMITIVE_FOR_PARSING,
DM_CONVERT_CASE,
DM_DEFAULT_ENCODING,
IS2_INCONSISTENT_SYNC,
NP_GUARANTEED_DEREF,
SIO_SUPERFLUOUS_INSTANCEOF,
NP_BOOLEAN_RETURN_NULL,
SF_SWITCH_FALLTHROUGH,
DP_DO_INSIDE_DO_PRIVILEGED,
DMI_INVOKING_TOSTRING_ON_ARRAY,
NP_NONNULL_PARAM_VIOLATION,
CN_IDIOM_NO_SUPER_CALL,
UL_UNRELEASED_LOCK_EXCEPTION_PATH,
RC_REF_COMPARISON_BAD_PRACTICE,
NP_ALWAYS_NULL,
NP_NULL_PARAM_DEREF,
GC_UNRELATED_TYPES,
DLS_DEAD_LOCAL_STORE_OF_NULL,
RpC_REPEATED_CONDITIONAL_TEST,
ODR_OPEN_DATABASE_RESOURCE_EXCEPTION_PATH,
DB_DUPLICATE_SWITCH_CLAUSES,
RV_RETURN_VALUE_IGNORED,
ES_COMPARING_PARAMETER_STRING_WITH_EQ,
RCN_REDUNDANT_NULLCHECK_OF_NULL_VALUE,
ICAST_IDIV_CAST_TO_DOUBLE,
BC_VACUOUS_INSTANCEOF,
EC_UNRELATED_TYPES,
ICAST_INTEGER_MULTIPLY_CAST_TO_LONG,
DM_STRING_TOSTRING,
DMI_HARDCODED_ABSOLUTE_FILENAME,
MS_MUTABLE_COLLECTION_PKGPROTECT,
NP_NULL_ON_SOME_PATH_EXCEPTION,
SQL_NONCONSTANT_STRING_PASSED_TO_EXECUTE,
EQ_COMPARETO_USE_OBJECT_EQUALS,
BX_UNBOXING_IMMEDIATELY_REBOXED,
HE_EQUALS_USE_HASHCODE,
DM_EXIT,
FE_FLOATING_POINT_EQUALITY,
RCN_REDUNDANT_NULLCHECK_OF_NONNULL_VALUE,
BC_UNCONFIRMED_CAST,
REC_CATCH_EXCEPTION,
BC_UNCONFIRMED_CAST_OF_RETURN_VALUE,
VA_FORMAT_STRING_USES_NEWLINE,
RV_RETURN_VALUE_IGNORED_BAD_PRACTICE,
EI_EXPOSE_REP2,
WMI_WRONG_MAP_ITERATOR,
EI_EXPOSE_REP,
NP_LOAD_OF_KNOWN_NULL_VALUE,
DM_NUMBER_CTOR,
SBSC_USE_STRINGBUFFER_CONCATENATION,
OS_OPEN_STREAM_EXCEPTION_PATH,
NP_NONNULL_RETURN_VIOLATION,
SF_SWITCH_NO_DEFAULT,
RV_RETURN_VALUE_IGNORED_NO_SIDE_EFFECT,
ODR_OPEN_DATABASE_RESOURCE,
PZLA_PREFER_ZERO_LENGTH_ARRAYS,
NP_NULL_ON_SOME_PATH_FROM_RETURN_VALUE,
UCF_USELESS_CONTROL_FLOW,
UC_USELESS_CONDITION,
NP_NULL_ON_SOME_PATH,
DM_FP_NUMBER_CTOR,
SQL_PREPARED_STATEMENT_GENERATED_FROM_NONCONSTANT_STRING,
OBL_UNSATISFIED_OBLIGATION_EXCEPTION_EDGE,
ES_COMPARING_STRINGS_WITH_EQ,
OS_OPEN_STREAM,
RCN_REDUNDANT_NULLCHECK_WOULD_HAVE_BEEN_A_NPE,
NP_PARAMETER_MUST_BE_NONNULL_BUT_MARKED_AS_NULLABLE,
UC_USELESS_OBJECT,
OBL_UNSATISFIED_OBLIGATION,
}
src/main/java/edu/lu/uni/serval/FixPatternParser/violations/FixedViolationHunkParser.java
+5 -246
View File
@@ -1,25 +1,11 @@
package edu.lu.uni.serval.FixPatternParser.violations;
import java.io.*;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.github.gumtreediff.actions.model.Delete;
import com.github.gumtreediff.actions.model.Insert;
import com.github.gumtreediff.actions.model.Move;
import com.github.gumtreediff.actions.model.Update;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.FixPatternParser.Tokenizer;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.diffentry.DiffEntryHunk;
import edu.lu.uni.serval.diffentry.DiffEntryReader;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
import edu.lu.uni.serval.gumtree.regroup.HunkActionFilter;
import edu.lu.uni.serval.gumtree.regroup.SimplifyTree;
import edu.lu.uni.serval.violation.code.parser.ViolationSourceCodeTree;
import java.io.*;
import java.util.List;
/**
* Parse fix violations with GumTree in terms of multiple statements.
@@ -97,232 +83,5 @@ public class FixedViolationHunkParser extends FixedViolationParser {
}
// public void parseFixPatterns(File prevFile, File revFile, File diffentryFile) {
// // GumTree results
// List<HierarchicalActionSet> actionSets = parseChangedSourceCodeWithGumTree2(prevFile, revFile);
//
//
// if (this.resultType != 0) {
//// String type = "";
//// if (this.resultType == 1) {
//// type = "#NullGumTreeResult:";
//// } else if (this.resultType == 2) {
//// type = "#NoSourceCodeChange:";
//// } else if (this.resultType == 3) {
//// type = "#NoStatementChange:";
//// }
// } else {
// List<DiffEntryHunk> diffentryHunks = new DiffEntryReader().readHunks3(diffentryFile);
//
// //Filter out the modify actions, which are not in the DiffEntry hunks.
// HunkActionFilter hunkFilter = new HunkActionFilter();
// List<DiffEntryHunk> selectedPatchHunks = hunkFilter.filterActionsByModifiedRange2(diffentryHunks, actionSets, revFile, prevFile);
//
// for (DiffEntryHunk patchHunk : selectedPatchHunks) {
// List<HierarchicalActionSet> hunkActionSets = patchHunk.getActionSets();
// // multiple UPD, and some UPD contain other UPD.
// removeOverlapperdUPD(hunkActionSets);
//
// // Range of buggy source code
// int bugStartLine = 0;
// int bugEndLine = 0;
// // Range of fixing source code
// int fixStartLine = 0;
// int fixEndLine = 0;
// int bugEndPosition = 0;
// int fixEndPosition = 0;
// int hunkSet = 0;
// for (HierarchicalActionSet hunkActionSet : hunkActionSets) {
//
//
// FileOutputStream f = null;
// try {
// String pj = diffentryFile.getParent().split("GumTreeInputBug4")[1];
// String root = "/Users/anilkoyuncu/bugStudy/dataset/GumTreeOutput2";
// String hunkTreeFileName = root+pj.replace("DiffEntries","ASTDumps/") + diffentryFile.getName() + "_" + String.valueOf(hunkSet);
// f = new FileOutputStream(new File(hunkTreeFileName));
// ObjectOutputStream o = new ObjectOutputStream(f);
// o.writeObject(hunkActionSet.getNode());
//
// o.close();
// f.close();
// } catch (FileNotFoundException e) {
// e.printStackTrace();
// } catch (IOException e) {
// e.printStackTrace();
// }
// hunkSet++;
//
//
//
// int actionBugStart = hunkActionSet.getBugStartLineNum();
// int actionBugEnd = hunkActionSet.getBugEndLineNum();
// int actionFixStart = hunkActionSet.getFixStartLineNum();
// int actionFixEnd = hunkActionSet.getFixEndLineNum();
// if (bugStartLine == 0) {
// bugStartLine = actionBugStart;
// } else if (actionBugStart != 0 && actionBugStart < bugStartLine) {
// bugStartLine = actionBugStart;
// }
// if (fixStartLine == 0) {
// fixStartLine = actionFixStart;
// } else if (actionFixStart != 0 && actionFixStart < fixStartLine) {
// fixStartLine = actionFixStart;
// }
// if (bugEndLine < actionBugEnd) {
// bugEndLine = actionBugEnd;
// bugEndPosition = hunkActionSet.getBugEndPosition();
// }
// if (fixEndLine < actionFixEnd) {
// fixEndLine = actionFixEnd;
// fixEndPosition = hunkActionSet.getFixEndPosition();
// }
// }
//
// if (fixStartLine == 0 && bugStartLine == 0) {
// this.unfixedViolations += "#WRONG: " + revFile.getName() + ":" + patchHunk.getBugLineStartNum() + ", " + patchHunk.getBuggyHunkSize() + "\n";
// this.nullMappingGumTreeResult ++;
// continue;
// }
//
// if (fixStartLine == 0 && bugStartLine != 0) {// pure delete actions.
// // get the exact buggy code by violation's position. TODO later
// }
//
//// if (children.size() == 0) continue;
// boolean isPureInsert = false;
// if (bugStartLine == 0 && patchHunk.getBugLineStartNum() > 0) {
// bugStartLine = patchHunk.getBugLineStartNum();
// bugEndLine = bugStartLine + patchHunk.getBuggyHunkSize() - 1;
// isPureInsert = true;
//// continue;
// }
//// if ((bugEndLine - bugStartLine > Configuration.HUNK_SIZE ) || fixEndLine - fixStartLine > Configuration.HUNK_SIZE) {
//// continue; //TODO hunk size
//// }
// if(patchHunk.getBuggyHunkSize() > Configuration.HUNK_SIZE || patchHunk.getFixedHunkSize() > Configuration.HUNK_SIZE){
// continue;
// }
//
// /**
// * Select edit scripts for deep learning.
// * Edit scripts will be used to mine common fix patterns.
// */
// // 1. First level: AST node type.
// String astEditScripts = getASTEditScriptsDeepFirst(hunkActionSets, bugEndPosition, fixEndPosition);
// if (astEditScripts.contains("\n") || astEditScripts.split(" ").length % 3 != 0) {
// System.err.println("===+++===: " + revFile.getName() + ":" +patchHunk.getBugLineStartNum() + ", " + patchHunk.getBuggyHunkSize());
// }
// // 2. source code: raw tokens
// // 3. abstract identifiers:
// // 4. semi-source code:
// String[] editScriptTokens = astEditScripts.split(" ");
// int size = editScriptTokens.length;
// if (size == 1) {
// this.nullMappingGumTreeResult ++;
// this.unfixedViolations += "#NullMatchedGumTreeResult1:" + revFile.getName() + ":" + patchHunk.getBugLineStartNum() + ", " + patchHunk.getBuggyHunkSize() + "\n";
// continue;
// }
//
// String patchPosition = "\n" + revFile.getName() + "\n@@ -" + bugStartLine + ", " + bugEndLine + " +" + fixStartLine + ", " + fixEndLine + "@@\n";
//// String info = Configuration.PATCH_SIGNAL + "\n" + patchPosition + patchHunk.getHunk() + "\nAST Diff###:\n" + getAstEditScripts(hunkActionSets, bugEndPosition, fixEndPosition) + "\n";
////TODO uncomment the line below for more detailed gumtree input.
// String info = Configuration.PATCH_SIGNAL + "\n" + patchPosition + patchHunk.getHunk() + "\nAST Diff###:\n" + getAstEditScripts(hunkActionSets) + "\n";
//// if (noUpdate(editScriptTokens)) {
//// }
//
// String canonicalVariableNames = getBuggyCodeTree(patchHunk, prevFile, revFile);
// this.patchesSourceCode += info;
// this.patchesSourceCode += "\nRenamed_Variables###:\n" + canonicalVariableNames;
// this.sizes += size + "\n";
// this.astEditScripts += astEditScripts + "\n";
//// String tokens = Tokenizer.getTokensDeepFirst(simpleTree).trim();
//// this.tokensOfSourceCode += tokens + "\n";
// }
// }
// }
private String getAstEditScripts(List<HierarchicalActionSet> hunkActionSets) {
String scripts = "";
for (HierarchicalActionSet hunkActionSet : hunkActionSets) {
scripts += hunkActionSet.toString() + "\n";
}
return scripts;
}
private String getBuggyCodeTree(DiffEntryHunk patchHunk, File prevFile, File revFile) {
int bugStartLine = patchHunk.getBugLineStartNum();
int bugEndLine = bugStartLine + patchHunk.getBugRange() - 1;
ViolationSourceCodeTree parser = new ViolationSourceCodeTree(prevFile, bugStartLine, bugEndLine);
parser.extract();
List<ITree> matchedTrees = parser.getViolationSourceCodeTrees();
Map<String, String> renamedVariablesMap = new HashMap<>();
Map<String, Integer> canonicalVariables = new HashMap<>();
if (matchedTrees.size() > 0) {
SimplifyTree st = new SimplifyTree();
for (ITree matchedTree : matchedTrees) {
st.canonicalizeSourceCodeTree(matchedTree);
}
renamedVariablesMap = st.canonicalVariableMap;
canonicalVariables = st.canonicalVariables;
}
int fixStartLine = patchHunk.getFixLineStartNum();
int fixEndLine = fixStartLine + patchHunk.getFixRange() - 1;
ViolationSourceCodeTree fixedParser = new ViolationSourceCodeTree(revFile, fixStartLine, fixEndLine);
fixedParser.extract();
matchedTrees = fixedParser.getViolationSourceCodeTrees();
if (matchedTrees.size() > 0) {
SimplifyTree st = new SimplifyTree();
st.canonicalVariableMap = renamedVariablesMap;
st.canonicalVariables = canonicalVariables;
for (ITree matchedTree : matchedTrees) {
st.canonicalizeSourceCodeTree(matchedTree);
}
renamedVariablesMap = st.canonicalVariableMap;
}
StringBuilder builder = new StringBuilder();
for (Map.Entry<String, String> entry : renamedVariablesMap.entrySet()) {
builder.append(entry.getKey()).append(" = ").append(entry.getValue()).append("\n");
}
return builder.toString();
}
private void removeOverlapperdUPD(List<HierarchicalActionSet> actionSets) {
if (actionSets.size() == 1) {
return;
}
List<HierarchicalActionSet> updates = new ArrayList<>();
for (HierarchicalActionSet actionSet : actionSets) {
if (actionSet.getActionString().startsWith("UPD")) {
updates.add(actionSet);
}
}
List<HierarchicalActionSet> overlappedUpdates = new ArrayList<>();
if (updates.size() > 1) {
for (HierarchicalActionSet update : updates) {
int startLine = update.getBugStartLineNum();
int endLine = update.getBugEndLineNum();
int endPosition = update.getBugEndPosition();
for (HierarchicalActionSet update2 : updates) {
if (update.equals(update2)) continue;
int startLine2 = update2.getBugStartLineNum();
int endLine2 = update2.getBugEndLineNum();
int endPosition2 = update2.getBugEndPosition();
if (startLine <= startLine2
&& endLine2 <= endLine && endPosition2 < endPosition) {
if (!overlappedUpdates.contains(update)) {
overlappedUpdates.add(update);
}
break;
}
}
}
}
actionSets.removeAll(overlappedUpdates);
}
}
src/main/java/edu/lu/uni/serval/FixPatternParser/violations/FixedViolationParser.java
+8 -71
View File
@@ -1,21 +1,16 @@
package edu.lu.uni.serval.FixPatternParser.violations;
import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.StringReader;
import java.util.ArrayList;
import java.util.List;
import com.github.gumtreediff.actions.model.Action;
import edu.lu.uni.serval.FixPatternParser.Parser;
import edu.lu.uni.serval.gumtree.GumTreeComparer;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalRegrouper;
import edu.lu.uni.serval.utils.FileHelper;
import edu.lu.uni.serval.utils.ListSorter;
import java.io.File;
import java.util.ArrayList;
import java.util.List;
/**
* Parse fix patterns with GumTree.
*
@@ -60,16 +55,7 @@ public class FixedViolationParser extends Parser {
} else {
// Regroup GumTre results.
List<HierarchicalActionSet> allActionSets = new HierarchicalRegrouper().regroupGumTreeResults(gumTreeResults);
// for (HierarchicalActionSet actionSet : allActionSets) {
// String astNodeType = actionSet.getAstNodeType();
// if (astNodeType.endsWith("Statement") || "FieldDeclaration".equals(astNodeType)) {
// actionSets.add(actionSet);
// }
// }
// Filter out modified actions of changing method names, method parameters, variable names and field names in declaration part.
// variable effects range, sub-actions are these kinds of modification?
// actionSets.addAll(new ActionFilter().filterOutUselessActions(allActionSets));
ListSorter<HierarchicalActionSet> sorter = new ListSorter<>(allActionSets);
actionSets = sorter.sortAscending();
@@ -82,63 +68,14 @@ public class FixedViolationParser extends Parser {
}
}
/**
* Read patch source code from buggy and fixed files.
* @param prevFile
* @param revFile
* @param bugStartLineNum
* @param bugEndLineNum
* @param fixStartLineNum
* @param fixEndLineNum
* @param isInsert
* @return
*/
protected String getPatchSourceCode(File prevFile, File revFile, int bugStartLineNum, int bugEndLineNum, int fixStartLineNum, int fixEndLineNum, boolean isInsert) {
String buggyStatements = "";
if (isInsert) {
buggyStatements = readSourceCode(prevFile, bugStartLineNum, bugEndLineNum, "");
} else {
buggyStatements = readSourceCode(prevFile, bugStartLineNum, bugEndLineNum, "-");
}
String fixedStatements = readSourceCode(revFile, fixStartLineNum, fixEndLineNum, "+");
return buggyStatements + fixedStatements;
}
private String readSourceCode(File file, int startLineNum, int endLineNum, String type) {
String sourceCode = "";
String fileContent = FileHelper.readFile(file);
BufferedReader reader = null;
try {
reader = new BufferedReader(new StringReader(fileContent));
String line = null;
int lineIndex = 0;
while ((line = reader.readLine()) != null) {
lineIndex ++;
if (lineIndex >= startLineNum && lineIndex <= endLineNum) {
sourceCode += type + line + "\n";
}
if (lineIndex == endLineNum) break;
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sourceCode;
}
public String getAlarmTypes() {
return violationTypes;
}
@Override
public void parseFixPatterns(File prevFile, File revFile, File diffEntryFile) {
}
// @Override
// public void parseFixPatterns(File prevFile, File revFile, File diffEntryFile) {
//
// }
// public void setUselessViolations(List<Violation> uselessViolations) {
// this.uselessViolations = uselessViolations;
src/main/java/edu/lu/uni/serval/FixPatternParser/violations/FixedViolationSingleStatementParser.java
-156
View File
@@ -1,156 +0,0 @@
package edu.lu.uni.serval.FixPatternParser.violations;
import java.io.File;
import java.util.List;
import org.eclipse.jdt.core.dom.CompilationUnit;
import com.github.gumtreediff.actions.model.Move;
import com.github.gumtreediff.actions.model.Update;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.FixPattern.utils.Checker;
import edu.lu.uni.serval.FixPatternParser.CUCreator;
import edu.lu.uni.serval.FixPatternParser.Tokenizer;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
import edu.lu.uni.serval.gumtree.regroup.SimpleTree;
import edu.lu.uni.serval.gumtree.regroup.SimplifyTree;
/**
* Parse fixed violations with GumTree in terms of single statement.
*
* @author kui.liu
*
*/
public class FixedViolationSingleStatementParser extends FixedViolationParser {
@Override
public void parseFixPatterns(File prevFile, File revFile, File diffentryFile) {
// GumTree results
List<HierarchicalActionSet> actionSets = parseChangedSourceCodeWithGumTree(prevFile, revFile);
if (actionSets.size() > 0) {
CUCreator cuCreator = new CUCreator();
CompilationUnit prevUnit = cuCreator.createCompilationUnit(prevFile);
CompilationUnit revUnit = cuCreator.createCompilationUnit(revFile);
if (prevUnit == null || revUnit == null) {
return;
}
// Read the positions of checked violations
for (HierarchicalActionSet actionSet : actionSets) {
// position of buggy statements
int startPosition = 0;
int endPosition = 0;
// position of fixed statements
int startPosition2 = 0;
int endPosition2 = 0;
String actionStr = actionSet.getActionString();
String astNodeType = actionSet.getAstNodeType();
if (actionStr.startsWith("INS")) {
startPosition2 = actionSet.getStartPosition();
endPosition2 = startPosition2 + actionSet.getLength();
List<Move> firstAndLastMov = getFirstAndLastMoveAction(actionSet);
if (firstAndLastMov != null) {
startPosition = firstAndLastMov.get(0).getNode().getPos();
ITree lastTree = firstAndLastMov.get(1).getNode();
endPosition = lastTree.getPos() + lastTree.getLength();
} else { // Ignore the pure insert actions without any move actions.
continue;
}
} else if (actionStr.startsWith("UPD")) {
startPosition = actionSet.getStartPosition();
endPosition = startPosition + actionSet.getLength();
Update update = (Update) actionSet.getAction();
ITree newNode = update.getNewNode();
startPosition2 = newNode.getPos();
endPosition2 = startPosition2 + newNode.getLength();
if (Checker.containsBodyBlock(astNodeType)) {
List<ITree> children = update.getNode().getChildren();
endPosition = getEndPosition(children);
List<ITree> newChildren = newNode.getChildren();
endPosition2 = getEndPosition(newChildren);
if (endPosition == 0) {
endPosition = startPosition + actionSet.getLength();
}
if (endPosition2 == 0) {
endPosition2 = startPosition2 + newNode.getLength();
}
}
} else {// DEL actions and MOV actions: we don't need these actions, as for now.
continue;
}
if (startPosition == 0 || startPosition2 == 0) {
continue;
}
// Get line numbers.
int startLine = prevUnit.getLineNumber(startPosition);
int endLine = prevUnit.getLineNumber(endPosition);
int startLine2 = revUnit.getLineNumber(startPosition2);
int endLine2 = revUnit.getLineNumber(endPosition2);
Violation violation = null;//findViolation(startLine, endLine, violations);
if (violation == null) continue;
if (endLine - startLine >= Configuration.HUNK_SIZE - 2 || endLine2 - startLine2 >= Configuration.HUNK_SIZE - 2 ) continue;
/*
* Convert the ITree of buggy code to a simple tree.
* It will be used to compute the similarity.
*/
SimplifyTree abstractIdentifier = new SimplifyTree();
abstractIdentifier.abstractTree(actionSet);
SimpleTree simpleTree = actionSet.getSimpleTree();
if (simpleTree == null) { // Failed to get the simple tree for INS actions.
continue;
}
/**
* Select edit scripts for deep learning.
* Edit scripts will be used to mine common fix patterns.
*/
// 1. First level: AST node type.
String astEditScripts = getASTEditScriptsBreadthFirst(actionSet);
int size = astEditScripts.split(" ").length;
if (size == 1) {
continue;
}
// Source Code of patches.
String patchSourceCode = getPatchSourceCode(prevFile, revFile, startLine, endLine, startLine2, endLine2, false);
this.patchesSourceCode += Configuration.PATCH_SIGNAL + "\n" + revFile.getName() + "\n" + patchSourceCode + "\n";
this.sizes += size + "\n";
this.astEditScripts += astEditScripts + "\n";
this.violationTypes += violation.getViolationType() + "\n";
// 2. source code: raw tokens
// String rawTokenEditScripts = getRawTokenEditScripts(actionSet);
// // 3. abstract identifiers:
// String abstractIdentifiersEditScripts = getAbstractIdentifiersEditScripts(actionSet);
// // 4. semi-source code:
// String semiSourceCodeEditScripts = getSemiSourceCodeEditScripts(actionSet);
// this.buggyTrees += Configuration.BUGGY_TREE_TOKEN + "\n" + simpleTree.toString() + "\n";
this.tokensOfSourceCode += Tokenizer.getTokensDeepFirst(simpleTree).trim() + "\n";
// this.actionSets += Configuration.BUGGY_TREE_TOKEN + "\n" + readActionSet(actionSet, "") + "\n";
// this.originalTree += Configuration.BUGGY_TREE_TOKEN + "\n" + actionSet.getOriginalTree().toString() + "\n";
}
actionSets.clear();
} else {
System.out.println(1);
}
}
protected Violation findViolation(int startLine, int endLine, List<Violation> violations) {
for (Violation violation : violations) {
int vStartLine = violation.getStartLineNum();
int vEndLine = violation.getEndLineNum();
if (!(startLine > vEndLine && endLine< vStartLine)) return violation;
}
return null;
}
}
src/main/java/edu/lu/uni/serval/FixPatternParser/violations/Message.java
-20
View File
@@ -1,20 +0,0 @@
package edu.lu.uni.serval.FixPatternParser.violations;
import java.io.File;
public class Message {
private String name;
private String inputPath;
private String innerPort;
public Message(String name , String inputPath, String innerPort) {
super();
this.name = name;
this.inputPath = inputPath;
this.innerPort = innerPort;
}
}
src/main/java/edu/lu/uni/serval/FixPatternParser/violations/Producer.java
-150
View File
@@ -1,150 +0,0 @@
package edu.lu.uni.serval.FixPatternParser.violations;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.io.FilenameFilter;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.TimeoutException;
import java.util.stream.Collectors;
import java.util.stream.Stream;
/**
* Created by anilkoyuncu on 03/04/2018.
*/
public class Producer {
private static Logger log = LoggerFactory.getLogger(Producer.class);
public static void main(String[] args) {
String inputPath;
String outputPath;
String port;
String portInner;
String pairsCSVPath;
String importScript;
String pairsCompletedPath;
String serverWait;
if (args.length > 0) {
inputPath = args[0];
port = args[1];
portInner = args[2];
serverWait = args[3];
// pairsCSVPath = args[3];
// importScript = args[4];
// pairsCompletedPath = args[3];
} else {
inputPath = "/Users/anilkoyuncu/bugStudy/dataset/GumTreeOutput2";
outputPath = "/Users/anilkoyuncu/bugStudy/dataset/";
port = "6379";
portInner = "6380";
serverWait = "10000";
pairsCSVPath = "/Users/anilkoyuncu/bugStudy/dataset/pairs/test";
importScript = "/Users/anilkoyuncu/bugStudy/dataset/pairs/test2.sh";
pairsCompletedPath = "/Users/anilkoyuncu/bugStudy/dataset/pairs_completed";
}
calculatePairs(inputPath, port);
log.info("Calculate pairs done");
}
public static void calculatePairs(String inputPath,String port) {
File folder = new File(inputPath);
File[] listOfFiles = folder.listFiles();
Stream<File> stream = Arrays.stream(listOfFiles);
List<File> pjs = stream
.filter(x -> !x.getName().startsWith("."))
.collect(Collectors.toList());
List<File> fileToCompare = new ArrayList<>();
for (File pj : pjs) {
File[] files = pj.listFiles(new FilenameFilter() {
public boolean accept(File dir, String name) {
return name.startsWith("ActionSetDumps");
}
});
Collections.addAll(fileToCompare, files[0].listFiles());
}
System.out.println("a");
// compareAll(fileToCompare);
readMessageFiles(fileToCompare, port);
}
private static void readMessageFiles(List<File> folders, String port) {
List<String> treesFileNames = new ArrayList<>();
for (File target : folders) {
treesFileNames.add(target.toString());
}
log.info("Calculating pairs");
int fileCounter = 0;
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("10.240.5.10");
// factory.setVirtualHost("treeCompare");
factory.setPort(5672);
factory.setUsername("anil");
factory.setPassword("changeme2018");
factory.setConnectionTimeout(1000);
Connection connection = null;
Channel channel = null;
try {
connection = factory.newConnection();
channel = connection.createChannel();
channel.queueDeclare("tree_queue", false, false, false, null);
} catch (IOException e) {
e.printStackTrace();
} catch (TimeoutException e) {
e.printStackTrace();
}
for (int i = 0; i < treesFileNames.size(); i++) {
for (int j = i + 1; j < treesFileNames.size(); j++) {
// do operations with jedis resource
String key = "pair_" + String.valueOf(i) + "_" + String.valueOf(j);
// String value = treesFileNames.get(i).split("GumTreeOutput2")[1] +","+treesFileNames.get(j).split("GumTreeOutput2")[1];
// jedis.set(key,value);
String message = key +","+treesFileNames.get(i).split("GumTreeOutput2")[1] + "," + treesFileNames.get(j).split("GumTreeOutput2")[1];
try {
channel.basicPublish("", "tree_queue", null, message.getBytes());
} catch (IOException e) {
e.printStackTrace();
}
}
}
log.info("Done pairs");
}
}
src/main/java/edu/lu/uni/serval/FixPatternParser/violations/Violation.java
-133
View File
@@ -1,133 +0,0 @@
package edu.lu.uni.serval.FixPatternParser.violations;
import java.util.ArrayList;
import java.util.List;
import edu.lu.uni.serval.diffentry.DiffEntryHunk;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
public class Violation implements Comparable<Violation> {
private String fileName = "";
private Integer startLineNum;
private int endLineNum;
private int bugStartLineNum;
private int bugEndLineNum;
private int fixStartLineNum = 0;
private int fixEndLineNum;
private String violationType;
private List<DiffEntryHunk> hunks = new ArrayList<>();
private List<HierarchicalActionSet> actionSets;
private int bugFixStartLineNum = 0; // the heuristic matched fix start line of a violation
private int bugFixEndLineNum = 0; // the heuristic matched fix end line of a violation
public Violation(Integer startLineNum, int endLineNum, String violationType) {
super();
this.startLineNum = startLineNum;
this.endLineNum = endLineNum;
this.violationType = violationType;
this.actionSets = new ArrayList<>();
}
public String getFileName() {
return fileName;
}
public void setFileName(String fileName) {
this.fileName = fileName;
}
public Integer getStartLineNum() {
return startLineNum;
}
public int getEndLineNum() {
return endLineNum;
}
public int getBugStartLineNum() {
return bugStartLineNum;
}
public void setBugStartLineNum(int bugStartLineNum) {
this.bugStartLineNum = bugStartLineNum;
}
public int getBugEndLineNum() {
return bugEndLineNum;
}
public void setBugEndLineNum(int bugEndLineNum) {
this.bugEndLineNum = bugEndLineNum;
}
public int getFixStartLineNum() {
return fixStartLineNum;
}
public void setFixStartLineNum(int fixStartLineNum) {
this.fixStartLineNum = fixStartLineNum;
}
public int getFixEndLineNum() {
return fixEndLineNum;
}
public void setFixEndLineNum(int fixEndLineNum) {
this.fixEndLineNum = fixEndLineNum;
}
public List<DiffEntryHunk> getHunks() {
return hunks;
}
public void setHunks(List<DiffEntryHunk> hunks) {
this.hunks = hunks;
}
public String getViolationType() {
return violationType;
}
public List<HierarchicalActionSet> getActionSets() {
return actionSets;
}
public int getBugFixStartLineNum() {
return bugFixStartLineNum;
}
public void setBugFixStartLineNum(int bugFixStartLineNum) {
this.bugFixStartLineNum = bugFixStartLineNum;
}
public int getBugFixEndLineNum() {
return bugFixEndLineNum;
}
public void setBugFixEndLineNum(int bugFixEndLineNum) {
this.bugFixEndLineNum = bugFixEndLineNum;
}
@Override
public int compareTo(Violation v) {
return this.startLineNum.compareTo(v.startLineNum);
}
@Override
public String toString() {
return this.startLineNum + " : " + this.endLineNum + " : " + this.violationType;
}
@Override
public boolean equals(Object obj) {
if (obj instanceof Violation) {
Violation v = (Violation) obj;
if (this.fileName.equals(v.fileName) && this.violationType.equals(v.violationType) && this.startLineNum == v.startLineNum && this.endLineNum == v.endLineNum) {
return true;
}
}
return false;
}
}
src/main/java/edu/lu/uni/serval/MultipleThreadsParser/AkkaParser.java
-121
View File
@@ -1,121 +0,0 @@
package edu.lu.uni.serval.MultipleThreadsParser;
import java.io.File;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import akka.actor.ActorRef;
import akka.actor.ActorSystem;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.utils.FileHelper;
/**
* Multi-thread parser of parsing the difference between buggy code file and fixed code file.
*
* @author kui.liu
*
*/
public class AkkaParser {
private static Logger log = LoggerFactory.getLogger(AkkaParser.class);
/**
* Two parameters:
* First one: the root path of input data.
* Second one: the number of threads.
* Third one: the threshold of selecting patch hunks.
* @param args
*/
@SuppressWarnings("deprecation")
public static void main(String[] args) {
// String inputRootPath = args[0];
int numberOfWorkers = 5; //Integer.parseInt(args[1]);
int hunkThreshold = 10;
// try {
// hunkThreshold = Integer.parseInt(args[2]);
// } catch (NumberFormatException e1) {
// hunkThreshold = 10;
// }
// Configuration.ROOT_PATH = inputRootPath;
Configuration.HUNK_SIZE = hunkThreshold;
List<String> pjList = Arrays.asList("APACHE-CAMEL","APACHE-HBASE","APACHE-HIVE","COMMONS-CODEC","COMMONS-COLLECTIONS",
"COMMONS-COMPRESS","COMMONS-CONFIGURATION","COMMONS-CRYPTO","COMMONS-CSV","COMMONS-IO","COMMONS-LANG",
"COMMONS-MATH","COMMONS-WEAVER","JBOSS-ENTESB","JBOSS-JBMETA","SPRING-AMQP","SPRING-ANDROID","SPRING-BATCH",
"SPRING-BATCHADM","SPRING-DATACMNS","SPRING-DATAGRAPH","SPRING-DATAJPA","SPRING-DATAMONGO","SPRING-DATAREDIS",
"SPRING-DATAREST","SPRING-LDAP","SPRING-MOBILE","SPRING-ROO","SPRING-SEC","SPRING-SECOAUTH","SPRING-SGF","SPRING-SHDP",
"SPRING-SOCIAL","SPRING-SOCIALFB","SPRING-SOCIALLI","SPRING-SOCIALTW","SPRING-SPR","SPRING-SWF","SPRING-SWS","WILDFLY-ELY",
"WILDFLY-SWARM","WILDFLY-WFARQ","WILDFLY-WFCORE","WILDFLY-WFLY","WILDFLY-WFMP");
for (String pj : pjList) {
String[] split = pj.split("-");
String pjPath = split[0];
String pjName = split[1];
// String rootPath = "/Volumes/data/bugStudy/";
// String DATASET_FILE_PATH = rootPath + "/dataset/GumTreeInput/" + pjName;
// String GIT_REPOSITORY_PATH = "/Users/anilkoyuncu/bugLocalizationStudy/irblsensitivity/data/" + pjPath + "/" + pjName + "/gitrepo/.git";
log.info(pjName);
// input data
log.info("Get the input data...");
final List<MessageFile> msgFiles = getMessageFiles(Configuration.GUM_TREE_INPUT +pjName +"gitrepo/");
log.info("MessageFiles: " + msgFiles.size());
// output path
final String editScriptsFilePath = Configuration.EDITSCRIPTS_FILE_PATH + pjName + "/";
final String patchesSourceCodeFilePath = Configuration.PATCH_SOURCECODE_FILE_PATH+ pjName + "/";
final String buggyTokensFilePath = Configuration.BUGGY_CODE_TOKEN_FILE_PATH+ pjName + "/";
final String editScriptSizesFilePath = Configuration.EDITSCRIPT_SIZES_FILE_PATH+ pjName + "/";
FileHelper.deleteDirectory(editScriptsFilePath);
FileHelper.deleteDirectory(patchesSourceCodeFilePath);
FileHelper.deleteDirectory(buggyTokensFilePath);
FileHelper.deleteDirectory(editScriptSizesFilePath);
ActorSystem system = null;
ActorRef parsingActor = null;
final WorkMessage msg = new WorkMessage(0, msgFiles);
try {
log.info("Akka begins...");
system = ActorSystem.create("Mining-FixPattern-System");
// parsingActor = system.actorOf(ParseFixPatternActor.props(numberOfWorkers, editScriptsFilePath,
// patchesSourceCodeFilePath, buggyTokensFilePath, editScriptSizesFilePath), "mine-fix-pattern-actor");
parsingActor.tell(msg, ActorRef.noSender());
} catch (Exception e) {
system.shutdown();
e.printStackTrace();
}
}
}
/**
* Get violation-related files.
*
* @param gumTreeInput
* @return
*/
public static List<MessageFile> getMessageFiles(String gumTreeInput) {
String inputPath = gumTreeInput; // prevFiles revFiles diffentryFile positionsFile
File revFilesPath = new File(inputPath + "revFiles/");
File[] revFiles = revFilesPath.listFiles(); // project folders
List<MessageFile> msgFiles = new ArrayList<>();
for (File revFile : revFiles) {
if (revFile.getName().endsWith(".java")) {
String fileName = revFile.getName();
File prevFile = new File(gumTreeInput + "prevFiles/prev_" + fileName);// previous file
fileName = fileName.replace(".java", ".txt");
File diffentryFile = new File(gumTreeInput + "diffentries/" + fileName); // DiffEntry file
File positionFile = new File(gumTreeInput + "positions/" + fileName); // position file
MessageFile msgFile = new MessageFile(revFile, prevFile, diffentryFile);
msgFile.setPositionFile(positionFile);
msgFiles.add(msgFile);
}
}
return msgFiles;
}
}
src/main/java/edu/lu/uni/serval/MultipleThreadsParser/AkkaParser2.java
-146
View File
@@ -1,146 +0,0 @@
package edu.lu.uni.serval.MultipleThreadsParser;
import akka.actor.ActorRef;
import akka.actor.ActorSystem;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.utils.FileHelper;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
/**
* Multi-thread parser of parsing the difference between buggy code file and fixed code file.
*
* @author kui.liu
*
*/
public class AkkaParser2 {
private static Logger log = LoggerFactory.getLogger(AkkaParser2.class);
/**
* Two parameters:
* First one: the root path of input data.
* Second one: the number of threads.
* Third one: the threshold of selecting patch hunks.
* @param args
*/
@SuppressWarnings("deprecation")
public static void main(String[] args) {
String inputRootPath;
String outputRootPath;
int numberOfWorkers;
if(args.length > 0){
inputRootPath = args[0];
outputRootPath = args[1];
numberOfWorkers = Integer.parseInt(args[2]);
}else{
inputRootPath = "/Users/anilkoyuncu/bugStudy/dataset/GumTreeInputBug";
outputRootPath = "/Users/anilkoyuncu/bugStudy/dataset/GumTreeOutputBug/";
numberOfWorkers = 1;
}
// try {
// hunkThreshold = Integer.parseInt(args[2]);
// } catch (NumberFormatException e1) {
// hunkThreshold = 10;
// }
// Configuration.ROOT_PATH = inputRootPath;
// log.info(Configuration.ROOT_PATH);
// Configuration.HUNK_SIZE = hunkThreshold;
// input data
// String GUM_TREE_INPUT = inputRootPath + "GumTreeInput/";
log.info("Get the input data..." + inputRootPath );
log.info("Set the output data..." + outputRootPath );
File folder = new File(inputRootPath);
File[] listOfFiles = folder.listFiles();
Stream<File> stream = Arrays.stream(listOfFiles);
List<File> folders = stream
.filter(x -> !x.getName().startsWith("."))
.collect(Collectors.toList());
for (File target : folders) {
log.info("MessageFiles: " + target.toString());
final List<MessageFile> msgFiles = getMessageFiles(target.toString() + "/");
log.info("MessageFiles: " + msgFiles.size());
// output path
String pjName = target.getName();
// output path
String GUM_TREE_OUTPUT = outputRootPath + pjName + "/";
final String editScriptsFilePath = GUM_TREE_OUTPUT + "editScripts/";
final String patchesSourceCodeFilePath =GUM_TREE_OUTPUT + "patchSourceCode/";
final String buggyTokensFilePath = GUM_TREE_OUTPUT + "tokens/";
final String editScriptSizesFilePath = GUM_TREE_OUTPUT + "editScriptSizes/";
final String alarmTypesFilePath = GUM_TREE_OUTPUT + "alarmTypes/";
FileHelper.deleteDirectory(editScriptsFilePath);
FileHelper.deleteDirectory(patchesSourceCodeFilePath);
FileHelper.deleteDirectory(buggyTokensFilePath);
FileHelper.deleteDirectory(editScriptSizesFilePath);
ActorSystem system = null;
ActorRef parsingActor = null;
final WorkMessage msg = new WorkMessage(0, msgFiles);
try {
log.info("Akka begins...");
system = ActorSystem.create("Mining-FixPattern-System");
// parsingActor = system.actorOf(ParseFixPatternActor.props(numberOfWorkers, editScriptsFilePath,
// patchesSourceCodeFilePath, buggyTokensFilePath, editScriptSizesFilePath), "mine-fix-pattern-actor");
parsingActor.tell(msg, ActorRef.noSender());
} catch (Exception e) {
system.shutdown();
e.printStackTrace();
}
}
}
/**
* Get violation-related files.
*
* @param gumTreeInput
* @return
*/
public static List<MessageFile> getMessageFiles(String gumTreeInput) {
String inputPath = gumTreeInput; // prevFiles revFiles diffentryFile positionsFile
File revFilesPath = new File(inputPath + "revFiles/");
File[] revFiles = revFilesPath.listFiles(); // project folders
List<MessageFile> msgFiles = new ArrayList<>();
for (File revFile : revFiles) {
// if (revFile.getName().endsWith(".c") || revFile.getName().endsWith(".h")) {
if (revFile.getName().endsWith(".java")) {
String fileName = revFile.getName();
File prevFile = new File(gumTreeInput + "prevFiles/prev_" + fileName);// previous file
fileName = fileName.replace(".java", ".txt");
File diffentryFile = new File(gumTreeInput + "DiffEntries/" + fileName); // DiffEntry file
File positionFile = new File(gumTreeInput + "positions/" + fileName); // position file
MessageFile msgFile = new MessageFile(revFile, prevFile, diffentryFile);
msgFile.setPositionFile(positionFile);
msgFiles.add(msgFile);
}
// }
}
return msgFiles; //.subList(10,20);
}
}
src/main/java/edu/lu/uni/serval/MultipleThreadsParser/ParseFixPatternActor.java
+4 -5
View File
@@ -1,15 +1,14 @@
package edu.lu.uni.serval.MultipleThreadsParser;
import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import akka.actor.ActorRef;
import akka.actor.Props;
import akka.actor.UntypedActor;
import akka.japi.Creator;
import akka.routing.RoundRobinPool;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.List;
public class ParseFixPatternActor extends UntypedActor {
src/main/java/edu/lu/uni/serval/MultipleThreadsParser/ParseFixPatternWorker.java
+7 -47
View File
@@ -1,28 +1,17 @@
package edu.lu.uni.serval.MultipleThreadsParser;
import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.StringReader;
import java.util.List;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import akka.actor.Props;
import akka.actor.UntypedActor;
import akka.japi.Creator;
import edu.lu.uni.serval.FixPatternParser.RunnableParser;
import edu.lu.uni.serval.FixPatternParser.violations.FixedViolationHunkParser;
import edu.lu.uni.serval.FixPatternParser.violations.Violation;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.utils.FileHelper;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.util.List;
import java.util.concurrent.*;
public class ParseFixPatternWorker extends UntypedActor {
private static Logger log = LoggerFactory.getLogger(ParseFixPatternActor.class);
@@ -79,10 +68,6 @@ public class ParseFixPatternWorker extends UntypedActor {
File diffentryFile = msgFile.getDiffEntryFile();
// File positionFile = msgFile.getPositionFile();
/*if (revFile.getName().toLowerCase().contains("test")) {
continue;
}*/
FixedViolationHunkParser parser = new FixedViolationHunkParser();
final ExecutorService executor = Executors.newSingleThreadExecutor();
@@ -103,13 +88,7 @@ public class ParseFixPatternWorker extends UntypedActor {
String editScript = parser.getAstEditScripts();
if ("".equals(editScript)) {
// if (parser.resultType == 1) {
// nullGumTreeResults += countAlarms(positionFile, "");
// } else if (parser.resultType == 2) {
// noSourceCodeChanges += countAlarms(positionFile, "");
// } else if (parser.resultType == 3) {
// noStatementChanges += countAlarms(positionFile, "");
// }
} else {
editScripts.append(editScript);
patchesSourceCode.append(parser.getPatchesSourceCode());
@@ -118,29 +97,21 @@ public class ParseFixPatternWorker extends UntypedActor {
counter ++;
if (counter % 100 == 0) {
// FileHelper.outputToFile(editScriptsFilePath + "edistScripts_" + id + ".list", editScripts, true);
// FileHelper.outputToFile(patchesSourceCodeFilePath + "patches_" + id + ".list", patchesSourceCode, true);
// FileHelper.outputToFile(editScriptSizesFilePath + "sizes_" + id + ".list", sizes, true);
// FileHelper.outputToFile(buggyTokensFilePath + "tokens_" + id + ".list", tokens, true);
editScripts.setLength(0);
patchesSourceCode.setLength(0);
sizes.setLength(0);
tokens.setLength(0);
log.info("Worker #" + id +" finialized parsing " + counter + " files...");
// FileHelper.outputToFile("OUTPUT/testingInfo_" + id + ".list", testingInfo, true);
testingInfo.setLength(0);
}
}
} catch (TimeoutException e) {
future.cancel(true);
// timeouts += countAlarms(positionFile, "#Timeout:");
System.err.println("#Timeout: " + revFile.getName());
} catch (InterruptedException e) {
// timeouts += countAlarms(positionFile, "#TimeInterrupted:");
System.err.println("#TimeInterrupted: " + revFile.getName());
e.printStackTrace();
} catch (ExecutionException e) {
// timeouts += countAlarms(positionFile, "#TimeAborted:");
System.err.println("#TimeAborted: " + revFile.getName());
e.printStackTrace();
} finally {
@@ -149,24 +120,13 @@ public class ParseFixPatternWorker extends UntypedActor {
}
if (sizes.length() > 0) {
// FileHelper.outputToFile(editScriptsFilePath + "editScripts_" + id + ".list", editScripts, true);
// FileHelper.outputToFile(patchesSourceCodeFilePath + "patches_" + id + ".list", patchesSourceCode, true);
// FileHelper.outputToFile(editScriptSizesFilePath + "sizes_" + id + ".list", sizes, true);
// FileHelper.outputToFile(buggyTokensFilePath + "tokens_" + id + ".list", tokens, true);
editScripts.setLength(0);
patchesSourceCode.setLength(0);
sizes.setLength(0);
tokens.setLength(0);
// FileHelper.outputToFile("OUTPUT/testingInfo_" + id + ".list", testingInfo, true);
testingInfo.setLength(0);
}
// String statistic = "\nNullGumTreeResults: " + nullGumTreeResults + "\nNoSourceCodeChanges: " + noSourceCodeChanges +
// "\nNoStatementChanges: " + noStatementChanges + "\nNullDiffEntry: " + nullDiffEntry + "\nNullMatchedGumTreeResults: " + nullMappingGumTreeResults +
// "\nPureDeletion: " + pureDeletion + "\nLargeHunk: " + largeHunk + "\nNullSourceCode: " + nullSourceCode +
// "\nTestingInfo: " + testInfos + "\nTimeout: " + timeouts;
// FileHelper.outputToFile("OUTPUT/statistic_" + id + ".list", statistic, false);
// FileHelper.outputToFile("OUTPUT/UnfixedV_" + id + ".list", builder, false);
log.info("Worker #" + id +"finialized parsing " + counter + " files...");
log.info("Worker #" + id + " finialized the work...");
src/main/java/edu/lu/uni/serval/MultipleThreadsParser/SingleThreadParser.java
-190
View File
@@ -1,190 +0,0 @@
package edu.lu.uni.serval.MultipleThreadsParser;
import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.StringReader;
import java.util.List;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import edu.lu.uni.serval.FixPatternParser.RunnableParser;
import edu.lu.uni.serval.FixPatternParser.violations.FixedViolationHunkParser;
import edu.lu.uni.serval.FixPatternParser.violations.Violation;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.utils.FileHelper;
public class SingleThreadParser {
public static void main(String[] args) {
// output path
final String editScriptsFilePath = Configuration.EDITSCRIPTS_FILE_PATH;
final String patchesSourceCodeFilePath = Configuration.PATCH_SOURCECODE_FILE_PATH;
final String buggyTokensFilePath = Configuration.BUGGY_CODE_TOKEN_FILE_PATH;
final String editScriptSizesFilePath = Configuration.EDITSCRIPT_SIZES_FILE_PATH;
final String alarmTypesFilePath = Configuration.ALARM_TYPES_FILE_PATH;
FileHelper.deleteDirectory(editScriptsFilePath);
FileHelper.deleteDirectory(patchesSourceCodeFilePath);
FileHelper.deleteDirectory(buggyTokensFilePath);
FileHelper.deleteDirectory(editScriptSizesFilePath);
FileHelper.deleteDirectory(alarmTypesFilePath);
final List<MessageFile> files = AkkaParser.getMessageFiles(Configuration.GUM_TREE_INPUT);
StringBuilder editScripts = new StringBuilder();
StringBuilder patchesSourceCode = new StringBuilder();
StringBuilder sizes = new StringBuilder();
StringBuilder tokens = new StringBuilder();
StringBuilder alarmTypes = new StringBuilder();
StringBuilder testingInfo = new StringBuilder();
int counter = 0;
int testViolations = 0;
int nullGumTreeResults = 0;
int noSourceCodeChanges = 0;
int noStatementChanges = 0;
int nullDiffEntry = 0;
int nullMappingGumTreeResults = 0;
int pureDeletion = 0;
int largeHunk = 0;
int nullSourceCode = 0;
int testInfos = 0;
int timeouts = 0;
StringBuilder builder = new StringBuilder();
for (MessageFile msgFile : files) {
File revFile = msgFile.getRevFile();
File prevFile = msgFile.getPrevFile();
File diffentryFile = msgFile.getDiffEntryFile();
File positionFile = msgFile.getPositionFile();
if (revFile.getName().toLowerCase().contains("test#") || revFile.getName().toLowerCase().contains("tests#")) {
testViolations += countAlarms(positionFile, "#TestViolation:");
continue;
}
FixedViolationHunkParser parser = new FixedViolationHunkParser();
// parser.setUselessViolations(uselessViolations);
final ExecutorService executor = Executors.newSingleThreadExecutor();
// schedule the work
final Future<?> future = executor.submit(new RunnableParser(prevFile, revFile, diffentryFile, parser));
try {
// wait for task to complete
future.get(Configuration.SECONDS_TO_WAIT, TimeUnit.SECONDS);
nullDiffEntry += parser.nullMatchedDiffEntry;
nullMappingGumTreeResults += parser.nullMappingGumTreeResult;
pureDeletion += parser.pureDeletions;
largeHunk += parser.largeHunk;
nullSourceCode += parser.nullSourceCode;
testInfos += parser.testInfos;
testingInfo.append(parser.testingInfo);
builder.append(parser.unfixedViolations);
String editScript = parser.getAstEditScripts();
if ("".equals(editScript)) {
if (parser.resultType == 1) {
nullGumTreeResults += countAlarms(positionFile, "");
} else if (parser.resultType == 2) {
noSourceCodeChanges += countAlarms(positionFile, "");
} else if (parser.resultType == 3) {
noStatementChanges += countAlarms(positionFile, "");
// } else if (parser.resultType == 4) {
}
} else {
editScripts.append(editScript);
patchesSourceCode.append(parser.getPatchesSourceCode());
sizes.append(parser.getSizes());
alarmTypes.append(parser.getAlarmTypes());
tokens.append(parser.getTokensOfSourceCode());
counter ++;
if (counter % 5000 == 0) {
FileHelper.outputToFile(editScriptsFilePath + "edistScripts.list", editScripts, true);
FileHelper.outputToFile(patchesSourceCodeFilePath + "patches.list", patchesSourceCode, true);
FileHelper.outputToFile(editScriptSizesFilePath + "sizes.list", sizes, true);
FileHelper.outputToFile(buggyTokensFilePath + "tokens.list", tokens, true);
editScripts.setLength(0);
patchesSourceCode.setLength(0);
sizes.setLength(0);
tokens.setLength(0);
FileHelper.outputToFile(alarmTypesFilePath + "alarmTypes.list", alarmTypes, true);
alarmTypes.setLength(0);
FileHelper.outputToFile("OUTPUT/testingInfo.list", testingInfo, true);
testingInfo.setLength(0);
}
}
} catch (TimeoutException e) {
// err.println("task timed out");
future.cancel(true);
timeouts += countAlarms(positionFile, "#Timeout:");
// System.err.println("#Timeout: " + revFile.getName());
} catch (InterruptedException e) {
timeouts += countAlarms(positionFile, "#TimeInterrupted:");
// err.println("task interrupted");
// System.err.println("#TimeInterrupted: " + revFile.getName());
} catch (ExecutionException e) {
timeouts += countAlarms(positionFile, "#TimeAborted:");
// err.println("task aborted");
// System.err.println("#TimeAborted: " + revFile.getName());
} finally {
executor.shutdownNow();
}
}
if (sizes.length() > 0) {
FileHelper.outputToFile(editScriptsFilePath + "edistScripts.list", editScripts, true);
FileHelper.outputToFile(patchesSourceCodeFilePath + "patches.list", patchesSourceCode, true);
FileHelper.outputToFile(editScriptSizesFilePath + "sizes.list", sizes, true);
FileHelper.outputToFile(buggyTokensFilePath + "tokens.list", tokens, true);
editScripts.setLength(0);
patchesSourceCode.setLength(0);
sizes.setLength(0);
tokens.setLength(0);
FileHelper.outputToFile(alarmTypesFilePath + "alarmTypes.list", alarmTypes, true);
alarmTypes.setLength(0);
FileHelper.outputToFile("OUTPUT/testingInfo.list", testingInfo, true);
testingInfo.setLength(0);
}
String statistic = "TestViolations: " + testViolations + "\nNullGumTreeResults: " + nullGumTreeResults + "\nNoSourceCodeChanges: " + noSourceCodeChanges +
"\nNoStatementChanges: " + noStatementChanges + "\nNullDiffEntry: " + nullDiffEntry + "\nNullMatchedGumTreeResults: " + nullMappingGumTreeResults +
"\nPureDeletion: " + pureDeletion + "\nLargeHunk: " + largeHunk + "\nNullSourceCode: " + nullSourceCode +
"\nTestingInfo: " + testInfos + "\nTimeout: " + timeouts;
FileHelper.outputToFile("OUTPUT/statistic.list", statistic, false);
FileHelper.outputToFile("OUTPUT/UnfixedV.list", builder, false);
}
private static int countAlarms(File positionFile, String type) {//, List<Violation> uselessViolations) {
int counter = 0;
String content = FileHelper.readFile(positionFile);
BufferedReader reader = new BufferedReader(new StringReader(content));
String line = null;
try {
while ((line = reader.readLine()) != null) {
String[] elements = line.split(":");
Violation v = new Violation(Integer.parseInt(elements[1]), Integer.parseInt(elements[2]), elements[0]);
String fileName = positionFile.getName().replace(".txt", ".java");
v.setFileName(fileName);
counter ++;
if (!"".equals(type)) {
System.err.println(type + fileName + ":" + elements[1] + ":" + elements[2] + ":" + elements[0]);
}
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return counter;
}
}
src/main/java/edu/lu/uni/serval/MultipleThreadsParser2/AkkaMatcher.java
-214
View File
@@ -1,214 +0,0 @@
package edu.lu.uni.serval.MultipleThreadsParser2;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Scanner;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import akka.actor.ActorRef;
import akka.actor.ActorSystem;
import edu.lu.uni.serval.MultipleThreadsParser.MessageFile;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.utils.FileHelper;
import edu.lu.uni.serval.utils.MapSorter;
/**
* Multi-thread parser of parsing the difference between buggy code file and fixed code file.
*
* @author kui.liu
*
*/
public class AkkaMatcher {
private static Logger log = LoggerFactory.getLogger(AkkaMatcher.class);
public static void main(String[] args) {
List<Double[]> extractedFeatures = readStringList(Configuration.ROOT_PATH + "TestData/2_CNNinput.csv");
int size = extractedFeatures.size();
List<Double[]> trainingFeatures = extractedFeatures.subList(0, size - 178);
List<Double[]> bugFeatures = extractedFeatures.subList(size - 178, size);
for (int index = 0; index < 178; index ++) {
// AkkaMatcher computor = new AkkaMatcher();
// computor.matchFixPatterns(bugFeatures.get(index), index, trainingFeatures);
Map<Integer, Double> similarities = new HashMap<>();
for (int i = 0; i < size - 178; i ++) {
Double similarity = Math.abs(computeSimilarity(bugFeatures.get(index), trainingFeatures.get(i)));
similarities.put(i + 1, similarity);
}
MapSorter<Integer, Double> mapSorter = new MapSorter<Integer, Double>();
Map<Integer, Double> sortedSimilarities = mapSorter.sortByValueDescending(similarities);
List<Integer> similarityList = new ArrayList<>();
double similarity = 0;
int num = 0;
for (Map.Entry<Integer, Double> entry : sortedSimilarities.entrySet()) {
if (entry.getValue().equals(Double.NaN)) {
continue;
}
if (entry.getValue() == similarity) {
continue;
}
similarityList.add(entry.getKey());
similarity = entry.getValue();
if (++ num % 100 == 0) {
break;
}
}
outputMatchedPatterns(similarityList, index + 1);
}
}
private static Double computeSimilarity(Double[] feature, Double[] trainingFeature) {
Double similarity = DistanceCalculator.cosineSimilarityDistance(trainingFeature, feature);
return similarity;
}
private static void outputMatchedPatterns(List<Integer> similarityList, int bugId) {
String outputFile = Configuration.ROOT_PATH + "TestData/MatchedFixPatterns/Bug_" + bugId + ".list";
StringBuilder builder = new StringBuilder("BugId: " + bugId + "\n\n\n");
for (int i = 0, size = similarityList.size(); i < size; i ++) {
int patternPosition = similarityList.get(i);
builder.append(readPattern(patternPosition));
}
FileHelper.outputToFile(outputFile, builder, false);
}
private static String readPattern(int patternPosition) {
String patternInfo = "";
FileInputStream fis = null;
Scanner scanner = null;
try {
fis = new FileInputStream(Configuration.SELECTED_PATCHES_SOURE_CODE_FILE);
scanner = new Scanner(fis);
int index = 0;
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
if (line.equals("PATCH###")) {
if (patternPosition == index) {
break;
}
index ++;
patternInfo = "";
}
patternInfo += line + "\n";
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} finally {
try {
scanner.close();
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return patternInfo;
}
@SuppressWarnings("deprecation")
public void matchFixPatterns(Double[] bugFeature, int bugIndex, List<Double[]> trainingFeatures) {
ActorSystem system = null;
ActorRef parsingActor = null;
int numberOfWorkers = 100;
final WorkMessage msg = new WorkMessage(bugIndex, bugFeature, trainingFeatures);
try {
log.info("Akka begins...");
system = ActorSystem.create("Matching-FixPattern-System");
parsingActor = system.actorOf(MatchFixPatternActor.props(numberOfWorkers), "match-fix-pattern-actor");
parsingActor.tell(msg, ActorRef.noSender());
} catch (Exception e) {
system.shutdown();
e.printStackTrace();
}
}
/**
* Get bug commit-related files.
*
* @return
*/
public static List<MessageFile> getMessageFiles() {
String inputPath = Configuration.GUM_TREE_INPUT; //DiffEntries prevFiles revFiles
File inputFileDirector = new File(inputPath);
File[] files = inputFileDirector.listFiles(); // project folders
log.info("Projects: " + files.length);
List<MessageFile> msgFiles = new ArrayList<>();
for (File file : files) {
if (!file.isDirectory()) continue;
// if (!(file.getName().startsWith("k") || file.getName().startsWith("l"))) continue;
if (file.getName().startsWith("a") || file.getName().startsWith("b")
|| file.getName().startsWith("c") || file.getName().startsWith("d")
|| file.getName().startsWith("e") || file.getName().startsWith("f")
|| file.getName().startsWith("g") || file.getName().startsWith("h")
||file.getName().startsWith("h") || file.getName().startsWith("i")
|| file.getName().startsWith("k") || file.getName().startsWith("l")
|| file.getName().startsWith("j") || file.getName().startsWith("t")) continue;
// if (!file.getName().startsWith("j")) continue;
log.info("Project name: " + file.getName());
String projectFolder = file.getPath();
File revFileFolder = new File(projectFolder + "/revFiles/");// revised file folder
File[] revFiles = revFileFolder.listFiles();
for (File revFile : revFiles) {
if (revFile.getName().endsWith(".java")) {
File prevFile = new File(projectFolder + "/prevFiles/prev_" + revFile.getName());// previous file
File diffentryFile = new File(projectFolder + "/DiffEntries/" + revFile.getName().replace(".java", ".txt")); // DiffEntry file
MessageFile msgFile = new MessageFile(revFile, prevFile, diffentryFile);
msgFiles.add(msgFile);
}
}
}
return msgFiles;
}
public static List<Double[]> readStringList(String inputFile) {
List<Double[]> list = new ArrayList<>();
FileInputStream fis = null;
Scanner scanner = null;
try {
fis = new FileInputStream(inputFile);
scanner = new Scanner(fis);
while(scanner.hasNextLine()) {
String line = scanner.nextLine();
Double[] features = doubleParseFeature(line);
list.add(features);
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} finally {
try {
scanner.close();
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return list;
}
private static Double[] doubleParseFeature(String feature) {
String[] features = feature.split(", ");
int length = features.length;
Double[] doubleFeatures = new Double[length];
for (int i = 0; i < length; i ++) {
doubleFeatures[i] = Double.parseDouble(features[i]);
}
return doubleFeatures;
}
}
src/main/java/edu/lu/uni/serval/MultipleThreadsParser2/DistanceCalculator.java
-68
View File
@@ -1,68 +0,0 @@
package edu.lu.uni.serval.MultipleThreadsParser2;
public class DistanceCalculator {
public static double calculateDistance(DistanceFunction distanceType, Double[] targetPoint, Double[] selfPoint) {
double distance = 0;
switch (distanceType) {
case EUCLIDEAN:
distance = DistanceCalculator.euclideanDistance(targetPoint, selfPoint);
break;
case COSINESIMILARITY:
distance = DistanceCalculator.cosineSimilarityDistance(targetPoint, selfPoint);
break;
case MANHATTAN:
distance = DistanceCalculator.manhattanDistance(targetPoint, selfPoint);
break;
default:
distance = DistanceCalculator.euclideanDistance(targetPoint, selfPoint);
break;
}
return distance;
}
public static double euclideanDistance(Double[] targetPoint, Double[] selfPoint) {
double sum = 0.0;
for (int i = 0, length = targetPoint.length; i < length; i++) {
double diff = targetPoint[i] - selfPoint[i];
sum += diff * diff;
}
return Math.sqrt(sum);
}
public static Double cosineSimilarityDistance(Double[] targetPoint, Double[] selfPoint) {
Double sim = 0.0d;
int length = targetPoint.length;
double dot = 0.0d;
double mag1 = 0.0d;
double mag2 = 0.0d;
for (int i = 0; i < length; i ++) {
dot += targetPoint[i] * selfPoint[i];
mag1 += Math.pow(targetPoint[i], 2);
mag2 += Math.pow(selfPoint[i], 2);
}
sim = dot / (Math.sqrt(mag1) * Math.sqrt(mag2));
return sim;
}
public static Double manhattanDistance(Double[] targetPoint, Double[] selfPoint) {
double result = 0.0;
for (int i = 0; i < targetPoint.length; i++) {
result += Math.abs(selfPoint[i] - targetPoint[i]);
}
return result;
}
public static Double minkowskiDistance(Double[] targetPoint, Double[] selfPoint) {
return null;
}
public static Double jaccardSimilarity(Double[] targetPoint, Double[] selfPoint) {
return null;
}
}
src/main/java/edu/lu/uni/serval/MultipleThreadsParser2/DistanceFunction.java
-5
View File
@@ -1,5 +0,0 @@
package edu.lu.uni.serval.MultipleThreadsParser2;
public enum DistanceFunction {
EUCLIDEAN, COSINESIMILARITY, MANHATTAN
}
src/main/java/edu/lu/uni/serval/MultipleThreadsParser2/MatchFixPatternActor.java
-172
View File
@@ -1,172 +0,0 @@
package edu.lu.uni.serval.MultipleThreadsParser2;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Scanner;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import akka.actor.ActorRef;
import akka.actor.Props;
import akka.actor.UntypedActor;
import akka.japi.Creator;
import akka.routing.RoundRobinPool;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.utils.FileHelper;
import edu.lu.uni.serval.utils.ListSorter;
import edu.lu.uni.serval.utils.MapSorter;
public class MatchFixPatternActor extends UntypedActor {
private static Logger logger = LoggerFactory.getLogger(MatchFixPatternActor.class);
private ActorRef mineRouter;
private final int numberOfWorkers;
private int counter = 0;
public MatchFixPatternActor(int numberOfWorkers) {
mineRouter = this.getContext().actorOf(new RoundRobinPool(numberOfWorkers)
.props(MatchFixPatternWorker.props()), "match-fix-pattern-router");
this.numberOfWorkers = numberOfWorkers;
}
public static Props props(final int numberOfWorkers) {
return Props.create(new Creator<MatchFixPatternActor>() {
private static final long serialVersionUID = 9207427376110704705L;
@Override
public MatchFixPatternActor create() throws Exception {
return new MatchFixPatternActor(numberOfWorkers);
}
});
}
@SuppressWarnings("deprecation")
@Override
public void onReceive(Object message) throws Exception {
if (message instanceof WorkMessage) {
WorkMessage msg = (WorkMessage) message;
List<Double[]> trainingFeatures = msg.getTrainingFeatures();
int size = trainingFeatures.size();
int average = size / numberOfWorkers;
this.bugId = msg.getId();
Double[] bugFeature = msg.getBugFeature();
for (int i = 0; i < numberOfWorkers; i ++) {
int fromIndex = i * average;
int toIndex = (i + 1) * average;
if (i == numberOfWorkers - 1) {
toIndex = size;
}
List<Double[]> subTrainingFeatures = new ArrayList<>();
subTrainingFeatures.addAll(trainingFeatures.subList(fromIndex, toIndex));
final WorkMessage workMsg = new WorkMessage(bugId, bugFeature, subTrainingFeatures);
workMsg.setNum(i + 1);
mineRouter.tell(workMsg, getSelf());
logger.info("Assign a task to worker #" + (i + 1) + "...");
}
} else if (message instanceof ReturnMessage) {
counter ++;
logger.info(counter + " workers finished their work...");
ReturnMessage rMsg = (ReturnMessage) message;
returnMessages.add(rMsg);
if (counter >= numberOfWorkers) {
ListSorter<ReturnMessage> sorter = new ListSorter<ReturnMessage>(returnMessages);
returnMessages = sorter.sortAscending();
Map<Integer, Double> similarities = new HashMap<>();
int index = 0;
for (int i = 0; i < numberOfWorkers; i ++) {
ReturnMessage returnMessage = returnMessages.get(i);
List<Double> similarity = returnMessage.getSimilarities();
for (int j = 0, size = similarity.size(); j < size; j ++) {
index ++;
similarities.put(index, similarity.get(j));
}
}
MapSorter<Integer, Double> mapSorter = new MapSorter<Integer, Double>();
Map<Integer, Double> sortedSimilarities = mapSorter.sortByValueDescending(similarities);
List<Integer> similarityList = new ArrayList<>();
double similarity = 0;
int num = 0;
for (Map.Entry<Integer, Double> entry : sortedSimilarities.entrySet()) {
if (entry.getValue() == similarity) {
continue;
}
similarityList.add(entry.getKey());
similarity = entry.getValue();
if (++ num % 100 == 0) {
break;
}
}
outputMatchedPatterns(similarityList);
logger.info("All workers finished their work...");
this.getContext().stop(mineRouter);
this.getContext().stop(getSelf());
this.getContext().system().shutdown();
}
} else {
unhandled(message);
}
}
private void outputMatchedPatterns(List<Integer> similarityList) {
String outputFile = Configuration.ROOT_PATH + "TestData/MatchedFixPatterns/Bug_" + bugId + ".list";
StringBuilder builder = new StringBuilder("BugId: " + bugId + "\n\n\n");
for (int i = 0, size = similarityList.size(); i < size; i ++) {
int patternPosition = similarityList.get(i);
builder.append(readPattern(patternPosition));
}
FileHelper.outputToFile(outputFile, builder, false);
}
private String readPattern(int patternPosition) {
String patternInfo = "";
FileInputStream fis = null;
Scanner scanner = null;
try {
fis = new FileInputStream(Configuration.SELECTED_PATCHES_SOURE_CODE_FILE);
scanner = new Scanner(fis);
int index = 0;
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
if (line.equals("PATCH###")) {
if (patternPosition == index) {
break;
}
index ++;
patternInfo = "";
}
patternInfo += line + "\n";
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} finally {
try {
scanner.close();
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return patternInfo;
}
private int bugId;
private List<ReturnMessage> returnMessages = new ArrayList<>();
}
src/main/java/edu/lu/uni/serval/MultipleThreadsParser2/MatchFixPatternWorker.java
-60
View File
@@ -1,60 +0,0 @@
package edu.lu.uni.serval.MultipleThreadsParser2;
import java.util.ArrayList;
import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import akka.actor.Props;
import akka.actor.UntypedActor;
import akka.japi.Creator;
public class MatchFixPatternWorker extends UntypedActor {
private static Logger log = LoggerFactory.getLogger(MatchFixPatternActor.class);
public MatchFixPatternWorker() {
}
public static Props props() {
return Props.create(new Creator<MatchFixPatternWorker>() {
private static final long serialVersionUID = -7615153844097275009L;
@Override
public MatchFixPatternWorker create() throws Exception {
return new MatchFixPatternWorker();
}
});
}
@Override
public void onReceive(Object message) throws Exception {
if (message instanceof WorkMessage) {
WorkMessage msg = (WorkMessage) message;
List<Double[]> trainingFeatures = msg.getTrainingFeatures();
Double[] bugFeature = msg.getBugFeature();
int bugID = msg.getId();
int workNum = msg.getNum();
List<Double> similarities = new ArrayList<>();
for (int i = 0, size = trainingFeatures.size(); i < size; i ++) {
Double similarity = Math.abs(computeSimilarity(bugFeature, trainingFeatures.get(i)));
similarities.add(similarity);
}
final ReturnMessage rMsg = new ReturnMessage(bugID, workNum, similarities);
log.info("Worker #" + workNum + " finished the work...");
this.getSender().tell(rMsg, getSelf());
} else {
unhandled(message);
}
}
private Double computeSimilarity(Double[] feature, Double[] trainingFeature) {
Double similarity = DistanceCalculator.cosineSimilarityDistance(trainingFeature, feature);
return similarity;
}
}
src/main/java/edu/lu/uni/serval/MultipleThreadsParser2/ReturnMessage.java
-35
View File
@@ -1,35 +0,0 @@
package edu.lu.uni.serval.MultipleThreadsParser2;
import java.util.List;
public class ReturnMessage implements Comparable<ReturnMessage>{
private int bugId;
private Integer workerId;
private List<Double> similarities;
public ReturnMessage(int bugId, int workerId, List<Double> similarities) {
super();
this.bugId = bugId;
this.workerId = workerId;
this.similarities = similarities;
}
public int getBugId() {
return bugId;
}
public int getWorkerId() {
return workerId;
}
public List<Double> getSimilarities() {
return similarities;
}
@Override
public int compareTo(ReturnMessage o) {
return this.workerId.compareTo(o.workerId);
}
}
src/main/java/edu/lu/uni/serval/MultipleThreadsParser2/WorkMessage.java
-39
View File
@@ -1,39 +0,0 @@
package edu.lu.uni.serval.MultipleThreadsParser2;
import java.util.List;
public class WorkMessage {
private int id;
private Double[] bugFeature;
private List<Double[]> trainingFeatures;
private int num;
public WorkMessage(int id, Double[] bugFeature, List<Double[]> trainingFeatures) {
super();
this.id = id;
this.bugFeature = bugFeature;
this.trainingFeatures = trainingFeatures;
}
public int getId() {
return id;
}
public Double[] getBugFeature() {
return bugFeature;
}
public List<Double[]> getTrainingFeatures() {
return trainingFeatures;
}
public int getNum() {
return num;
}
public void setNum(int num) {
this.num = num;
}
}
src/main/java/edu/lu/uni/serval/MultipleThreadsParser3/AkkaParser.java
-90
View File
@@ -1,90 +0,0 @@
package edu.lu.uni.serval.MultipleThreadsParser3;
import java.io.File;
import java.util.ArrayList;
import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import akka.actor.ActorRef;
import akka.actor.ActorSystem;
import edu.lu.uni.serval.MultipleThreadsParser.MessageFile;
import edu.lu.uni.serval.MultipleThreadsParser.WorkMessage;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.utils.FileHelper;
/**
* Multi-thread parser of parsing source code of unfixed violations.
*
* @author kui.liu
*
*/
public class AkkaParser {
private static Logger log = LoggerFactory.getLogger(AkkaParser.class);
@SuppressWarnings("deprecation")
public static void main(String[] args) {
String violationType = Configuration.GUM_TREE_INPUT + "UnfixedViolations/";
File file = new File(violationType);
File[] violationTypes = file.listFiles();
for (File violationT : violationTypes) {
if (violationT.isDirectory()) {
violationType = violationT.getName();
if (//violationType.equals("SF_SWITCH_NO_DEFAULT") ||
// violationType.equals("SE_NO_SERIALVERSIONID") ||
violationType.equals("DM_DEFAULT_ENCODING")) {
// input data
log.info("Get the input data...");
final List<MessageFile> msgFiles = getMessageFiles(Configuration.GUM_TREE_INPUT, violationType);
log.info("MessageFiles: " + msgFiles.size());
// output path
final String sourceCodeFilesPath = Configuration.ROOT_PATH + "UnfixedViolations_RQ3/" + violationType + "/";
FileHelper.deleteDirectory(sourceCodeFilesPath);
ActorSystem system = null;
ActorRef parsingActor = null;
int numberOfWorkers = 200;
final WorkMessage msg = new WorkMessage(0, msgFiles);
try {
log.info("Akka begins...");
system = ActorSystem.create("Mining-Pattern-System-" + violationType);
parsingActor = system.actorOf(ParseFixPatternActor.props(numberOfWorkers, sourceCodeFilesPath, violationType), "mine-pattern-actor-" + violationType);
parsingActor.tell(msg, ActorRef.noSender());
} catch (Exception e) {
system.shutdown();
e.printStackTrace();
}
}
}
}
}
/**
* Get violation-related files.
*
* @param gumTreeInput
* @return
*/
public static List<MessageFile> getMessageFiles(String inputPath, String violationType) {
File sourceCodeFilesPath = new File(inputPath + "UnfixedViolations/" + violationType + "/");
File[] sourceCodeFiles = sourceCodeFilesPath.listFiles(); // project folders
List<MessageFile> msgFiles = new ArrayList<>();
for (File sourceCodeFile : sourceCodeFiles) {
if (sourceCodeFile.getName().endsWith(".java")) {
String fileName = sourceCodeFile.getName();
fileName = fileName.substring(8).replace(".java", ".txt");
File positionFile = new File(inputPath + "UnFV_positions/" + violationType + "/" + fileName); // position file
MessageFile msgFile = new MessageFile(null, sourceCodeFile, null);
msgFile.setPositionFile(positionFile);
msgFiles.add(msgFile);
}
}
return msgFiles;
}
}
src/main/java/edu/lu/uni/serval/MultipleThreadsParser3/AkkaParser2.java
-78
View File
@@ -1,78 +0,0 @@
package edu.lu.uni.serval.MultipleThreadsParser3;
import java.io.File;
import java.util.ArrayList;
import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import akka.actor.ActorRef;
import akka.actor.ActorSystem;
import edu.lu.uni.serval.MultipleThreadsParser.MessageFile;
import edu.lu.uni.serval.MultipleThreadsParser.WorkMessage;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.utils.FileHelper;
/**
* Multi-thread parser of parsing source code of fixed violations.
*
* @author kui.liu
*
*/
public class AkkaParser2 {
private static Logger log = LoggerFactory.getLogger(AkkaParser2.class);
@SuppressWarnings("deprecation")
public static void main(String[] args) {
// input data
log.info("Get the input data...");
final List<MessageFile> msgFiles = getMessageFiles(Configuration.GUM_TREE_INPUT);
log.info("MessageFiles: " + msgFiles.size());
// output path
final String sourceCodeFilesPath = Configuration.ROOT_PATH + "fixedViolations/";
FileHelper.deleteDirectory(sourceCodeFilesPath);
ActorSystem system = null;
ActorRef parsingActor = null;
int numberOfWorkers = 431;
final WorkMessage msg = new WorkMessage(0, msgFiles);
try {
log.info("Akka begins...");
system = ActorSystem.create("Mining-Pattern-System");
parsingActor = system.actorOf(ParseFixPatternActor.props(numberOfWorkers, sourceCodeFilesPath, "Type"), "mine-pattern-actor");
parsingActor.tell(msg, ActorRef.noSender());
} catch (Exception e) {
system.shutdown();
e.printStackTrace();
}
}
/**
* Get violation-related files.
*
* @param gumTreeInput
* @return
*/
public static List<MessageFile> getMessageFiles(String inputPath) {
File sourceCodeFilesPath = new File(inputPath + "prevFiles/");
File[] sourceCodeFiles = sourceCodeFilesPath.listFiles();
List<MessageFile> msgFiles = new ArrayList<>();
for (File sourceCodeFile : sourceCodeFiles) {
if (sourceCodeFile.getName().endsWith(".java")) {
String fileName = sourceCodeFile.getName();
fileName = fileName.substring(5).replace(".java", ".txt");
File positionFile = new File(inputPath + "positions/" + fileName); // position file
MessageFile msgFile = new MessageFile(null, sourceCodeFile, null);
msgFile.setPositionFile(positionFile);
msgFiles.add(msgFile);
}
}
return msgFiles;
}
}
src/main/java/edu/lu/uni/serval/MultipleThreadsParser3/ParseFixPatternActor.java
-80
View File
@@ -1,80 +0,0 @@
package edu.lu.uni.serval.MultipleThreadsParser3;
import java.util.ArrayList;
import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import akka.actor.ActorRef;
import akka.actor.Props;
import akka.actor.UntypedActor;
import akka.japi.Creator;
import akka.routing.RoundRobinPool;
import edu.lu.uni.serval.MultipleThreadsParser.MessageFile;
import edu.lu.uni.serval.MultipleThreadsParser.WorkMessage;
public class ParseFixPatternActor extends UntypedActor {
private static Logger logger = LoggerFactory.getLogger(ParseFixPatternActor.class);
private ActorRef mineRouter;
private final int numberOfWorkers;
private int counter = 0;
public ParseFixPatternActor(int numberOfWorkers, String sourceCodeFilesPath, String vType) {
mineRouter = this.getContext().actorOf(new RoundRobinPool(numberOfWorkers)
.props(ParseFixPatternWorker.props(sourceCodeFilesPath)), "mine-pattern-router-" + vType);
this.numberOfWorkers = numberOfWorkers;
}
public static Props props(final int numberOfWorkers, final String sourceCodeFilesPath, final String vType) {
return Props.create(new Creator<ParseFixPatternActor>() {
private static final long serialVersionUID = 9207427376110704705L;
@Override
public ParseFixPatternActor create() throws Exception {
return new ParseFixPatternActor(numberOfWorkers, sourceCodeFilesPath, vType);
}
});
}
@SuppressWarnings("deprecation")
@Override
public void onReceive(Object message) throws Exception {
if (message instanceof WorkMessage) {
List<MessageFile> files = ((WorkMessage) message).getMsgFiles();
int size = files.size();
int average = size / numberOfWorkers;
for (int i = 0; i < numberOfWorkers; i ++) {
int fromIndex = i * average;
int toIndex = (i + 1) * average;
if (i == numberOfWorkers - 1) {
toIndex = size;
}
List<MessageFile> filesOfWorkers = new ArrayList<>();
filesOfWorkers.addAll(files.subList(fromIndex, toIndex));
final WorkMessage workMsg = new WorkMessage(i + 1, filesOfWorkers);
mineRouter.tell(workMsg, getSelf());
logger.info("Assign a task to worker #" + (i + 1) + "...");
}
} else if ("STOP".equals(message.toString())) {
counter ++;
logger.info(counter + " workers finished their work...");
if (counter >= numberOfWorkers) {
logger.info("All workers finished their work...");
this.getContext().stop(mineRouter);
this.getContext().stop(getSelf());
this.getContext().system().shutdown();
}
} else {
unhandled(message);
}
}
}
src/main/java/edu/lu/uni/serval/MultipleThreadsParser3/ParseFixPatternWorker.java
-122
View File
@@ -1,122 +0,0 @@
package edu.lu.uni.serval.MultipleThreadsParser3;
import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.StringReader;
import java.util.ArrayList;
import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import akka.actor.Props;
import akka.actor.UntypedActor;
import akka.japi.Creator;
import edu.lu.uni.serval.MultipleThreadsParser.MessageFile;
import edu.lu.uni.serval.MultipleThreadsParser.WorkMessage;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.utils.FileHelper;
import edu.lu.uni.serval.violation.code.parser.ViolationCodeParser;
public class ParseFixPatternWorker extends UntypedActor {
private static Logger log = LoggerFactory.getLogger(ParseFixPatternActor.class);
private String sourceCodeFilesPath;
public ParseFixPatternWorker(String sourceCodeFilesPath) {
this.sourceCodeFilesPath = sourceCodeFilesPath;
}
public static Props props(final String sourceCodeFilesPath) {
return Props.create(new Creator<ParseFixPatternWorker>() {
private static final long serialVersionUID = -7615153844097275009L;
@Override
public ParseFixPatternWorker create() throws Exception {
return new ParseFixPatternWorker(sourceCodeFilesPath);
}
});
}
@Override
public void onReceive(Object message) throws Exception {
if (message instanceof WorkMessage) {
WorkMessage msg = (WorkMessage) message;
List<MessageFile> files = msg.getMsgFiles();
StringBuilder sourceCode = new StringBuilder();
StringBuilder sizes = new StringBuilder();
StringBuilder tokens = new StringBuilder();
List<String> violationTypes = readTypes();
int id = msg.getId();
int counter = 0;
for (MessageFile msgFile : files) {
File prevFile = msgFile.getPrevFile();
File positionFile = msgFile.getPositionFile();
if (prevFile.getName().toLowerCase().contains("test")) {
continue;
}
ViolationCodeParser parser = new ViolationCodeParser();
parser.parse(prevFile, positionFile);
parser.setTypes(violationTypes);
String sourceCodeStr = parser.sourceCode;
if ("".equals(sourceCodeStr)) {
} else {
sourceCode.append(sourceCodeStr);
sizes.append(parser.sizes);
tokens.append(parser.tokens);
counter ++;
if (counter % 100 == 0) {
FileHelper.outputToFile(sourceCodeFilesPath + "SourceCode/worker_" + id + ".list", sourceCode, true);
FileHelper.outputToFile(sourceCodeFilesPath + "Sizes/worker_" + id + ".list", sizes, true);
FileHelper.outputToFile(sourceCodeFilesPath + "Tokens/worker_" + id + ".list", tokens, true);
sourceCode.setLength(0);
sizes.setLength(0);
tokens.setLength(0);
}
}
}
if (sizes.length() > 0) {
FileHelper.outputToFile(sourceCodeFilesPath + "SourceCode/worker_" + id + ".list", sourceCode, true);
FileHelper.outputToFile(sourceCodeFilesPath + "Sizes/worker_" + id + ".list", sizes, true);
FileHelper.outputToFile(sourceCodeFilesPath + "Tokens/worker_" + id + ".list", tokens, true);
sourceCode.setLength(0);
sizes.setLength(0);
tokens.setLength(0);
}
log.info("Worker #" + id +"Finish of parsing " + counter + " files...");
log.info("Worker #" + id + " finished the work...");
this.getSender().tell("STOP", getSelf());
} else {
unhandled(message);
}
}
private List<String> readTypes() {
String fileName = Configuration.ROOT_PATH + "fixedViolations/types.list";
String content = FileHelper.readFile(fileName);
List<String> types = new ArrayList<>();
BufferedReader reader = new BufferedReader(new StringReader(content));
try {
String line = null;
while ((line = reader.readLine()) != null) {
types.add(line);
}
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
// System.err.println(types.size());
return types;
}
}
src/main/java/edu/lu/uni/serval/TreeToString.java
-47
View File
@@ -1,47 +0,0 @@
package edu.lu.uni.serval;
import java.util.ArrayList;
import java.util.List;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.FixPattern.utils.ASTNodeMap;
public class TreeToString {
private List<String> strList = new ArrayList<>();
public String toString(ITree tree) {
List<ITree> children = tree.getChildren();
String str = ASTNodeMap.map.get(tree.getType());
if (strList.size() == 0) {
strList.add(str);
for (ITree child : children) {
TreeToString t2s = new TreeToString();
t2s.toString(child);
List<String> strList1 = t2s.strList;
for (String str1 : strList1) {
strList.add("---" + str1);
}
}
} else {
strList.clear();
strList.add(str);
for (ITree child : children) {
TreeToString t2s = new TreeToString();
t2s.toString(child);
List<String> strList1 = t2s.strList;
for (String str1 : strList1) {
strList.add("---" + str1);
}
}
}
str = "";
for (String str1 : strList) {
str += str1 + "\n";
}
return str;
}
}
src/main/java/edu/lu/uni/serval/config/Configuration.java
-84
View File
@@ -4,88 +4,4 @@ public class Configuration {
public static final long SECONDS_TO_WAIT = 900L;
// public static String ROOT_PATH = "/Volumes/data/bugStudy/dataset/"; // The root path of all output data.
public static String ROOT_PATH = "/Users/anilkoyuncu/bugStudy/dataset/"; // The root path of all output data.
public static int HUNK_SIZE = 10; // The limitation of source code lines of each DiffEntry, which will be selected as training data.
public static final String BUGGY_TREE_SIGNAL = "BUGGY_TREE###"; // The starting signal of the tree of buggy source code .
public static final String PATCH_SIGNAL = "PATCH###"; // Th starting signal of each patch.
// input path of GumTree. (i.e., Fix patterns parser)
public static final String GUM_TREE_INPUT = ROOT_PATH + "GumTreeInput/";// Buggy version file VS. Fixing version file, (DiffEntry File)
// the output path of GumTree results.
public static final String GUM_TREE_OUTPUT = ROOT_PATH + "GumTreeResults/";
public static final String EDITSCRIPTS_FILE_PATH = GUM_TREE_OUTPUT + "editScripts/";
public static final String PATCH_SOURCECODE_FILE_PATH = GUM_TREE_OUTPUT + "sourceCode/";
public static final String BUGGYTREE_FILE_PATH = GUM_TREE_OUTPUT + "buggyTrees/";
public static final String BUGGY_CODE_TOKEN_FILE_PATH = GUM_TREE_OUTPUT + "tokens/";
public static final String EDITSCRIPT_SIZES_FILE_PATH = GUM_TREE_OUTPUT + "editScriptSizes/";
public static final String ALARM_TYPES_FILE_PATH = GUM_TREE_OUTPUT + "alarmTypes/";
public static final String EDITSCRIPTS_FILE = GUM_TREE_OUTPUT + "editScripts.list";
public static final String PATCH_SOURCECODE_FILE = GUM_TREE_OUTPUT + "patchSourceCode.list";
public static final String BUGGYTREES_FILE = GUM_TREE_OUTPUT + "buggyTrees.list";
public static final String BUGGY_CODE_TOKENS_FILE = GUM_TREE_OUTPUT + "tokens.list";
public static final String EDITSCRIPT_SIZES_FILE = GUM_TREE_OUTPUT + "editScriptSizes.csv";
public static final String ALARM_TYPES_FILE = GUM_TREE_OUTPUT + "alarmTypes.list";
public static final int VECTOR_SIZE_OF_EMBEDED_TOKEN1 = 100; // tokens of edit scripts.
public static final int VECTOR_SIZE_OF_EMBEDED_TOKEN2 = 200; // tokens of source code
// the input path of fix patterns mining.
private static final String MINING_INPUT = ROOT_PATH + "MiningInput/";
public static final String MAX_TOKEN_VECTORS_SIZE_OF_EDIT_SCRIPTS = MINING_INPUT + "/MaxTokenVectorSizeOfEditScripts.list"; // The max size of edit scripts: upper limitation of max size.
public static final String MAX_TOKEN_VECTORS_SIZE_OF_SOURCE_CODE = MINING_INPUT + "/MaxTokenVectorSizeOfBuggySourceCode.list"; // The max size of all buggy source code token vectors.
// the input path of token embedding.
public static final String EMBEDDING_INPUT = MINING_INPUT + "Embedding/";
public static final String SELECTED_PATCHES_SOURE_CODE_FILE = EMBEDDING_INPUT + "patchSourceCode.list";// Selected patches.
public static final String SELECTED_BUGGY_TREE_FILE = EMBEDDING_INPUT + "buggyTrees.list";
public static final String SELECTED_BUGGY_TOKEN_FILE = EMBEDDING_INPUT + "tokens.list"; // Selected token vectors of buggy source code.
public static final String SELECTED_EDITSCRIPTES_FILE = EMBEDDING_INPUT + "editScripts.list"; // Selected edit script vectors.
public static final String SELECTED_ALARM_TYPES_FILE = EMBEDDING_INPUT + "alarmTypes.list"; // Selected edit script vectors.
// the input path of feature learning.
public static final String FEATURE_LEARNING_INPUT = MINING_INPUT + "FeatureLearning/";
public static final String EMBEDDED_EDIT_SCRIPT_TOKENS = FEATURE_LEARNING_INPUT + "embeddedEditScriptTokens.list"; // All embedded tokens of selected edit scripts.
public static final String VECTORIED_EDIT_SCRIPTS = FEATURE_LEARNING_INPUT + "vectorizedEditScripts.csv"; // Embedded and vectorized edit script vectors.
// the input path of clustering.
public static final String EXTRACTED_FEATURES = MINING_INPUT + "ExtractedFeatures/"; // Extracted features of all edit scripts.
public static final String CLUSTER_INPUT = MINING_INPUT + "ClusteringInput/input.arff";
// the output path of fix patterns mining.
private static final String MINING_OUTPUT = ROOT_PATH + "MiningOutput/";
public static final String CLUSTER_OUTPUT = MINING_OUTPUT + "ClusteringOutput/clusterResults.list";
public static final String CLUSTERED_PATCHES_FILE = MINING_OUTPUT + "ClusteredPatches/";
public static final String CLUSTERED_TOKENSS_FILE = MINING_OUTPUT + "ClusteredTokens/"; // Token vectors of buggy source code.
public static final String COMMON_CLUSTERS_SIZES = MINING_OUTPUT + "ClusteringOutput/CommonClusterSizes.list";
// evaluation data
public static final String TEST_INPUT = ROOT_PATH + "TestProjects/";
public static final String TEST_POSITION_FILE = ROOT_PATH + "TestData/Positions/"; // Positions of all test statements.
public static final String TEST_DATA_FILE = ROOT_PATH + "TestData/TestStatements/"; // Token vectors of all test statements.
public static final String NUMBER_OF_TRAINING_DATA = ROOT_PATH + "TestData/NumberOfTrainingData.list";;
// data of unsupervised learning
public static final String EMBEDDING_DATA_TOKENS1 = ROOT_PATH + "TestData/AllTokenVectorsForEvaluation.list";
public static final String EMBEDDED_ALL_TOKENS1 = ROOT_PATH + "TestData/AllEmbeddedTokens.list";
public static final String VECTORIED_ALL_SOURCE_CODE1 = ROOT_PATH + "TestData/AllVectorizedSourceCode/";
public static final String EXTRACTED_FEATURES_EVALUATION = ROOT_PATH + "TestDataExtractedFeatures/"; // extracted features of all source code (training data and testing data)
// Data of supervised learning
public static final String CLUSTERNUMBER_LABEL_MAP = ROOT_PATH + "TestData/clusterMappingLabel.list";
public static final String EMBEDDING_DATA_TOKENS2 = ROOT_PATH + "TestData/AllTokenVectorsForSupervisedEvaluation.list";
public static final String EMBEDDED_ALL_TOKENS2 = ROOT_PATH + "TestData/AllEmbeddedTokensForSuperVisedEvaluation.list";
public static final String TRAINING_DATA = ROOT_PATH + "TestData/TrainingData.csv"; // Training data of supervised learning
public static final String TESTING_DATA = ROOT_PATH + "TestData/SupervisedLearning/"; // testing data of supervised learning
public static final String FEATURES_OF_TRAINING_DATA = ROOT_PATH + "TestingOutput/TraingFeatures/";
public static final String FEATURES_OF_TESTING_DATA = ROOT_PATH + "TestingOutput/TestingFeatures/";
public static final String POSSIBILITIES_OF_TESTING_DATA = ROOT_PATH + "TestingOutput/Posibilities/";
public static final String PREDICTED_RESULTS_OF_TESTING_DATA = ROOT_PATH + "TestingOutput/Prediction/";
public static final String SUPERVISED_LEARNING_MODEL = ROOT_PATH + "TestingOutput/SupervisedLearningModel.zip";
public static final String FEATURES_OF_COMMON_CLUSTERS = ROOT_PATH + "FeaturesOfCommonClusters/";
}
src/main/java/edu/lu/uni/serval/defects4j/Bug.java
-43
View File
@@ -1,43 +0,0 @@
package edu.lu.uni.serval.defects4j;
public class Bug {
private String project;
private String fileName;
private int startLine;
private int endLine;
private String type;
public Bug(String project, String fileName, int startLine, int endLine, String type) {
super();
this.project = project;
this.fileName = fileName;
this.startLine = startLine;
this.endLine = endLine;
this.type = type;
}
public String getProject() {
return project;
}
public String getFileName() {
return fileName;
}
public int getStartLine() {
return startLine;
}
public int getEndLine() {
return endLine;
}
public String getType() {
return type;
}
}
src/main/java/edu/lu/uni/serval/defects4j/BugParser.java
-206
View File
@@ -1,206 +0,0 @@
package edu.lu.uni.serval.defects4j;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.StringReader;
import java.util.ArrayList;
import java.util.List;
import java.util.Scanner;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.FixPatternParser.Tokenizer;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.gumtree.regroup.SimpleTree;
import edu.lu.uni.serval.gumtree.regroup.SimplifyTree;
import edu.lu.uni.serval.utils.FileHelper;
import edu.lu.uni.serval.violation.code.parser.ViolationSourceCodeTree;
public class BugParser {
public static void main(String[] args) throws IOException {
List<Bug> bugs = new ArrayList<>();
String bugsFile = "Dataset/Defects4j/Bugs.txt";
String content = FileHelper.readFile(bugsFile);
BufferedReader reader = new BufferedReader(new StringReader(content));
String line = null;
StringBuilder tokensBuilder = new StringBuilder();
StringBuilder buggyCodeBuilder = new StringBuilder();
while ((line = reader.readLine()) != null) {
String[] elements = line.split(" : ");
String type = elements[0];
String project = elements[1];
project = project.substring(0, project.lastIndexOf("."));
String fileName = elements[2];
int startLine = Integer.parseInt(elements[3]);
int endLine = Integer.parseInt(elements[4]);
Bug bug = new Bug(project, fileName, startLine, endLine, type);
bugs.add(bug);
}
int i = 0;
for (Bug bug : bugs) {
String type = bug.getType();
String project = bug.getProject();
String fileName = bug.getFileName();
int startLine = bug.getStartLine();
int endLine = bug.getEndLine();
File buggyFile = null;
String[] elements = project.split("_");
if (elements.length < 2) System.out.println(type + "-" + project + "-" + fileName + "-" + startLine + "-" + endLine);
List<File> javaFiles = FileHelper.getAllFiles(Configuration.ROOT_PATH + "Testing/projects1/" + elements[0] + "/" + elements[1] + "-b", ".java");
for (File javaFile : javaFiles) {
if (javaFile.getPath().endsWith(fileName)) {
buggyFile = javaFile;
break;
}
}
if (i == 31) {
i = i + 1 - 1;
}
SimpleTree simpleTree = null;
if ("EQ_DOESNT_OVERRIDE_EQUALS".equals(type)|| "HE_EQUALS_USE_HASHCODE".equals(type) || "HE_INHERITS_EQUALS_USE_HASHCODE".equals(type)||
"SE_NO_SUITABLE_CONSTRUCTOR".equals(type) || "RI_REDUNDANT_INTERFACES".equals(type)
// ||"CN_IDIOM_NO_SUPER_CALL".equals(type)
||"SE_NO_SERIALVERSIONID".equals(type) || "SE_COMPARATOR_SHOULD_BE_SERIALIZABLE".equals(type)) {
ViolationSourceCodeTree parser = new ViolationSourceCodeTree(buggyFile, startLine, endLine);
ITree classNameTree = parser.getClassNameTokens();
simpleTree = new SimplifyTree().canonicalizeSourceCodeTree(classNameTree, null);
startLine = parser.getViolationFinalStartLine();
endLine = parser.getViolationFinalEndLine();
} else {
ViolationSourceCodeTree alarmTree = new ViolationSourceCodeTree(buggyFile, startLine, endLine);
alarmTree.extract();
List<ITree> matchedTrees = alarmTree.getViolationSourceCodeTrees();
if (matchedTrees.size() == 0) {
System.err.println("#Null_Violation_Hunk: " + buggyFile.getName() + ":" + startLine + ":" + endLine);
continue;
}
simpleTree = new SimpleTree();
simpleTree.setLabel("Block");
simpleTree.setNodeType("Block");
List<SimpleTree> children = new ArrayList<>();
for (ITree matchedTree : matchedTrees) {
SimpleTree simpleT = new SimplifyTree().canonicalizeSourceCodeTree(matchedTree, null);
children.add(simpleT);
}
simpleTree.setChildren(children);
startLine = alarmTree.getViolationFinalStartLine();
endLine = alarmTree.getViolationFinalEndLine();
}
String tokens = Tokenizer.getTokensDeepFirst(simpleTree);
String[] tokensArray = tokens.split(" ");
int length = tokensArray.length;
System.out.println((++ i) + "==" + length);
// sizes += length + "\n";
// this.tokens += tokens + "\n";
String sourceCode = readSourceCode(buggyFile, startLine, endLine, type);
// this.sourceCode += sourceCode + "\n";
// tokensBuilder.append(type).append(":").append(tokens).append("\n");
tokensBuilder.append(tokens).append("\n");
buggyCodeBuilder.append(sourceCode).append("\n");
}
FileHelper.outputToFile("Dataset/Defects4j/buggyTokens.list", tokensBuilder, false);
FileHelper.outputToFile("Dataset/Defects4j/buggySourceCode.list", buggyCodeBuilder, false);
}
private static String readSourceCode(File javaFile, int startLine, int endLine, String violationType) {
StringBuilder sourceCode = new StringBuilder("##Source_Code:\n");
sourceCode.append(violationType).append("\n");
sourceCode.append(javaFile.getName().replaceAll("#", "/")).append("\nPosition: ").append(startLine).append(" : ").append(endLine).append("\n");
FileInputStream fis = null;
Scanner scanner = null;
try {
fis = new FileInputStream(javaFile);
scanner = new Scanner(fis);
int counter = 0;
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
counter ++;
if (startLine <= counter && counter <= endLine) {
sourceCode.append(line + "\n");
}
if (counter == endLine) break;
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} finally {
try {
scanner.close();
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sourceCode.toString();
}
}
/*
20
32
20
8
8
16
22
10
10
10
12
16
18
10
12
8
14
44
8
26
20
10
12
12
10
4
10
8
8
12
16
1776
8
10
10
12
8
50
18
8
12
38
4
4
14
16
20
26
8
14
12
14
*/
src/main/java/edu/lu/uni/serval/diffentry/DiffEntryHunk.java
-85
View File
@@ -1,85 +0,0 @@
package edu.lu.uni.serval.diffentry;
import java.util.ArrayList;
import java.util.List;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
public class DiffEntryHunk {
private int bugLineStartNum;
private int fixLineStartNum;
private int bugRange;
private int fixRange;
private String hunk;
private int buggyHunkSize;
private int fixedHunkSize;
private String file;
private List<HierarchicalActionSet> actionSets = new ArrayList<>();
public DiffEntryHunk(int bugLineStartNum, int fixLineStartNum, int bugRange, int fixRange) {
super();
this.bugLineStartNum = bugLineStartNum;
this.fixLineStartNum = fixLineStartNum;
this.bugRange = bugRange;
this.fixRange = fixRange;
}
public int getBugLineStartNum() {
return bugLineStartNum;
}
public int getFixLineStartNum() {
return fixLineStartNum;
}
public int getBugRange() {
return bugRange;
}
public int getFixRange() {
return fixRange;
}
public String getHunk() {
return hunk;
}
public void setHunk(String hunk) {
this.hunk = hunk;
}
public int getBuggyHunkSize() {
return buggyHunkSize;
}
public void setBuggyHunkSize(int buggyHunkSize) {
this.buggyHunkSize = buggyHunkSize;
}
public int getFixedHunkSize() {
return fixedHunkSize;
}
public void setFixedHunkSize(int fixedHunkSize) {
this.fixedHunkSize = fixedHunkSize;
}
public String getFile() {
return file;
}
public void setFile(String file) {
this.file = file;
}
public List<HierarchicalActionSet> getActionSets() {
return actionSets;
}
@Override
public String toString() {
return "@@ -" + this.bugLineStartNum + ", " + this.bugRange + " +" + this.fixLineStartNum + ", " + this.fixRange + "\n" + this.hunk;
}
}
src/main/java/edu/lu/uni/serval/diffentry/DiffEntryReader.java
-250
View File
@@ -1,250 +0,0 @@
package edu.lu.uni.serval.diffentry;
import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.StringReader;
import java.util.ArrayList;
import java.util.List;
import java.util.regex.Pattern;
import edu.lu.uni.serval.utils.FileHelper;
public class DiffEntryReader {
public List<DiffEntryHunk> readHunks(File diffentryFile) {
List<DiffEntryHunk> diffentryHunks = new ArrayList<>();
String content = FileHelper.readFile(diffentryFile);
BufferedReader reader = null;
try {
reader = new BufferedReader(new StringReader(content));
String line = null;
int startLine = 0;
int range = 0;
int startLine2 = 0;
int range2 = 0;
StringBuilder hunk = new StringBuilder();
boolean sourceCode = false;
while ((line = reader.readLine()) != null) {
if (RegExp.filterSignal(line.trim())) {
sourceCode = true;
if (hunk.length() > 0) {
if ((range < 7 && range2 < 7) || range == 0 || range2 == 0) { // filter out big hunks
DiffEntryHunk diffEntryHunk = new DiffEntryHunk(startLine, startLine2, range, range2);
diffEntryHunk.setHunk(hunk.toString());
diffentryHunks.add(diffEntryHunk);
}
hunk.setLength(0);
}
int plusIndex = line.indexOf("+");
String lineNum = line.substring(4, plusIndex);
String[] nums = lineNum.split(",");
startLine = Integer.parseInt(nums[0].trim());
if (nums.length == 2) {
range = Integer.parseInt(nums[1].trim());
}
String lineNum2 = line.substring(plusIndex) .trim();
lineNum2 = lineNum2.substring(1, lineNum2.length() - 2);
String[] nums2 = lineNum2.split(",");
startLine2 = Integer.parseInt(nums2[0].trim());
if (nums2.length == 2) {
range2 = Integer.parseInt(nums2[1].trim());
}
continue;
}
if (sourceCode) hunk.append(line + "\n");
}
if ((range < 7 && range2 < 7) || range == 0 || range2 == 0) { // filter out big hunks
DiffEntryHunk diffEntryHunk = new DiffEntryHunk(startLine, startLine2, range, range2);
diffEntryHunk.setHunk(hunk.toString());
diffentryHunks.add(diffEntryHunk);
}
hunk.setLength(0);
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (reader != null) {
reader.close();
reader = null;
}
} catch (IOException e) {
e.printStackTrace();
}
}
return diffentryHunks;
}
/**
* Read all hunks without considering their sizes.
*
* @param diffentryFile
* @return
*/
public List<DiffEntryHunk> readHunks2(File diffentryFile) {
List<DiffEntryHunk> diffentryHunks = new ArrayList<>();
String content = FileHelper.readFile(diffentryFile);
BufferedReader reader = null;
try {
reader = new BufferedReader(new StringReader(content));
String line = null;
int startLine = 0;
int range = 0;
int startLine2 = 0;
int range2 = 0;
StringBuilder hunk = new StringBuilder();
boolean sourceCode = false;
while ((line = reader.readLine()) != null) {
if (RegExp.filterSignal(line.trim())) {
// line = Pattern.compile("^@@\\s\\-\\d+,*\\d*\\s\\+\\d+,*\\d*\\s@@").split(line)[1];
sourceCode = true;
if (hunk.length() > 0) {
if (startLine > 0) {
DiffEntryHunk diffEntryHunk = new DiffEntryHunk(startLine, startLine2, range, range2);
diffEntryHunk.setHunk(hunk.toString());
diffentryHunks.add(diffEntryHunk);
}
hunk.setLength(0);
}
int plusIndex = line.indexOf("+");
String lineNum = line.substring(4, plusIndex);
String[] nums = lineNum.split(",");
startLine = Integer.parseInt(nums[0].trim());
if (nums.length == 2) {
range = Integer.parseInt(nums[1].trim());
}
String lineNum2 = line.substring(plusIndex) .trim();
lineNum2 = lineNum2.substring(1, lineNum2.length() - 2);
String[] nums2 = lineNum2.split(",");
startLine2 = Integer.parseInt(nums2[0].trim());
if (nums2.length == 2) {
range2 = Integer.parseInt(nums2[1].trim());
}
continue;
}
if (sourceCode) hunk.append(line + "\n");
}
if (startLine > 0) {
DiffEntryHunk diffEntryHunk = new DiffEntryHunk(startLine, startLine2, range, range2);
diffEntryHunk.setHunk(hunk.toString());
diffentryHunks.add(diffEntryHunk);
}
hunk.setLength(0);
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (reader != null) {
reader.close();
reader = null;
}
} catch (IOException e) {
e.printStackTrace();
}
}
return diffentryHunks;
}
/**
* Read all hunks with pure deleted lines and added lines.
*
* @param diffentryFile
* @return
*/
public List<DiffEntryHunk> readHunks3(File diffentryFile) {
List<DiffEntryHunk> diffentryHunks = new ArrayList<>();
String content = FileHelper.readFile(diffentryFile);
BufferedReader reader = null;
try {
reader = new BufferedReader(new StringReader(content));
String line = null;
int buggyStartLine = 0;
int buggyRange = 0;
int fixedStartLine = 0;
int fixedRange = 0;
int buggyHunkSize = 0;
int fixedHunkSize = 0;
StringBuilder hunk = new StringBuilder();
boolean sourceCode = false;
while ((line = reader.readLine()) != null) {
if (RegExp.filterSignal(line.trim())) {
sourceCode = true;
if (hunk.length() > 0) {
if (buggyStartLine > 0) {
DiffEntryHunk diffEntryHunk = new DiffEntryHunk(buggyStartLine, fixedStartLine, buggyRange, fixedRange);
diffEntryHunk.setHunk(hunk.toString());
diffEntryHunk.setBuggyHunkSize(buggyHunkSize);
diffEntryHunk.setFixedHunkSize(fixedHunkSize);
diffEntryHunk.setFile(diffentryFile.getPath());
diffentryHunks.add(diffEntryHunk);
}
hunk.setLength(0);
buggyStartLine = 0;
buggyRange = 0;
fixedStartLine = 0;
fixedRange = 0;
buggyHunkSize = 0;
fixedHunkSize = 0;
}
int plusIndex = line.indexOf("+");
String lineNum = line.substring(4, plusIndex);
String[] nums = lineNum.split(",");
buggyStartLine = Integer.parseInt(nums[0].trim());
if (nums.length == 2) {
buggyRange = Integer.parseInt(nums[1].trim());
}
String lineNum2 = line.substring(plusIndex) .trim();
lineNum2 = lineNum2.substring(1, lineNum2.length() - 2);
String[] nums2 = lineNum2.split(",");
fixedStartLine = Integer.parseInt(nums2[0].trim());
if (nums2.length == 2) {
fixedRange = Integer.parseInt(nums2[1].trim());
}
continue;
} else if (sourceCode) {
if (line.startsWith("-")) {
buggyHunkSize ++;
} else if (line.startsWith("+")) {
fixedHunkSize ++;
}
hunk.append(line + "\n");
}
}
if (buggyStartLine > 0 && hunk.length() > 0) {
DiffEntryHunk diffEntryHunk = new DiffEntryHunk(buggyStartLine, fixedStartLine, buggyRange, fixedRange);
diffEntryHunk.setHunk(hunk.toString());
diffEntryHunk.setBuggyHunkSize(buggyHunkSize);
diffEntryHunk.setFixedHunkSize(fixedHunkSize);
diffEntryHunk.setFile(diffentryFile.getPath());
diffentryHunks.add(diffEntryHunk);
hunk.setLength(0);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (reader != null) {
reader.close();
reader = null;
}
} catch (IOException e) {
e.printStackTrace();
}
}
return diffentryHunks;
}
}
src/main/java/edu/lu/uni/serval/diffentry/RegExp.java
-20
View File
@@ -1,20 +0,0 @@
package edu.lu.uni.serval.diffentry;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class RegExp {
private static final String REGULAR_EXPRESSION = "^@@\\s\\-\\d+,*\\d*\\s\\+\\d+,*\\d*\\s@@$"; //@@ -21,0 +22,2 @@
private static Pattern pattern = Pattern.compile(REGULAR_EXPRESSION);
public static boolean filterSignal(String string) {
boolean flag = false;
Matcher res = pattern.matcher(string);
if (res.matches()) {
flag = true;
}
return flag;
}
}
src/main/java/edu/lu/uni/serval/FixPatternParser/violations/TestHunkParser.java → src/main/java/edu/lu/uni/serval/fixminer/EnhancedASTDiff.java
+8 -39
View File
@@ -1,4 +1,4 @@
package edu.lu.uni.serval.FixPatternParser.violations;
package edu.lu.uni.serval.fixminer;
import akka.actor.ActorRef;
import akka.actor.ActorSystem;
@@ -16,10 +16,10 @@ import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
public class TestHunkParser {
public class EnhancedASTDiff {
private static Logger log = LoggerFactory.getLogger(EnhancedASTDiff.class);
private static Logger log = LoggerFactory.getLogger(TestHunkParser.class);
// public static void main(String[] args) {
public static void main(String inputPath, String outputPath,String numOfWorkers,String project) {
@@ -44,29 +44,13 @@ public class TestHunkParser {
continue;
String GUM_TREE_OUTPUT = outputPath + "/"+ pjName + "/";
// final String editScriptsFilePath = GUM_TREE_OUTPUT + "editScripts.list";
// final String patchesSourceCodeFilePath =GUM_TREE_OUTPUT + "patchSourceCode.list";
// final String buggyTokensFilePath = GUM_TREE_OUTPUT + "tokens.list";
// final String editScriptSizesFilePath = GUM_TREE_OUTPUT + "editScriptSizes.csv";
// final String alarmTypesFilePath = GUM_TREE_OUTPUT + "alarmTypes.list";
FileHelper.createDirectory(GUM_TREE_OUTPUT + "/UPD");
FileHelper.createDirectory(GUM_TREE_OUTPUT + "/INS");
FileHelper.createDirectory(GUM_TREE_OUTPUT + "/DEL");
FileHelper.createDirectory(GUM_TREE_OUTPUT + "/MOV");
FileHelper.createDirectory(GUM_TREE_OUTPUT + "/ALL");
// FileHelper.deleteDirectory(editScriptsFilePath);
// FileHelper.deleteDirectory(patchesSourceCodeFilePath);
// FileHelper.deleteDirectory(buggyTokensFilePath);
// FileHelper.deleteDirectory(editScriptSizesFilePath);
// FileHelper.deleteDirectory(alarmTypesFilePath);
// StringBuilder astEditScripts = new StringBuilder();
// StringBuilder tokens = new StringBuilder();
// StringBuilder sizes = new StringBuilder();
// StringBuilder patches = new StringBuilder();
// StringBuilder alarmTypes = new StringBuilder();
int a = 0;
@@ -76,7 +60,7 @@ public class TestHunkParser {
try {
log.info("Akka begins...");
system = ActorSystem.create("Mining-FixPattern-System");
System.out.println(system.settings());
parsingActor = system.actorOf(ParseFixPatternActor.props(Integer.valueOf(numOfWorkers), project), "mine-fix-pattern-actor");
parsingActor.tell(msg, ActorRef.noSender());
} catch (Exception e) {
@@ -84,20 +68,6 @@ public class TestHunkParser {
e.printStackTrace();
}
// FileHelper.outputToFile(editScriptsFilePath, astEditScripts, true);
// FileHelper.outputToFile(buggyTokensFilePath, tokens, true);
// FileHelper.outputToFile(editScriptSizesFilePath, sizes, true);
// FileHelper.outputToFile(patchesSourceCodeFilePath, patches, true);
// FileHelper.outputToFile(alarmTypesFilePath, alarmTypes, true);
// astEditScripts.setLength(0);
// tokens.setLength(0);
// sizes.setLength(0);
// patches.setLength(0);
// alarmTypes.setLength(0);
// System.out.println(a);
// classifyByAlarmTypes();
}
}
@@ -109,16 +79,15 @@ public class TestHunkParser {
List<MessageFile> msgFiles = new ArrayList<>();
if (revFiles.length >= 0) {
for (File revFile : revFiles) {
// if (revFile.getName().endsWith(".java")) {
String fileName = revFile.getName();
File prevFile = new File(gumTreeInput + "prevFiles/prev_" + fileName);// previous file
fileName = fileName.replace(".java", ".txt");
File diffentryFile = new File(gumTreeInput + "DiffEntries/" + fileName); // DiffEntry file
// File positionFile = new File(gumTreeInput + "positions/" + fileName); // position file
MessageFile msgFile = new MessageFile(revFile, prevFile, diffentryFile);
// msgFile.setPositionFile(positionFile);
msgFiles.add(msgFile);
// }
}
return msgFiles;
src/main/java/edu/lu/uni/serval/fixminer/Launcher.java
+32 -41
View File
@@ -1,6 +1,5 @@
package edu.lu.uni.serval.fixminer;
import edu.lu.uni.serval.FixPatternParser.violations.TestHunkParser;
import edu.lu.uni.serval.fixminer.cluster.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -27,34 +26,34 @@ public class Launcher {
appProps.load(new FileInputStream(appConfigPath));
String portInner = appProps.getProperty("portInner","6380");
String serverWait = appProps.getProperty("serverWait", "50000");
// String serverWait = appProps.getProperty("serverWait", "50000");
String numOfWorkers = appProps.getProperty("numOfWorkers", "10");
String jobType = appProps.getProperty("jobType","ALL");
String portDumps = appProps.getProperty("portDumps","6399");
String pythonPath = appProps.getProperty("pythonPath","/Users/anilkoyuncu/bugStudy/code/python");
String datasetPath = appProps.getProperty("datasetPath","/Users/anilkoyuncu/bugStudy/dataset");
String pjName = appProps.getProperty("pjName","allDataset");
String dbNo = appProps.getProperty("dbNo","0");
// String dbNo = appProps.getProperty("dbNo","0");
String actionType = appProps.getProperty("actionType","ALL");
String threshold = appProps.getProperty("threshold","1");
String cursor = appProps.getProperty("cursor","10000000");
String chunk = appProps.getProperty("chunk","1.txt");
String parameters = String.format("\nportInner %s " +
"\nserverWait %s \nnumOfWorkers %s " +
"\nnumOfWorkers %s " +
"\njobType %s \nport %s " +
"\npythonPath %s \ndatasetPath %s" +
"\npjName %s \ndbNo %s \nactionType %s \nthreshold %s \ncursor %s"
, portInner, serverWait, numOfWorkers, jobType, portDumps, pythonPath,datasetPath,pjName,dbNo,actionType,threshold,cursor);
"\npjName %s \nactionType %s \nthreshold %s \ncursor %s"
, portInner, numOfWorkers, jobType, portDumps, pythonPath,datasetPath,pjName,actionType,threshold,cursor);
log.info(parameters);
mainLaunch(portInner, serverWait, numOfWorkers, jobType, portDumps, pythonPath,datasetPath,pjName,dbNo,actionType,threshold,cursor,chunk);
mainLaunch(portInner, numOfWorkers, jobType, portDumps, pythonPath,datasetPath,pjName,actionType,threshold,cursor,chunk);
}
public static void mainLaunch(String portInner, String serverWait, String numOfWorkers, String jobType, String portDumps, String pythonPath, String datasetPath, String pjName, String dbNo, String actionType, String threshold, String cursor, String chunk){
public static void mainLaunch(String portInner, String numOfWorkers, String jobType, String portDumps, String pythonPath, String datasetPath, String pjName, String actionType, String threshold, String cursor, String chunk){
String dbDir;
@@ -72,43 +71,35 @@ public class Launcher {
try {
switch (jobType) {
case "ENHANCEDASTDIFF":
TestHunkParser.main(gumInput, gumOutput, numOfWorkers, pjName);
EnhancedASTDiff.main(gumInput, gumOutput, numOfWorkers, pjName);
break;
case "CACHE":
StoreFile.main(gumOutput, portDumps, serverWait, dbDir, actionType+dumpsName,actionType);
StoreEDiffInCache.main(gumOutput, portDumps, dbDir, actionType+dumpsName,actionType);
break;
case "COMP":
CalculatePairs.main(serverWait, dbDir, actionType+dumpsName, portDumps, pairsPath+actionType, pjName+actionType);
ImportPairs2DB.main(pairsPath+actionType, portInner, serverWait, dbDir,datasetPath);
case "SI":
CalculatePairs.main(dbDir, actionType+dumpsName, portDumps, pairsPath+actionType, pjName+actionType);
ImportPairs2DB.main(pairsPath+actionType, portInner, dbDir,datasetPath);
break;
case "AKKA":
AkkaTreeLoader.main(portInner, serverWait, dbDir, pjName +actionType+chunk+".rdb" , portDumps, actionType+dumpsName,pairsPath+actionType,numOfWorkers,cursor);
case "SIMI":
AkkaTreeLoader.main(portInner, dbDir, pjName +actionType+chunk+".rdb" , portDumps, actionType+dumpsName,pairsPath+actionType,numOfWorkers,cursor,chunk);
break;
case "LEVEL1":
level1(portInner, serverWait, portDumps, pythonPath, datasetPath, pjName, actionType, threshold, dbDir, pairsPath, dumpsName, gumInput);
level1(portInner, portDumps, pythonPath, datasetPath, pjName, actionType, threshold, dbDir, pairsPath, dumpsName, gumInput);
break;
//CALC python abstractPatch.py to from cluster folder
case "LEVEL2":
level2(serverWait, portDumps, pythonPath, datasetPath, pjName, actionType, threshold, dbDir, dumpsName, gumInput);
level2(portDumps, pythonPath, datasetPath, pjName, actionType, threshold, dbDir, dumpsName, gumInput);
break;
//CALC via python
case "LEVEL3":
level3(serverWait, portDumps, pythonPath, datasetPath, pjName, actionType, threshold, dbDir, dumpsName, gumInput);
break;
case "ALL":
// TestHunkParser.main(gumInput, gumOutput, numOfWorkers, pjName);
// StoreFile.main(gumOutput, portInner, serverWait, dbDir, actionType+dumpsName,actionType);
// level1(portInner, serverWait, port, pythonPath, datasetPath, pjName, actionType, threshold, dbDir, pairsPath, dumpsName, gumInput);
// level2(serverWait, port, pythonPath, datasetPath, pjName, actionType, threshold, dbDir, dumpsName, gumInput);
// level3(serverWait, port, pythonPath, datasetPath, pjName, actionType, threshold, dbDir, dumpsName, gumInput);
level3(portDumps, pythonPath, datasetPath, pjName, actionType, threshold, dbDir, dumpsName, gumInput);
break;
case "EXTRACTPATTERN":
PatternExtractor.mainLaunch(portInner,serverWait,numOfWorkers,jobType,portDumps,pythonPath,datasetPath,pjName,dbNo,actionType,threshold);
PatternExtractor.mainLaunch(portInner,numOfWorkers,jobType,portDumps,pythonPath,datasetPath,pjName,actionType,threshold);
break;
case "GETPATTERN":
PatternExtractor.mainLaunch(portInner,serverWait,numOfWorkers,jobType,portDumps,pythonPath,datasetPath,pjName,dbNo,actionType,threshold);
PatternExtractor.mainLaunch(portInner,numOfWorkers,jobType,portDumps,pythonPath,datasetPath,pjName,actionType,threshold);
break;
}
} catch (Exception e) {
@@ -120,63 +111,63 @@ public class Launcher {
}
private static void level1(String portInner, String serverWait, String port, String pythonPath, String datasetPath, String pjName, String actionType, String threshold, String dbDir, String pairsPath, String dumpsName, String gumInput) throws Exception {
private static void level1(String portInner, String port, String pythonPath, String datasetPath, String pjName, String actionType, String threshold, String dbDir, String pairsPath, String dumpsName, String gumInput) throws Exception {
TreeLoaderClusterL1.main(portInner, serverWait, port, dbDir, "level1-"+pjName+ actionType+".rdb", dbDir ,pjName + actionType);
TreeLoaderClusterL1.main(portInner, port, dbDir, "level1-"+pjName+ actionType+".rdb", dbDir ,pjName + actionType);
CallShell cs1 =new CallShell();
String db1 = "bash "+dbDir + "/" + "startServer.sh" +" %s %s %s";
String db11 = String.format(db1, dbDir,"level1-"+pjName+ actionType+".rdb" ,Integer.valueOf(port));
cs1.runShell(db11,serverWait, port);
cs1.runShell(db11, port);
String runpy = "bash "+datasetPath + "/" + "launchPy.sh" +" %s %s %s %s %s %s";
String formatRunPy = String.format(runpy,pythonPath +"/abstractPatch.py", gumInput, datasetPath + "/cluster"+pjName+ actionType, port, "matches" + pjName + actionType, threshold);
cs1.runShell(formatRunPy);
String stopServer = "bash "+dbDir + "/" + "stopServer.sh" +" %s";
stopServer = String.format(stopServer,Integer.valueOf(port));
cs1.runShell(stopServer,serverWait, port);
cs1.runShell(stopServer, port);
}
private static void level2(String serverWait, String port, String pythonPath, String datasetPath, String pjName, String actionType, String threshold, String dbDir, String dumpsName, String gumInput) throws Exception {
private static void level2(String port, String pythonPath, String datasetPath, String pjName, String actionType, String threshold, String dbDir, String dumpsName, String gumInput) throws Exception {
String stopServer;
MultiThreadTreeLoaderCluster.calculatePairsOfClusters(datasetPath + "/cluster"+pjName+ actionType, datasetPath,actionType);
MultiThreadTreeLoaderCluster.mainCompare("6300", datasetPath+"/pairs"+actionType, datasetPath + "/redisSingleImport.sh", dbDir, "clusterl1-"+pjName+actionType+".rdb", actionType+dumpsName, "6301",serverWait,actionType);
MultiThreadTreeLoaderCluster.mainCompare("6300", datasetPath+"/pairs"+actionType, datasetPath + "/redisSingleImport.sh", dbDir, "clusterl1-"+pjName+actionType+".rdb", actionType+dumpsName, "6301",actionType);
CallShell cs3 =new CallShell();
String db22 = "bash "+dbDir + "/" + "startServer.sh" +" %s %s %s";
String db1b = String.format(db22, dbDir,"clusterl1-"+pjName+actionType+".rdb",Integer.valueOf(port));
cs3.runShell(db1b,serverWait, port);
cs3.runShell(db1b, port);
String runpy2 = "bash "+datasetPath + "/" + "launchPy.sh" +" %s %s %s %s %s %s";
String formatRunPy1a = String.format(runpy2,pythonPath +"/abstractPatchCluster.py", gumInput, datasetPath + "/cluster"+pjName+ actionType, port, datasetPath + "/cluster-2l"+pjName+ actionType,threshold);
cs3.runShell(formatRunPy1a);
String stopServer1a = "bash "+dbDir + "/" + "stopServer.sh" +" %s";
stopServer = String.format(stopServer1a,Integer.valueOf(port));
cs3.runShell(stopServer,serverWait, port);
cs3.runShell(stopServer, port);
}
private static void level3(String serverWait, String port, String pythonPath, String datasetPath, String pjName, String actionType, String threshold, String dbDir, String dumpsName, String gumInput) throws Exception {
private static void level3(String port, String pythonPath, String datasetPath, String pjName, String actionType, String threshold, String dbDir, String dumpsName, String gumInput) throws Exception {
String stopServer;
MultiThreadTreeLoaderCluster3.calculatePairsOfClusters(datasetPath + "/cluster-2l"+pjName+ actionType, datasetPath,actionType);
MultiThreadTreeLoaderCluster3.mainCompare("6300", datasetPath+"/pairs-2l"+actionType, datasetPath + "/redisSingleImport.sh", dbDir, "clusterl2-"+pjName+actionType+".rdb", actionType+dumpsName, "6301",serverWait,actionType);
MultiThreadTreeLoaderCluster3.mainCompare("6300", datasetPath+"/pairs-2l"+actionType, datasetPath + "/redisSingleImport.sh", dbDir, "clusterl2-"+pjName+actionType+".rdb", actionType+dumpsName, "6301",actionType);
CallShell cs5 =new CallShell();
String dba = "bash "+dbDir + "/" + "startServer.sh" +" %s %s %s";
String dbaa = String.format(dba, dbDir,"clusterl2-"+pjName+actionType+".rdb",Integer.valueOf(port));
cs5.runShell(dbaa,serverWait, port);
cs5.runShell(dbaa, port);
String runpya = "bash "+datasetPath + "/" + "launchPy.sh" +" %s %s %s %s %s %s";
String formatRunPya = String.format(runpya,pythonPath +"/abstractPatchClusterLevel3.py", gumInput, datasetPath + "/cluster-3l"+pjName+ actionType, port, datasetPath + "/cluster-2l"+pjName+ actionType,threshold);
cs5.runShell(formatRunPya);
String stopServera = "bash "+dbDir + "/" + "stopServer.sh" +" %s";
stopServer = String.format(stopServera,Integer.valueOf(port));
cs5.runShell(stopServer,serverWait, port);
cs5.runShell(stopServer, port);
return;
}
// System.exit(1);
}
src/main/java/edu/lu/uni/serval/fixminer/PatternExtractor.java
+21 -24
View File
@@ -1,7 +1,5 @@
package edu.lu.uni.serval.fixminer;
import com.github.gumtreediff.tree.ITree;
import com.github.gumtreediff.tree.TreeContext;
import edu.lu.uni.serval.FixPattern.utils.Checker;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
import org.slf4j.Logger;
@@ -21,7 +19,6 @@ import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import static edu.lu.uni.serval.fixminer.cluster.AkkaTreeLoader.getASTTree;
/**
* Created by anilkoyuncu on 02/08/2018.
@@ -30,7 +27,7 @@ public class PatternExtractor {
private static Logger log = LoggerFactory.getLogger(PatternExtractor.class);
public static void mainLaunch(String portInner,String serverWait, String numOfWorkers,String jobType,String port, String pythonPath, String datasetPath, String pjName, String dbNo, String actionType,String threshold) {
public static void mainLaunch(String portInner,String numOfWorkers,String jobType,String port, String pythonPath, String datasetPath, String pjName, String actionType,String threshold) {
String dbDir;
@@ -55,7 +52,7 @@ public class PatternExtractor {
switch (jobType) {
case "EXTRACTPATTERN":
loadDB(gumOutput, portInner, serverWait, dbDir, actionType+dumpsName,actionType,fixes);
loadDB(gumOutput, portInner, dbDir, actionType+dumpsName,actionType,fixes);
break;
case "GETPATTERN":
getPattern(fixes,actionType);
@@ -67,7 +64,7 @@ public class PatternExtractor {
}
public static void loadDB(String inputPath,String portInner,String serverWait,String dbDir,String chunkName,String operation,List<String> fixes) throws Exception {
public static void loadDB(String inputPath,String portInner,String dbDir,String chunkName,String operation,List<String> fixes) throws Exception {
// String inputPath;
// String portInner;
// String serverWait;
@@ -246,24 +243,24 @@ public class PatternExtractor {
}
}
public static ITree getSimpliedTree(HierarchicalActionSet actionSet) {
ITree tree = null;
Jedis inner = null;
try {
ITree parent = null;
ITree children = null;
TreeContext tc = new TreeContext();
tree = getASTTree(actionSet, parent, children, tc);
// tree.setParent(null);
tc.validate();
} catch (Exception e) {
e.printStackTrace();
}
return tree;
}
// public static ITree getSimpliedTree(HierarchicalActionSet actionSet) {
//
// ITree tree = null;
// Jedis inner = null;
// try {
//
// ITree parent = null;
// ITree children = null;
// TreeContext tc = new TreeContext();
// tree = getASTTree(actionSet, parent, children, tc);
//// tree.setParent(null);
// tc.validate();
//
// } catch (Exception e) {
// e.printStackTrace();
// }
// return tree;
// }
src/main/java/edu/lu/uni/serval/FixPatternParser/violations/TestHunkParserSingleFile.java → src/main/java/edu/lu/uni/serval/fixminer/TestHunkParserSingleFile.java
+4 -127
View File
@@ -1,20 +1,18 @@
package edu.lu.uni.serval.FixPatternParser.violations;
package edu.lu.uni.serval.fixminer;
import akka.actor.ActorRef;
import akka.actor.ActorSystem;
import edu.lu.uni.serval.MultipleThreadsParser.MessageFile;
import edu.lu.uni.serval.MultipleThreadsParser.ParseFixPatternActor;
import edu.lu.uni.serval.MultipleThreadsParser.WorkMessage;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.utils.FileHelper;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.*;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
@@ -232,131 +230,10 @@ public class TestHunkParserSingleFile {
// }
}
public static void classifyByAlarmTypes() {
final String alarmTypesFilePath = Configuration.ALARM_TYPES_FILE;
List<String> alarmTypes = readStringList(alarmTypesFilePath);
//edit scripts, sizes of edit scripts, buggy tokens, patches.
classifyByAlarmTypes(alarmTypes, Configuration.EDITSCRIPT_SIZES_FILE);
classifyByAlarmTypes(alarmTypes, Configuration.EDITSCRIPTS_FILE);
classifyByAlarmTypes(alarmTypes, Configuration.BUGGY_CODE_TOKENS_FILE);
classifyByAlarmTypes2(alarmTypes, Configuration.PATCH_SOURCECODE_FILE);
}
private static void classifyByAlarmTypes(List<String> alarmTypes, String file) {
Map<String, StringBuilder> buildersMap = new HashMap<>();
FileInputStream fis = null;
Scanner scanner = null;
try {
fis = new FileInputStream(file);
scanner = new Scanner(fis);
int counter = 0;
while (scanner.hasNextLine()) {
String alarmType = alarmTypes.get(counter);
StringBuilder builder = getBuilder(buildersMap, alarmType);
builder.append(scanner.nextLine() + "\n");
counter ++;
if (counter % 1000 == 0) {
outputBuilders(buildersMap, file);
}
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} finally {
try {
scanner.close();
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
outputBuilders(buildersMap, file);
}
private static void classifyByAlarmTypes2(List<String> alarmTypes, String patchSourcecodeFile) {
Map<String, StringBuilder> buildersMap = new HashMap<>();
FileInputStream fis = null;
Scanner scanner = null;
try {
fis = new FileInputStream(patchSourcecodeFile);
scanner = new Scanner(fis);
int counter = 0;
String singlePatch = "";
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
if (Configuration.PATCH_SIGNAL.equals(line)) {
if (!"".equals(singlePatch)) {
String alarmType = alarmTypes.get(counter);
StringBuilder builder = getBuilder(buildersMap, alarmType);
builder.append(scanner.nextLine() + "\n");
counter ++;
if (counter % 2000 == 0) {
outputBuilders(buildersMap, patchSourcecodeFile);
}
}
singlePatch = line + "\n";
}
singlePatch += line + "\n";
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} finally {
try {
scanner.close();
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
outputBuilders(buildersMap, patchSourcecodeFile);
}
private static void outputBuilders(Map<String, StringBuilder> map, String fileNameStr) {
File file = new File(fileNameStr);
String fileName = file.getName();
String parentPath = file.getParent();
for (Map.Entry<String, StringBuilder> entry : map.entrySet()) {
String alarmType = entry.getKey();
StringBuilder builder = entry.getValue();
FileHelper.outputToFile(parentPath + "/" + alarmType + "/" + fileName, builder, true);
builder.setLength(0);
entry.setValue(builder);
}
}
public static List<String> readStringList(String inputFile) {
List<String> list = new ArrayList<>();
FileInputStream fis = null;
Scanner scanner = null;
try {
fis = new FileInputStream(inputFile);
scanner = new Scanner(fis);
while(scanner.hasNextLine()) {
list.add(scanner.nextLine());
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} finally {
try {
scanner.close();
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return list;
}
private static StringBuilder getBuilder(Map<String, StringBuilder> buildersMap, String alarmType) {
if (buildersMap.containsKey(alarmType)) {
return buildersMap.get(alarmType);
} else {
StringBuilder builder = new StringBuilder();
buildersMap.put(alarmType, builder);
return builder;
}
}
}
src/main/java/edu/lu/uni/serval/FixPatternParser/violations/TestTreeLoader.java → src/main/java/edu/lu/uni/serval/fixminer/TestTreeLoader.java
+96 -95
View File
@@ -1,15 +1,16 @@
package edu.lu.uni.serval.FixPatternParser.violations;
package edu.lu.uni.serval.fixminer;
import com.github.gumtreediff.actions.ActionGenerator;
import com.github.gumtreediff.actions.model.Action;
import com.github.gumtreediff.matchers.Mapping;
import com.github.gumtreediff.matchers.Matcher;
import com.github.gumtreediff.matchers.Matchers;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.gumtree.regroup.SimplifyTree;
import java.io.*;
import java.util.*;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
@@ -132,98 +133,98 @@ public class TestTreeLoader {
}
public static void compareAll(List<File> folders){
List<ITree> trees = new ArrayList<>();
HashMap<Integer, String> hmap = new HashMap<Integer, String>();
for (File target : folders) {
try {
FileInputStream fi = new FileInputStream(new File(target.toString()));
ObjectInputStream oi = new ObjectInputStream(fi);
ITree pr1 = (ITree) oi.readObject();
oi.close();
fi.close();
trees.add(pr1);
hmap.put(folders.indexOf(target), target.toString());
} catch (FileNotFoundException e) {
System.out.println("File not found");
} catch (IOException e) {
System.out.println("Error initializing stream");
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
for (ITree tree : trees) {
// SimplifyTree simplifyTree = new SimplifyTree();
// simplifyTree.canonicalizeSourceCodeTree(tree);
tree.setLabel("");
tree.setParent(null);
List<ITree> descendants = tree.getDescendants();
for (ITree descendant : descendants) {
descendant.setLabel("");
}
}
System.out.println("a");
try {
for (int i = 0; i < trees.size(); i++) {
for (int j = i + 1; j < trees.size(); j++) {
// compare list.get(i) and list.get(j)
BufferedWriter writer = new BufferedWriter(new FileWriter("output.txt",true));
ITree oldTree = trees.get(i);
ITree newTree = trees.get(j);
Matcher m = Matchers.getInstance().getMatcher(oldTree, newTree);
m.match();
ActionGenerator ag = new ActionGenerator(oldTree, newTree, m.getMappings());
ag.generate();
List<Action> actions = ag.getActions();
writer.write(String.valueOf(i));
writer.write("\t");
writer.write(String.valueOf(j));
writer.write("\t");
writer.write(String.format("%1.2f", m.chawatheSimilarity(oldTree, newTree)));
writer.write("\t");
writer.write(String.format("%1.2f", m.diceSimilarity(oldTree, newTree)));
writer.write("\t");
writer.write(String.format("%1.2f", m.jaccardSimilarity(oldTree, newTree)));
writer.write("\t");
writer.write(String.valueOf(actions.size()));
writer.write("\t");
writer.write(hmap.get(i));
writer.write("\t");
writer.write(hmap.get(j));
writer.write("\n");
writer.close();
}
}
} catch (FileNotFoundException e) {
System.out.println("File not found");
} catch (IOException e) {
System.out.println("Error initializing stream");
}
// if (actions.size() > 1) {
// Matcher m1 = Matchers.getInstance().getMatcher(actions.get(0).getNode(), actions.get(0).getNode());
// m1.match();
// Set<Mapping> mappingSet1 = m1.getMappingSet();
// public static void compareAll(List<File> folders){
// List<ITree> trees = new ArrayList<>();
// HashMap<Integer, String> hmap = new HashMap<Integer, String>();
// for (File target : folders) {
//
// try {
// FileInputStream fi = new FileInputStream(new File(target.toString()));
// ObjectInputStream oi = new ObjectInputStream(fi);
// ITree pr1 = (ITree) oi.readObject();
// oi.close();
// fi.close();
// trees.add(pr1);
// hmap.put(folders.indexOf(target), target.toString());
//
// } catch (FileNotFoundException e) {
// System.out.println("File not found");
// } catch (IOException e) {
// System.out.println("Error initializing stream");
// } catch (ClassNotFoundException e) {
// // TODO Auto-generated catch block
// e.printStackTrace();
// }
// }
//
// for (ITree tree : trees) {
//// SimplifyTree simplifyTree = new SimplifyTree();
//// simplifyTree.canonicalizeSourceCodeTree(tree);
// tree.setLabel("");
// tree.setParent(null);
// List<ITree> descendants = tree.getDescendants();
// for (ITree descendant : descendants) {
// descendant.setLabel("");
// }
//
// }
}
// System.out.println("a");
//
// try {
//
//
//
// for (int i = 0; i < trees.size(); i++) {
// for (int j = i + 1; j < trees.size(); j++) {
// // compare list.get(i) and list.get(j)
// BufferedWriter writer = new BufferedWriter(new FileWriter("output.txt",true));
// ITree oldTree = trees.get(i);
//
// ITree newTree = trees.get(j);
//
// Matcher m = Matchers.getInstance().getMatcher(oldTree, newTree);
// m.match();
//
// ActionGenerator ag = new ActionGenerator(oldTree, newTree, m.getMappings());
// ag.generate();
// List<Action> actions = ag.getActions();
// writer.write(String.valueOf(i));
// writer.write("\t");
// writer.write(String.valueOf(j));
// writer.write("\t");
//
// writer.write(String.format("%1.2f", m.chawatheSimilarity(oldTree, newTree)));
// writer.write("\t");
// writer.write(String.format("%1.2f", m.diceSimilarity(oldTree, newTree)));
// writer.write("\t");
// writer.write(String.format("%1.2f", m.jaccardSimilarity(oldTree, newTree)));
// writer.write("\t");
// writer.write(String.valueOf(actions.size()));
// writer.write("\t");
// writer.write(hmap.get(i));
// writer.write("\t");
// writer.write(hmap.get(j));
// writer.write("\n");
//
// writer.close();
// }
// }
//
//
// } catch (FileNotFoundException e) {
// System.out.println("File not found");
// } catch (IOException e) {
// System.out.println("Error initializing stream");
//
// }
//
//
//// if (actions.size() > 1) {
//// Matcher m1 = Matchers.getInstance().getMatcher(actions.get(0).getNode(), actions.get(0).getNode());
//// m1.match();
//// Set<Mapping> mappingSet1 = m1.getMappingSet();
////
//// }
// }
}
src/main/java/edu/lu/uni/serval/fixminer/cluster/AkkaTreeLoader.java
+50 -265
View File
@@ -1,20 +1,17 @@
package edu.lu.uni.serval.fixminer.cluster;
import com.github.gumtreediff.tree.ITree;
import com.github.gumtreediff.tree.TreeContext;
import edu.lu.uni.serval.FixPattern.utils.ASTNodeMap;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
import org.junit.Assert;
import edu.lu.uni.serval.FixPattern.utils.PoolBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.*;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import java.io.*;
import java.time.Duration;
import java.util.*;
import java.util.concurrent.Executors;
import java.util.function.Consumer;
import java.util.stream.Collectors;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import java.util.regex.Pattern;
import static edu.lu.uni.serval.fixminer.cluster.akka.AkkaTreeParser.akkaCompare;
@@ -23,58 +20,13 @@ import static edu.lu.uni.serval.fixminer.cluster.akka.AkkaTreeParser.akkaCompare
*/
public class AkkaTreeLoader {
private static class StreamGobbler implements Runnable {
private InputStream inputStream;
private Consumer<String> consumer;
public StreamGobbler(InputStream inputStream, Consumer<String> consumer) {
this.inputStream = inputStream;
this.consumer = consumer;
}
@Override
public void run() {
new BufferedReader(new InputStreamReader(inputStream)).lines()
.forEach(consumer);
}
}
private static Logger log = LoggerFactory.getLogger(AkkaTreeLoader.class);
public static void loadRedis(String cmd,String serverWait){
Process process;
try {
// String comd = String.format(cmd, f.getAbsoluteFile());
process = Runtime.getRuntime()
.exec(cmd);
public static void main(String portInner, String dbDir, String chunkName, String port, String dumpsName, String pairsPath, String numOfWorkers, String cursor,String chunk) throws Exception {
StreamGobbler streamGobbler =
new StreamGobbler(process.getInputStream(), System.out::println);
Executors.newSingleThreadExecutor().submit(streamGobbler);
int exitCode = process.waitFor();
assert exitCode == 0;
// Thread.sleep(Integer.valueOf(serverWait));
} catch (IOException e) {
e.printStackTrace();
}
catch (InterruptedException e) {
e.printStackTrace();
}
log.info("Load done");
}
private static Consumer<String> consumer = Assert::assertNotNull;
public static void main(String portInner, String serverWait, String dbDir, String chunkName, String port, String dumpsName, String pairsPath, String numOfWorkers, String cursor) throws Exception {
String parameters = String.format("\nportInner %s \nserverWait %s \nchunkName %s \ndbDir %s \ndumpsName %s",portInner,serverWait,chunkName,dbDir,dumpsName);
String parameters = String.format("\nportInner %s \nchunkName %s \ndbDir %s \ndumpsName %s",portInner,chunkName,dbDir,dumpsName);
log.info(parameters);
@@ -82,243 +34,76 @@ public class AkkaTreeLoader {
String cmd = "bash "+dbDir + "/" + "startServer.sh" +" %s %s %s";
String cmd1 = String.format(cmd, dbDir,dumpsName,Integer.valueOf(port));
//
cs.runShell(cmd1,serverWait,port);
cs.runShell(cmd1,port);
String cmdInner = "bash "+dbDir + "/" + "startServer.sh" +" %s %s %s";
String cmd2 = String.format(cmdInner, dbDir,chunkName,Integer.valueOf(portInner));
log.info(cmd1);
log.info(cmd2);
//
cs.runShell(cmd2,serverWait, portInner);
JedisPool outerPool = new JedisPool(poolConfig, "127.0.0.1",Integer.valueOf(port),20000000);
JedisPool innerPool = new JedisPool(poolConfig, "127.0.0.1",Integer.valueOf(portInner),20000000);
// Jedis jedis = new Jedis(new URI("redis://localhost:"+port));
// while (!jedis.ping().equals("PONG")){
// log.info("wait");
// }
//
// jedis = new Jedis(new URI("redis://localhost:"+portInner));
// while (!jedis.ping().equals("PONG")){
// log.info("wait");
// }
// String pairsIndexFile = pairsPath + "/"+ chunkName;
// pairsIndexFile = pairsIndexFile.replace("csv.rdb","index");
//
//
// Pattern pattern = Pattern.compile(",");
// String csvFile = pairsIndexFile;
// try {
// try (BufferedReader in = new BufferedReader(new FileReader(csvFile));){
// Map<String,String> namefreq = in
// .lines()
// // .skip(1)
// .map(x -> pattern.split(x))
// // .filter(x -> x[4].equals("CA") && x[3].equals("F"))
// .collect(HashMap::new, (map, x) ->
// map.put(x[0], x[1]),
// Map::putAll);
// // namefreq.forEach((k, v) -> System.out.println(k + " => " + v));
//
// Jedis inner = null;
// try {
// inner = outerPool.getResource();
//
// for (Map.Entry<String, String> entry : namefreq.entrySet()) {
// String key = entry.getKey();
// String value = entry.getValue();
// inner.select(1);
// inner.set(key,value);
// // ...
// }
//
//
// }finally {
// if (inner != null) {
// inner.close();
// }
// }
//
//
// }
// } catch (IOException e) {
// e.printStackTrace();
// }
cs.runShell(cmd2, portInner);
JedisPool outerPool = new JedisPool(PoolBuilder.getPoolConfig(), "127.0.0.1",Integer.valueOf(port),20000000);
JedisPool innerPool = new JedisPool(PoolBuilder.getPoolConfig(), "127.0.0.1",Integer.valueOf(portInner),20000000);
// comparePairs(innerPool,outerPool);
Thread.sleep(Integer.valueOf(serverWait));
akkaCompare(innerPool,outerPool,numOfWorkers,cursor);
// String stopServer = "bash "+dbDir + "/" + "stopServer.sh" +" %s";
// String stopServer1 = String.format(stopServer,Integer.valueOf(portInner));
//
// cs.runShell(stopServer1,serverWait);
// stopServer = "bash "+dbDir + "/" + "stopServer.sh" +" %s";
// String stopServer2 = String.format(stopServer,Integer.valueOf(port));
//
// cs.runShell(stopServer2,serverWait);
String pairsIndexFile = pairsPath + "/"+ chunkName;
//0.txt.rdb
pairsIndexFile = pairsIndexFile.replace(chunk+".rdb",".index");
}
Pattern pattern = Pattern.compile(",");
String csvFile = pairsIndexFile;
try {
try (BufferedReader in = new BufferedReader(new FileReader(csvFile));){
Map<String,String> namefreq = in
.lines()
.map(x -> pattern.split(x))
.collect(HashMap::new, (map, x) ->
map.put(x[0], x[1]),
Map::putAll);
public static void comparePairs(JedisPool innerPool,JedisPool outerPool){
ScanResult<String> scan;
try (Jedis inner = innerPool.getResource()) {
while (!inner.ping().equals("PONG")){
log.info("wait");
}
ScanParams sc = new ScanParams();
//150000000
log.info("Scanning ");
sc.count(250000000);
sc.match("pair_[0-9]*");
scan = inner.scan("0", sc);
int size = scan.getResult().size();
log.info("Scanned " + String.valueOf(size));
}
List<String> result = scan.getResult();
log.info("Getting results");
result
.parallelStream()
.forEach(m ->
{
Compare compare = new Compare();
compare.coreCompare(m, innerPool, outerPool);
;
}
);
// }
}
/** Read the object from Base64 string. */
private static Object fromString( String s ) throws IOException ,
ClassNotFoundException {
byte [] data = Base64.getDecoder().decode( s );
ObjectInputStream ois = new ObjectInputStream(
new ByteArrayInputStream( data ) );
Object o = ois.readObject();
ois.close();
return o;
}
public static ITree getSimpliedTree(String fn,JedisPool outerPool) {
ITree tree = null;
Jedis inner = null;
try {
inner = outerPool.getResource();
while (!inner.ping().equals("PONG")){
log.info("wait");
}
for (Map.Entry<String, String> entry : namefreq.entrySet()) {
String key = entry.getKey();
String value = entry.getValue();
inner.select(1);
String dist2load = inner.get(fn);
inner.select(0);
String s = inner.get(dist2load);
HierarchicalActionSet actionSet = (HierarchicalActionSet) fromString(s);
inner.set(key,value);
}
ITree parent = null;
ITree children =null;
TreeContext tc = new TreeContext();
tree = getASTTree(actionSet, parent, children,tc);
tree.setParent(null);
tc.validate();
} catch (IOException e) {
e.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
}finally {
if (inner != null) {
inner.close();
}
}
return tree;
}
public static <T, E> List<T> getKeysByValue(Map<T, E> map, E value) {
return map.entrySet()
.stream()
.filter(entry -> Objects.equals(entry.getValue(), value))
.map(Map.Entry::getKey)
.collect(Collectors.toList());
}
public static ITree getASTTree(HierarchicalActionSet actionSet, ITree parent, ITree children,TreeContext tc){
int newType = 0;
String astNodeType = actionSet.getAstNodeType();
List<Integer> keysByValue = getKeysByValue(ASTNodeMap.map, astNodeType);
if(keysByValue.size() != 1){
log.error("More than 1");
}
newType = keysByValue.get(0);
if(actionSet.getParent() == null){
//root
// parent = new Tree(newType,"");
parent = tc.createTree(newType, "", null);
tc.setRoot(parent);
}else{
// children = new Tree(newType,"");
// parent.addChild(children);
children = tc.createTree(newType, "", null);
children.setParentAndUpdateChildren(parent);
}
List<HierarchicalActionSet> subActions = actionSet.getSubActions();
if (subActions.size() != 0){
for (HierarchicalActionSet subAction : subActions) {
if(actionSet.getParent() == null){
children = parent;
}
getASTTree(subAction,children,null,tc);
}
}
return parent;
} catch (IOException e) {
e.printStackTrace();
}
static final JedisPoolConfig poolConfig = buildPoolConfig();
private static JedisPoolConfig buildPoolConfig() {
final JedisPoolConfig poolConfig = new JedisPoolConfig();
poolConfig.setMaxTotal(128);
poolConfig.setMaxIdle(128);
poolConfig.setMinIdle(16);
poolConfig.setTestOnBorrow(false);
poolConfig.setTestOnReturn(false);
poolConfig.setTestWhileIdle(true);
poolConfig.setMinEvictableIdleTimeMillis(Duration.ofMinutes(60).toMillis());
poolConfig.setTimeBetweenEvictionRunsMillis(Duration.ofHours(30).toMillis());
// poolConfig.setNumTestsPerEvictionRun(3);
poolConfig.setBlockWhenExhausted(true);
akkaCompare(innerPool,outerPool,numOfWorkers,cursor);
}
return poolConfig;
}
src/main/java/edu/lu/uni/serval/fixminer/cluster/CalculatePairs.java
+5 -5
View File
@@ -1,5 +1,6 @@
package edu.lu.uni.serval.fixminer.cluster;
import edu.lu.uni.serval.FixPattern.utils.PoolBuilder;
import edu.lu.uni.serval.utils.FileHelper;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -13,7 +14,6 @@ import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.util.List;
import static edu.lu.uni.serval.fixminer.cluster.TreeLoaderClusterL1.poolConfig;
/**
* Created by anilkoyuncu on 05/04/2018.
@@ -21,21 +21,21 @@ import static edu.lu.uni.serval.fixminer.cluster.TreeLoaderClusterL1.poolConfig;
public class CalculatePairs {
private static Logger log = LoggerFactory.getLogger(CalculatePairs.class);
public static void main(String serverWait,String dbDir,String chunkName,String port,String outputPath,String pjName) throws Exception {
public static void main(String dbDir,String chunkName,String port,String outputPath,String pjName) throws Exception {
String parameters = String.format("\nport %s \nserverWait %s \nchunkName %s \ndbDir %s",port,serverWait,chunkName,dbDir);
String parameters = String.format("\nport %s \nchunkName %s \ndbDir %s",port,chunkName,dbDir);
log.info(parameters);
CallShell cs =new CallShell();
String cmd = "bash "+dbDir + "/" + "startServer.sh" +" %s %s %s";
cmd = String.format(cmd, dbDir,chunkName,Integer.valueOf(port));
cs.runShell(cmd,serverWait, port);
cs.runShell(cmd, port);
FileHelper.createDirectory(outputPath);
JedisPool outerPool = new JedisPool(poolConfig, "127.0.0.1",Integer.valueOf(port),20000000);
JedisPool outerPool = new JedisPool(PoolBuilder.getPoolConfig(), "127.0.0.1",Integer.valueOf(port),20000000);
ScanResult<String> scan;
try (Jedis outer = outerPool.getResource()) {
src/main/java/edu/lu/uni/serval/fixminer/cluster/CallShell.java
+2 -2
View File
@@ -26,7 +26,7 @@ public class CallShell {
}
public static void runShell(String command, String serverWait, String port) throws Exception {
public static void runShell(String command, String port) throws Exception {
Process process = Runtime.getRuntime().exec(command);
BufferedReader reader = new BufferedReader(new InputStreamReader(
@@ -101,7 +101,7 @@ try{
public static void main(String[] args) throws Exception {
// runPing("redis-cli -p 6380 ping");
runShell("bash /Users/anilkoyuncu/bugStudy/release/code/redis/startServer.sh /Users/anilkoyuncu/bugStudy/release/code/redis Defects4JALL0.txt.rdb 6380","1","6380");
runShell("bash /Users/anilkoyuncu/bugStudy/release/code/redis/startServer.sh /Users/anilkoyuncu/bugStudy/release/code/redis Defects4JALL0.txt.rdb 6380","6380");
}
src/main/java/edu/lu/uni/serval/fixminer/cluster/Compare.java
+3 -3
View File
@@ -5,6 +5,7 @@ import com.github.gumtreediff.actions.model.Action;
import com.github.gumtreediff.matchers.Matcher;
import com.github.gumtreediff.matchers.Matchers;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.FixPattern.utils.EDiff;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.Jedis;
@@ -13,7 +14,6 @@ import redis.clients.jedis.JedisPool;
import java.util.List;
import java.util.Map;
import static edu.lu.uni.serval.fixminer.cluster.AkkaTreeLoader.getSimpliedTree;
/**
* Created by anilkoyuncu on 03/04/2018.
@@ -44,9 +44,9 @@ public class Compare {
// String firstValue = resultMap.get("0");
// String secondValue = resultMap.get("1");
oldTree = getSimpliedTree(i,outerPool);
oldTree = EDiff.getSimpliedTree(i,outerPool);
newTree = getSimpliedTree(j,outerPool);
newTree = EDiff.getSimpliedTree(j,outerPool);
Matcher m = Matchers.getInstance().getMatcher(oldTree, newTree);
m.match();
src/main/java/edu/lu/uni/serval/fixminer/cluster/ImportPairs2DB.java
+4 -7
View File
@@ -9,9 +9,6 @@ import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
//import static edu.lu.uni.serval.fixminer.cluster.AkkaTreeLoader.loadRedis;
//import static edu.lu.uni.serval.fixminer.cluster.AkkaTreeLoader.loadRedisWait;
/**
* Created by anilkoyuncu on 05/04/2018.
@@ -19,10 +16,10 @@ import java.util.stream.Stream;
public class ImportPairs2DB {
private static Logger log = LoggerFactory.getLogger(ImportPairs2DB.class);
public static void main(String csvInputPath,String portInner,String serverWait,String dbDir,String datasetPath) throws Exception {
public static void main(String csvInputPath,String portInner,String dbDir,String datasetPath) throws Exception {
String parameters = String.format("\nInput path %s \nportInner %s \nserverWait %s \ndbDir %s",csvInputPath,portInner,serverWait,dbDir);
String parameters = String.format("\nInput path %s \nportInner %s \ndbDir %s",csvInputPath,portInner,dbDir);
log.info(parameters);
@@ -40,7 +37,7 @@ public class ImportPairs2DB {
cmd = String.format(cmd, dbDir,pj.getName() +".rdb", portInt);
log.info(cmd);
CallShell cs = new CallShell();
cs.runShell(cmd,serverWait, portInner);
cs.runShell(cmd, portInner);
cmd = "bash "+datasetPath + "/redisSingleImport.sh" +" %s %s";
@@ -50,7 +47,7 @@ public class ImportPairs2DB {
String stopServer = "bash "+dbDir + "/" + "stopServer.sh" +" %s";
String stopServer2 = String.format(stopServer,portInt);
cs.runShell(stopServer2,serverWait, portInner);
cs.runShell(stopServer2, portInner);
portInt++;
src/main/java/edu/lu/uni/serval/fixminer/cluster/MultiThreadTreeLoader.java
+51 -52
View File
@@ -1,14 +1,13 @@
package edu.lu.uni.serval.fixminer.cluster;
import com.github.gumtreediff.actions.ActionGenerator;
import com.github.gumtreediff.actions.model.*;
import com.github.gumtreediff.actions.model.Action;
import com.github.gumtreediff.matchers.Matcher;
import com.github.gumtreediff.matchers.Matchers;
import com.github.gumtreediff.tree.ITree;
import com.github.gumtreediff.tree.Tree;
import edu.lu.uni.serval.FixPattern.utils.ASTNodeMap;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
import edu.lu.uni.serval.utils.FileHelper;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.*;
@@ -291,17 +290,17 @@ public class MultiThreadTreeLoader {
readMessageFiles(fileToCompare,port);
}
public static void processMessages(String inputPath, String outputPath) {
File folder = new File(outputPath + "pairs_splitted/");
File[] listOfFiles = folder.listFiles();
Stream<File> stream = Arrays.stream(listOfFiles);
List<File> pjs = stream
.filter(x -> !x.getName().startsWith("."))
.collect(Collectors.toList());
FileHelper.createDirectory(outputPath + "comparison_splitted/");
pjs.parallelStream()
.forEach(m -> coreLoop(m, outputPath,inputPath));
}
// public static void processMessages(String inputPath, String outputPath) {
// File folder = new File(outputPath + "pairs_splitted/");
// File[] listOfFiles = folder.listFiles();
// Stream<File> stream = Arrays.stream(listOfFiles);
// List<File> pjs = stream
// .filter(x -> !x.getName().startsWith("."))
// .collect(Collectors.toList());
// FileHelper.createDirectory(outputPath + "comparison_splitted/");
// pjs.parallelStream()
// .forEach(m -> coreLoop(m, outputPath,inputPath));
// }
public static ITree getSimpliedTree(String fn) {
@@ -377,45 +376,45 @@ public class MultiThreadTreeLoader {
return parent;
}
public static ITree getActionTree(HierarchicalActionSet actionSet, ITree parent, ITree children){
int newType = 0;
Action action = actionSet.getAction();
if (action instanceof Update){
newType = 101;
}else if(action instanceof Insert){
newType =100;
}else if(action instanceof Move){
newType = 102;
}else if(action instanceof Delete){
newType=103;
}else{
new Exception("unknow action");
}
if(actionSet.getParent() == null){
//root
parent = new Tree(newType,"");
}else{
children = new Tree(newType,"");
parent.addChild(children);
}
List<HierarchicalActionSet> subActions = actionSet.getSubActions();
if (subActions.size() != 0){
for (HierarchicalActionSet subAction : subActions) {
if(actionSet.getParent() == null){
children = parent;
}
getActionTree(subAction,children,null);
}
}
return parent;
}
// public static ITree getActionTree(HierarchicalActionSet actionSet, ITree parent, ITree children){
//
// int newType = 0;
//
// Action action = actionSet.getAction();
// if (action instanceof Update){
// newType = 101;
// }else if(action instanceof Insert){
// newType =100;
// }else if(action instanceof Move){
// newType = 102;
// }else if(action instanceof Delete){
// newType=103;
// }else{
// new Exception("unknow action");
// }
// if(actionSet.getParent() == null){
// //root
//
// parent = new Tree(newType,"");
// }else{
// children = new Tree(newType,"");
// parent.addChild(children);
// }
// List<HierarchicalActionSet> subActions = actionSet.getSubActions();
// if (subActions.size() != 0){
// for (HierarchicalActionSet subAction : subActions) {
//
// if(actionSet.getParent() == null){
// children = parent;
// }
// getActionTree(subAction,children,null);
//
// }
//
//
// }
// return parent;
// }
src/main/java/edu/lu/uni/serval/fixminer/cluster/MultiThreadTreeLoaderCluster.java
+22 -92
View File
@@ -1,22 +1,29 @@
package edu.lu.uni.serval.fixminer.cluster;
import com.github.gumtreediff.actions.ActionGenerator;
import com.github.gumtreediff.actions.model.*;
import com.github.gumtreediff.actions.model.Action;
import com.github.gumtreediff.matchers.Matcher;
import com.github.gumtreediff.matchers.Matchers;
import com.github.gumtreediff.tree.ITree;
import com.github.gumtreediff.tree.TreeContext;
import edu.lu.uni.serval.FixPattern.utils.EDiff;
import edu.lu.uni.serval.FixPattern.utils.EDiffHelper;
import edu.lu.uni.serval.FixPattern.utils.PoolBuilder;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
import edu.lu.uni.serval.utils.FileHelper;
import org.javatuples.Pair;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.*;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.ScanParams;
import redis.clients.jedis.ScanResult;
import java.io.*;
import java.time.Duration;
import java.util.*;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import java.util.stream.Stream;
@@ -28,28 +35,27 @@ public class MultiThreadTreeLoaderCluster {
private static Logger log = LoggerFactory.getLogger(MultiThreadTreeLoaderCluster.class);
public static void mainCompare(String port,String pairsCSVPath,String importScript,String dbDir,String chunkName,String dumpName,String portInner,String serverWait,String type) throws Exception {
public static void mainCompare(String port,String pairsCSVPath,String importScript,String dbDir,String chunkName,String dumpName,String portInner,String type) throws Exception {
CallShell cs = new CallShell();
String cmd1 = "bash "+dbDir + "/" + "startServer.sh" +" %s %s %s";
cmd1 = String.format(cmd1, dbDir,chunkName,Integer.valueOf(portInner));
// edu.lu.uni.serval.fixminer.cluster.AkkaTreeLoader.loadRedis(cmd1,"1000");
cs.runShell(cmd1,serverWait, port);
cs.runShell(cmd1, port);
String cmd2 = "bash "+dbDir + "/" + "startServer.sh" +" %s %s %s";
cmd2 = String.format(cmd2, dbDir,dumpName,Integer.valueOf(port));
cs.runShell(cmd2,serverWait, port);
// edu.lu.uni.serval.fixminer.cluster.AkkaTreeLoader.loadRedis(cmd2,"10000");
cs.runShell(cmd2, port);
String cmd3;
cmd3 = "bash " + importScript +" %s %s";
JedisPool jedisPool = new JedisPool(poolConfig, "127.0.0.1",Integer.valueOf(portInner),20000000);
JedisPool jedisPool = new JedisPool(PoolBuilder.getPoolConfig(), "127.0.0.1",Integer.valueOf(portInner),20000000);
JedisPool outerPool = new JedisPool(poolConfig, "127.0.0.1",Integer.valueOf(port),20000000);
JedisPool outerPool = new JedisPool(PoolBuilder.getPoolConfig(), "127.0.0.1",Integer.valueOf(port),20000000);
File folder = new File(pairsCSVPath);
@@ -104,29 +110,16 @@ public class MultiThreadTreeLoaderCluster {
}
String stopServer = "bash "+dbDir + "/" + "stopServer.sh" +" %s";
String stopServer1 = String.format(stopServer,Integer.valueOf(portInner));
// loadRedis(stopServer2,serverWait);
cs.runShell(stopServer1,serverWait, port);
cs.runShell(stopServer1, port);
String stopServer2 = String.format(stopServer,Integer.valueOf(port));
// loadRedis(stopServer2,serverWait);
cs.runShell(stopServer2,serverWait, port);
cs.runShell(stopServer2, port);
}
/** Read the object from Base64 string. */
public static Object fromString( String s ) throws IOException ,
ClassNotFoundException {
byte [] data = Base64.getDecoder().decode( s );
ObjectInputStream ois = new ObjectInputStream(
new ByteArrayInputStream( data ) );
Object o = ois.readObject();
ois.close();
return o;
}
public static Pair<ITree,String> getTree(String firstValue, JedisPool outerPool,String type){
@@ -148,13 +141,13 @@ public class MultiThreadTreeLoaderCluster {
inner = outerPool.getResource();
String filename = project + "/"+type+"/" + pureFileName + ".txt_" + actionSetPosition;
String si= inner.get(filename);
HierarchicalActionSet actionSet = (HierarchicalActionSet) fromString(si);
HierarchicalActionSet actionSet = (HierarchicalActionSet) EDiffHelper.fromString(si);
ITree parent = null;
ITree children =null;
TreeContext tc = new TreeContext();
tree = getActionTree(actionSet, parent, children,tc);
tree = EDiff.getActionTree(actionSet, parent, children,tc);
tree.setParent(null);
tc.validate();
@@ -179,50 +172,7 @@ public class MultiThreadTreeLoaderCluster {
}
public static ITree getActionTree(HierarchicalActionSet actionSet, ITree parent, ITree children,TreeContext tc){
int newType = 0;
Action action = actionSet.getAction();
if (action instanceof Update){
newType = 101;
}else if(action instanceof Insert){
newType =100;
}else if(action instanceof Move){
newType = 102;
}else if(action instanceof Delete){
newType=103;
}else{
new Exception("unknow action");
}
if(actionSet.getParent() == null){
//root
parent = tc.createTree(newType, "", null);
tc.setRoot(parent);
// parent = new Tree(newType,"");
}else{
children = tc.createTree(newType, "", null);
children.setParentAndUpdateChildren(parent);
// children = new Tree(newType,"");
// parent.addChild(children);
}
List<HierarchicalActionSet> subActions = actionSet.getSubActions();
if (subActions.size() != 0){
for (HierarchicalActionSet subAction : subActions) {
if(actionSet.getParent() == null){
children = parent;
}
getActionTree(subAction,children,null,tc);
}
}
return parent;
}
private static void coreCompare(String name , JedisPool jedisPool,String clusterName,JedisPool outerPool,String type) {
@@ -383,26 +333,6 @@ public class MultiThreadTreeLoaderCluster {
static final JedisPoolConfig poolConfig = buildPoolConfig();
private static JedisPoolConfig buildPoolConfig() {
final JedisPoolConfig poolConfig = new JedisPoolConfig();
poolConfig.setMaxTotal(128);
poolConfig.setMaxIdle(128);
poolConfig.setMinIdle(16);
poolConfig.setTestOnBorrow(true);
poolConfig.setTestOnReturn(true);
poolConfig.setTestWhileIdle(true);
poolConfig.setMinEvictableIdleTimeMillis(Duration.ofMinutes(60).toMillis());
poolConfig.setTimeBetweenEvictionRunsMillis(Duration.ofHours(30).toMillis());
poolConfig.setNumTestsPerEvictionRun(3);
poolConfig.setBlockWhenExhausted(true);
return poolConfig;
}
// return msgFiles;
}
src/main/java/edu/lu/uni/serval/fixminer/cluster/MultiThreadTreeLoaderCluster3.java
+20 -136
View File
@@ -2,24 +2,26 @@ package edu.lu.uni.serval.fixminer.cluster;
import com.github.gumtreediff.tree.ITree;
import com.github.gumtreediff.tree.TreeContext;
import edu.lu.uni.serval.FixPattern.utils.ASTNodeMap;
import edu.lu.uni.serval.FixPattern.utils.EDiffHelper;
import edu.lu.uni.serval.FixPattern.utils.PoolBuilder;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
import edu.lu.uni.serval.utils.FileHelper;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.text.similarity.JaroWinklerDistance;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.*;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.ScanParams;
import redis.clients.jedis.ScanResult;
import java.io.*;
import java.time.Duration;
import java.util.*;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import static edu.lu.uni.serval.fixminer.cluster.MultiThreadTreeLoader.getKeysByValue;
import static edu.lu.uni.serval.fixminer.cluster.MultiThreadTreeLoaderCluster.fromString;
/**
* Created by anilkoyuncu on 19/03/2018.
@@ -30,27 +32,24 @@ public class MultiThreadTreeLoaderCluster3 {
private static Logger log = LoggerFactory.getLogger(MultiThreadTreeLoaderCluster3.class);
// public static void mainCompare(String inputPath,String port,String pairsCSVPath,String importScript) {
public static void mainCompare(String port,String pairsCSVPath,String importScript,String dbDir,String chunkName,String dumpName,String portInner,String serverWait, String type) throws Exception {
public static void mainCompare(String port,String pairsCSVPath,String importScript,String dbDir,String chunkName,String dumpName,String portInner,String type) throws Exception {
CallShell cs = new CallShell();
String cmd1 = "bash "+dbDir + "/" + "startServer.sh" +" %s %s %s";
cmd1 = String.format(cmd1, dbDir,chunkName,Integer.valueOf(portInner));
// edu.lu.uni.serval.fixminer.cluster.AkkaTreeLoader.loadRedis(cmd1,"1000");
cs.runShell(cmd1,serverWait, port);
cs.runShell(cmd1, port);
String cmd2 = "bash "+dbDir + "/" + "startServer.sh" +" %s %s %s";
cmd2 = String.format(cmd2, dbDir,dumpName,Integer.valueOf(port));
// edu.lu.uni.serval.fixminer.cluster.AkkaTreeLoader.loadRedis(cmd2,"10000");
cs.runShell(cmd2,serverWait, port);
cs.runShell(cmd2, port);
String cmd3;
cmd3 = "bash " + importScript +" %s %s";
JedisPool jedisPool = new JedisPool(poolConfig, "127.0.0.1",Integer.valueOf(portInner),20000000);
JedisPool jedisPool = new JedisPool(PoolBuilder.getPoolConfig(), "127.0.0.1",Integer.valueOf(portInner),20000000);
JedisPool outerPool = new JedisPool(poolConfig, "127.0.0.1",Integer.valueOf(port),20000000);
JedisPool outerPool = new JedisPool(PoolBuilder.getPoolConfig(), "127.0.0.1",Integer.valueOf(port),20000000);
@@ -79,8 +78,7 @@ public class MultiThreadTreeLoaderCluster3 {
if (size == 0) {
String comd = String.format(cmd3, f.getPath(), portInner);
cs.runShell(comd);
// edu.lu.uni.serval.fixminer.cluster.MultiThreadTreeLoaderCluster.
// loadRedis(comd);
scan = jedis.scan("0", sc);
size = scan.getResult().size();
@@ -103,17 +101,10 @@ public class MultiThreadTreeLoaderCluster3 {
String stopServer = "bash "+dbDir + "/" + "stopServer.sh" +" %s";
String stopServer1 = String.format(stopServer,Integer.valueOf(portInner));
// loadRedis(stopServer2,serverWait);
cs.runShell(stopServer1,serverWait, port);
cs.runShell(stopServer1, port);
String stopServer2 = String.format(stopServer,Integer.valueOf(port));
// loadRedis(stopServer2,serverWait);
cs.runShell(stopServer2,serverWait, port);
// }
cs.runShell(stopServer2, port);
@@ -151,13 +142,13 @@ public class MultiThreadTreeLoaderCluster3 {
inner = outerPool.getResource();
String fn = project + "/"+type+"/" + s + ".txt_" + actionSetPosition;
String si= inner.get(fn);
HierarchicalActionSet actionSet = (HierarchicalActionSet) fromString(si);
HierarchicalActionSet actionSet = (HierarchicalActionSet) EDiffHelper.fromString(si);
ITree parent = null;
ITree children =null;
TreeContext tc = new TreeContext();
tree = getASTTree(actionSet, parent, children,tc);
tree = EDiffHelper.getASTTree(actionSet, parent, children,tc);
// tree.setParent(null);
tc.validate();
@@ -176,46 +167,7 @@ public class MultiThreadTreeLoaderCluster3 {
}
public static ITree getASTTree(HierarchicalActionSet actionSet, ITree parent, ITree children,TreeContext tc){
int newType = 0;
String astNodeType = actionSet.getAstNodeType();
String label = actionSet.getAction().toString();
List<Integer> keysByValue = getKeysByValue(ASTNodeMap.map, astNodeType);
if(keysByValue.size() != 1){
log.error("Birden cok astnodemapmapping");
}
newType = keysByValue.get(0);
if(actionSet.getParent() == null){
//root
// parent = new Tree(newType,"");
parent = tc.createTree(newType, label, null);
tc.setRoot(parent);
}else{
// children = new Tree(newType,"");
// parent.addChild(children);
children = tc.createTree(newType, label, null);
children.setParentAndUpdateChildren(parent);
}
List<HierarchicalActionSet> subActions = actionSet.getSubActions();
if (subActions.size() != 0){
for (HierarchicalActionSet subAction : subActions) {
if(actionSet.getParent() == null){
children = parent;
}
getASTTree(subAction,children,null,tc);
}
}
return parent;
}
public static List<String> getNames(ITree oldTree, List<String> oldTokens){
@@ -489,31 +441,6 @@ public class MultiThreadTreeLoaderCluster3 {
String firstValue = resultMap.get("0");
String secondValue = resultMap.get("1");
// if(firstValue.equals("7/0/7af9e0_e674f1_spring-social-web#src#main#java#org#springframework#social#connect#web#ConnectController.txt_0_SOCIAL") || secondValue.equals("7/0/7af9e0_e674f1_spring-social-web#src#main#java#org#springframework#social#connect#web#ConnectController.txt_0_SOCIAL")){
// log.info("");
// }
///35/1/22b5f7_84bf27_ui#org.eclipse.pde.runtime#src#org#eclipse#pde#internal#runtime#registry#RegistryBrowserLabelProvider.txt_2_PDE
// firstValue = inputPath + firstValue;
// secondValue = inputPath + secondValue;
// String[] firstValueSplit = firstValue.split("/");
// String[] secondValueSplit = secondValue.split("/");
//
// if (firstValueSplit.length == 1) {
// firstValue = inputPath + firstValueSplit[0];
// } else {
// firstValue = inputPath + firstValueSplit[1];
// }
//
// if (secondValueSplit.length == 1) {
// secondValue = inputPath + secondValueSplit[0];
// } else {
// secondValue = inputPath + secondValueSplit[1];
// }
try {
ITree oldTree = getTree(firstValue,outerPool,type);
@@ -526,30 +453,18 @@ public class MultiThreadTreeLoaderCluster3 {
return;
}
// ITree oldTree = oldPair.getValue0();
// ITree newTree = newPair.getValue0();
//
// String oldProject = oldPair.getValue1();
// String newProject = newPair.getValue1();
List<String> oldTokens = new ArrayList<>();
List<String> newTokens = new ArrayList<>();
// if(secondValue.startsWith("/2/")){
// log.info("newss");
// }
// 41 return statement
oldTokens = getNames(oldTree,oldTokens);
newTokens = getNames(newTree,newTokens);
// if(oldTokens.size() == 0 || newTokens.size() == 0){
// log.error("Cluster {} has no tokens on pair {}",clusterName , name);
// }
// Matcher m = Matchers.getInstance().getMatcher(oldTree, newTree);
// m.match();
CharSequence[] oldSequences = oldTokens.toArray(new CharSequence[oldTokens.size()]);
CharSequence[] newSequences = newTokens.toArray(new CharSequence[newTokens.size()]);
JaroWinklerDistance jwd = new JaroWinklerDistance();
@@ -572,14 +487,7 @@ public class MultiThreadTreeLoaderCluster3 {
int retval = Double.compare(overallSimi, Double.valueOf(0.8));
if(retval >= 0){
// log.info("YES");
// log.info(name);
// log.info(firstValue);
// log.info(secondValue);
// log.info("************");
// if(!overallSimi.equals(1.0)){
// log.info("");
// }
String matchKey = "match-"+clusterName+"_" + (String.valueOf(i)) + "_" + String.valueOf(j);
String result = firstValue + "," + secondValue + ","+String.join(",", oldTokens);
jedis.select(1);
@@ -705,30 +613,6 @@ public class MultiThreadTreeLoaderCluster3 {
static final JedisPoolConfig poolConfig = buildPoolConfig();
private static JedisPoolConfig buildPoolConfig() {
final JedisPoolConfig poolConfig = new JedisPoolConfig();
poolConfig.setMaxTotal(128);
poolConfig.setMaxIdle(128);
poolConfig.setMinIdle(16);
poolConfig.setTestOnBorrow(true);
poolConfig.setTestOnReturn(true);
poolConfig.setTestWhileIdle(true);
poolConfig.setMinEvictableIdleTimeMillis(Duration.ofMinutes(60).toMillis());
poolConfig.setTimeBetweenEvictionRunsMillis(Duration.ofHours(30).toMillis());
poolConfig.setNumTestsPerEvictionRun(3);
poolConfig.setBlockWhenExhausted(true);
return poolConfig;
}
// return msgFiles;
}
src/main/java/edu/lu/uni/serval/fixminer/cluster/StoreFile.java → src/main/java/edu/lu/uni/serval/fixminer/cluster/StoreEDiffInCache.java
+16 -52
View File
@@ -1,5 +1,7 @@
package edu.lu.uni.serval.fixminer.cluster;
import edu.lu.uni.serval.FixPattern.utils.EDiffHelper;
import edu.lu.uni.serval.FixPattern.utils.PoolBuilder;
import edu.lu.uni.serval.gumtree.regroup.HierarchicalActionSet;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -9,50 +11,28 @@ import redis.clients.jedis.JedisPool;
import java.io.*;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Base64;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import static edu.lu.uni.serval.fixminer.cluster.TreeLoaderClusterL1.poolConfig;
/**
* Created by anilkoyuncu on 03/04/2018.
*/
public class StoreFile {
public class StoreEDiffInCache {
private static Logger log = LoggerFactory.getLogger(StoreFile.class);
private static Logger log = LoggerFactory.getLogger(StoreEDiffInCache.class);
// public static void main(String[] args) {
public static void main(String inputPath,String portInner,String serverWait,String dbDir,String chunkName,String operation) throws Exception {
// String inputPath;
// String portInner;
// String serverWait;
// String dbDir;
// String chunkName;
// String numOfWorkers;
// if (args.length > 0) {
// inputPath = args[0];
// portInner = args[1];
// serverWait = args[2];
// chunkName = args[3];
// numOfWorkers = args[4];
// dbDir = args[5];
// } else {
// inputPath = "/Users/anilkoyuncu/bugStudy/dataset/GumTreeOutput2";
// portInner = "6399";
// serverWait = "10000";
// chunkName ="dumps.rdb";
// dbDir = "/Users/anilkoyuncu/bugStudy/dataset/redis";
// numOfWorkers = "1";
// }
String parameters = String.format("\nInput path %s \nportInner %s \nserverWait %s \nchunkName %s \ndbDir %s \noperation %s",inputPath,portInner,serverWait,chunkName,dbDir,operation);
public static void main(String inputPath,String portInner,String dbDir,String chunkName,String operation) throws Exception {
String parameters = String.format("\nInput path %s \nportInner %s \nchunkName %s \ndbDir %s \noperation %s",inputPath,portInner,chunkName,dbDir,operation);
log.info(parameters);
CallShell cs = new CallShell();
String cmd = "bash "+dbDir + "/" + "startServer.sh" +" %s %s %s";
cmd = String.format(cmd, dbDir,chunkName,Integer.valueOf(portInner));
// loadRedis(cmd,serverWait);
cs.runShell(cmd,serverWait, portInner);
cs.runShell(cmd, portInner);
File folder = new File(inputPath);
File[] subFolders = folder.listFiles();
@@ -98,7 +78,7 @@ public class StoreFile {
}
log.info(String.valueOf(workList.size()));
JedisPool innerPool = new JedisPool(poolConfig, "127.0.0.1",Integer.valueOf(portInner),20000000);
JedisPool innerPool = new JedisPool(PoolBuilder.getPoolConfig(), "127.0.0.1",Integer.valueOf(portInner),20000000);
workList.stream().parallel()
.forEach(m -> storeCore(innerPool, m.split(",")[1],m.split(",")[0]));
@@ -107,10 +87,12 @@ public class StoreFile {
String stopServer = "bash "+dbDir + "/" + "stopServer.sh" +" %s";
String stopServer2 = String.format(stopServer,Integer.valueOf(portInner));
// loadRedis(stopServer2,serverWait);
cs.runShell(stopServer2,serverWait, portInner);
cs.runShell(stopServer2, portInner);
}
public static void storeCore(JedisPool innerPool,String path,String key){
try {
@@ -141,7 +123,7 @@ public class StoreFile {
try (Jedis inner = innerPool.getResource()) {
inner.set(key,toString(actionSet));
inner.set(key, EDiffHelper.toString(actionSet));
}
@@ -155,25 +137,7 @@ public class StoreFile {
}
/** Read the object from Base64 string. */
private static Object fromString( String s ) throws IOException ,
ClassNotFoundException {
byte [] data = Base64.getDecoder().decode( s );
ObjectInputStream ois = new ObjectInputStream(
new ByteArrayInputStream( data ) );
Object o = ois.readObject();
ois.close();
return o;
}
/** Write the object to a Base64 string. */
private static String toString( Serializable o ) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream( baos );
oos.writeObject( o );
oos.close();
return Base64.getEncoder().encodeToString(baos.toByteArray());
}
}
src/main/java/edu/lu/uni/serval/fixminer/cluster/TreeActor.java
-3
View File
@@ -8,13 +8,10 @@ import akka.routing.RoundRobinPool;
import edu.lu.uni.serval.fixminer.cluster.akka.TreeMessage;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import java.util.List;
import static edu.lu.uni.serval.fixminer.cluster.AkkaTreeLoader.loadRedis;
public class TreeActor extends UntypedActor {
private static Logger logger = LoggerFactory.getLogger(TreeActor.class);
src/main/java/edu/lu/uni/serval/fixminer/cluster/TreeLoaderClusterL1.java
+15 -29
View File
@@ -1,11 +1,14 @@
package edu.lu.uni.serval.fixminer.cluster;
import edu.lu.uni.serval.FixPattern.utils.PoolBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.*;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.ScanParams;
import redis.clients.jedis.ScanResult;
import java.io.File;
import java.time.Duration;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;
@@ -18,19 +21,19 @@ public class TreeLoaderClusterL1 {
private static Logger log = LoggerFactory.getLogger(TreeLoaderClusterL1.class);
public static void main(String portInner,String serverWait,String port,String inputPath,String level1DB,String level1Path,String innerTypePrefix) throws Exception {
public static void main(String portInner,String port,String inputPath,String level1DB,String level1Path,String innerTypePrefix) throws Exception {
String parameters = String.format("\nInput path %s \nportInner %s \nserverWait %s \nport %s",inputPath,portInner,serverWait,port);
String parameters = String.format("\nInput path %s \nportInner %s \nport %s",inputPath,portInner,port);
log.info(parameters);
String cmd = "bash "+inputPath + "/" + "startServer.sh" +" %s %s %s";
cmd = String.format(cmd, inputPath,level1DB,Integer.valueOf(port));
CallShell cs = new CallShell();
cs.runShell(cmd,serverWait, port);
JedisPool outerPool = new JedisPool(poolConfig, "127.0.0.1",Integer.valueOf(port),20000000);
cs.runShell(cmd, port);
JedisPool outerPool = new JedisPool(PoolBuilder.getPoolConfig(), "127.0.0.1",Integer.valueOf(port),20000000);
File chunks = new File(level1Path);
@@ -44,8 +47,8 @@ public class TreeLoaderClusterL1 {
String cmdInner = "bash "+inputPath + "/" + "startServer.sh" +" %s %s %s";
cmdInner = String.format(cmdInner, inputPath,db.getName(),Integer.valueOf(portInner));
cs.runShell(cmdInner,serverWait, port);
JedisPool innerPool = new JedisPool(poolConfig, "127.0.0.1",Integer.valueOf(portInner),20000000);
cs.runShell(cmdInner, port);
JedisPool innerPool = new JedisPool(PoolBuilder.getPoolConfig(), "127.0.0.1",Integer.valueOf(portInner),20000000);
Jedis inner = null;
@@ -97,38 +100,21 @@ public class TreeLoaderClusterL1 {
String stopServer = "bash "+inputPath + "/" + "stopServer.sh" +" %s";
stopServer = String.format(stopServer,Integer.valueOf(portInner));
// loadRedis(stopServer,serverWait);
cs.runShell(stopServer,serverWait, port);
cs.runShell(stopServer, port);
}
String stopServer1 = "bash "+inputPath + "/" + "stopServer.sh" +" %s";
stopServer1 = String.format(stopServer1,Integer.valueOf(port));
// loadRedis(stopServer,serverWait);
cs.runShell(stopServer1,serverWait, port);
cs.runShell(stopServer1, port);
}
static final JedisPoolConfig poolConfig = buildPoolConfig();
private static JedisPoolConfig buildPoolConfig() {
final JedisPoolConfig poolConfig = new JedisPoolConfig();
poolConfig.setMaxTotal(128);
poolConfig.setMaxIdle(128);
poolConfig.setMinIdle(16);
poolConfig.setTestOnBorrow(true);
poolConfig.setTestOnReturn(true);
poolConfig.setTestWhileIdle(true);
poolConfig.setMinEvictableIdleTimeMillis(Duration.ofMinutes(60).toMillis());
poolConfig.setTimeBetweenEvictionRunsMillis(Duration.ofHours(30).toMillis());
poolConfig.setNumTestsPerEvictionRun(3);
poolConfig.setBlockWhenExhausted(true);
return poolConfig;
}
}
src/main/java/edu/lu/uni/serval/fixminer/cluster/TreeWorker.java
-4
View File
@@ -8,14 +8,10 @@ import edu.lu.uni.serval.fixminer.cluster.akka.TreeMessage;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;
import java.time.Duration;
import java.util.List;
import java.util.concurrent.*;
import static edu.lu.uni.serval.fixminer.cluster.AkkaTreeLoader.poolConfig;
public class TreeWorker extends UntypedActor {
private static Logger log = LoggerFactory.getLogger(TreeWorker.class);
src/main/java/edu/lu/uni/serval/gumtree/regroup/ActionFilter.java
-254
View File
@@ -1,254 +0,0 @@
package edu.lu.uni.serval.gumtree.regroup;
import java.util.ArrayList;
import java.util.List;
public class ActionFilter {
private List<String> methodNames = new ArrayList<>();
private List<String> variableNames = new ArrayList<>();
/**
* Filter out the modify actions of changing method names, method parameters, variable names and field names in declaration part.
*
* @param actionSets
* @return
*/
public List<HierarchicalActionSet> filterOutUselessActions(List<HierarchicalActionSet> actionSets) {
// Filter out modifications of variable names and method names.
List<HierarchicalActionSet> uselessActions = findoutUselessActions(actionSets);
actionSets.removeAll(uselessActions);
uselessActions.clear();
// Filter out non-UPD modifications, and modifications of variable names and method names.
uselessActions = findoutUselessActionSets(actionSets, true);
actionSets.removeAll(uselessActions);
uselessActions.clear();
return actionSets;
}
private List<HierarchicalActionSet> findoutUselessActionSets(List<HierarchicalActionSet> actionSets, boolean isRoot) {
List<HierarchicalActionSet> uselessActions = new ArrayList<>();
FindActionSet: {
for (HierarchicalActionSet actionSet : actionSets) {
if (!isRoot) {
String actionStr = actionSet.getActionString();
if (actionStr.startsWith("UPD MethodInvocation") || actionStr.startsWith("INS MethodInvocation") || actionStr.startsWith("DEL MethodInvocation")) {
String label = actionSet.getAction().getNode().getLabel();
for (String methodName : methodNames) {
if (actionSet.getActionString().startsWith("UPD MethodInvocation@@" + methodName + "(")
|| actionSet.getActionString().startsWith("INS MethodInvocation@@" + methodName + "(")
|| actionSet.getActionString().startsWith("DEL MethodInvocation@@" + methodName + "(")
|| label.contains("." + methodName + "(")) {
addToUselessActions(actionSet, uselessActions);
break FindActionSet;
}
}
} else if (actionStr.startsWith("UPD SimpleName") || actionStr.startsWith("INS SimpleName") || actionStr.startsWith("DEL SimpleName")) {
String label = actionSet.getAction().getNode().getLabel();
for (String variableName : variableNames) {
if (label.equals(variableName) || label.equals("Name:" + variableName)) {
addToUselessActions(actionSet, uselessActions);
break FindActionSet;
}
}
} else if (actionStr.startsWith("UPD StringLiteral") || actionStr.startsWith("INS StringLiteral")
|| actionStr.startsWith("DEL StringLiteral") || actionStr.startsWith("MOV StringLiteral")
|| actionStr.startsWith("UPD CharacterLiteral") || actionStr.startsWith("INS CharacterLiteral")
|| actionStr.startsWith("DEL CharacterLiteral") || actionStr.startsWith("MOV CharacterLiteral")) {
addToUselessActions(actionSet, uselessActions);
break FindActionSet;
}
List<HierarchicalActionSet> uselessActionSets = findoutUselessActionSets(actionSet.getSubActions(), false);
if (uselessActionSets.size() > 0) {
uselessActions.addAll(uselessActionSets);
break;
}
} else {
if (actionSet.getAstNodeType().endsWith("Statement") || "FieldDeclaration".equals(actionSet.getAstNodeType())) {
uselessActions.addAll(findoutUselessActionSets(actionSet.getSubActions(), false));
} else {
uselessActions.add(actionSet);
}
}
}
}
return uselessActions;
}
private void addToUselessActions(HierarchicalActionSet actionSet, List<HierarchicalActionSet> uselessActions) {
while (actionSet.getParent() != null) {
actionSet = actionSet.getParent();
}
if (!uselessActions.contains(actionSet)) {
uselessActions.add(actionSet);
}
}
/**
* Identify the the modify actions of changing method names, method parameters, variable names and field names in declaration part.
*
* @param actionSets
* @return
*/
private List<HierarchicalActionSet> findoutUselessActions(List<HierarchicalActionSet> actionSets) {
List<HierarchicalActionSet> uselessActions = new ArrayList<>();
for (HierarchicalActionSet actionSet : actionSets) {
String actionType = actionSet.getAstNodeType();
if (actionType.equals("MethodDeclaration")) {
addToUselessActions(actionSet, uselessActions);// INS, DEL: useful?, UPD, except the modifier actions
if (!actionSet.getActionString().startsWith("MOV ")) {
String label = actionSet.getNode().getLabel();
String methodName = label.substring(label.indexOf("MethodName:"));
methodName = methodName.substring(11, methodName.indexOf(","));
methodNames.add(methodName); // "MethodName:***"
// UPD, DEL, INS parameters.
List<HierarchicalActionSet> subActionSets = actionSet.getSubActions();
for (HierarchicalActionSet subActionSet : subActionSets) {
if (subActionSet.getAstNodeType().equals("SingleVariableDeclaration")) {
List<HierarchicalActionSet> subActionSets2 = subActionSet.getSubActions(); // <Type, identifier>
if (subActionSets2.size() == 0) {
String actSetStr = subActionSet.getActionString();
int index1 = actSetStr.indexOf("@@");
int index2 = 0;
if (actSetStr.startsWith("DEL")) {
index2 = actSetStr.indexOf("@AT@");
} else {
index2 = actSetStr.indexOf("@TO@");;
}
actSetStr = actSetStr.substring(index1, index2).trim();
String variableName = actSetStr.substring(actSetStr.lastIndexOf(" "));
variableNames.add(variableName); // "SimpleName:" + variableName TODO: effect range
} else {
HierarchicalActionSet actSet = subActionSets2.get(subActionSets2.size() - 1);
String actStr = actSet.getActionString();
if (actStr.startsWith("UPD SimpleName") || actStr.startsWith("INS SimpleName") || actStr.startsWith("DEL SimpleName")) {
String variableName = actSet.getNode().getLabel();
variableNames.add(variableName); // "SimpleName:" + variableName TODO: effect range
}
}
}
}
}
} else if (actionType.equals("FieldDeclaration") || actionType.equals("VariableDeclarationStatement")) {
// UPD VariableDeclarationFragment
if (!actionSet.getActionString().startsWith("MOV ")) {
List<HierarchicalActionSet> subActionSets = actionSet.getSubActions();
if (subActionSets.size() > 0) {
for (HierarchicalActionSet subActionSet : subActionSets) { // VariableDeclarationFragments
if (identifyUpdateVDF(subActionSet)) {
addToUselessActions(actionSet, uselessActions);
}
}
}
}
} else if (actionType.equals("TryStatement")) {
if (actionSet.getActionString().startsWith("UPD ")) {
List<HierarchicalActionSet> subActionSets = actionSet.getSubActions();
if (subActionSets.size() > 0) {
for (HierarchicalActionSet subActionSet : subActionSets) {
if (subActionSet.getActionString().startsWith("UPD VariableDeclarationExpression")) {
List<HierarchicalActionSet> subActionSets2 = subActionSet.getSubActions(); // VariableDeclarationFragments
for (HierarchicalActionSet subActionSet2 : subActionSets2) {
if (identifyUpdateVDF(subActionSet2)) {
addToUselessActions(actionSet, uselessActions);
}
}
} else {
break;
}
}
}
}
} else if (actionType.equals("EnhancedForStatement")) { // SingleVariableDeclaration
if (!actionSet.getActionString().startsWith("MOV ")) {
List<HierarchicalActionSet> subActionSets = actionSet.getSubActions();
if (subActionSets.size() > 0) {
HierarchicalActionSet subActionSet = subActionSets.get(0);
if (subActionSet.getActionString().startsWith("UPD SingleVariableDeclaration")) {
List<HierarchicalActionSet> subActionSets2 = subActionSet.getSubActions();
for (HierarchicalActionSet subActionSet2 : subActionSets2) { // Type or Identifier
if (subActionSet2.getActionString().startsWith("UPD SimpleName")) {
String variableName = subActionSet2.getNode().getLabel();
variableNames.add(variableName); // "SimpleName:" + variableName TODO: effect range
addToUselessActions(actionSet, uselessActions);
}
}
}
}
}
} else if (actionType.equals("SingleVariableDeclaration")) {
if (!actionSet.getActionString().startsWith("MOV ")) {
List<HierarchicalActionSet> subActionSets2 = actionSet.getSubActions(); // <Type, identifier>
if (subActionSets2.size() == 0) {
String actSetStr = actionSet.getActionString();
int index1 = actSetStr.indexOf("@@");
int index2 = 0;
if (actSetStr.startsWith("DEL")) {
index2 = actSetStr.indexOf("@AT@");
} else {
index2 = actSetStr.indexOf("@TO@");;
}
actSetStr = actSetStr.substring(index1, index2).trim();
String variableName = actSetStr.substring(actSetStr.lastIndexOf(" "));
variableNames.add(variableName); // "SimpleName:" + variableName TODO: effect range
addToUselessActions(actionSet, uselessActions);
} else {
HierarchicalActionSet actSet = subActionSets2.get(subActionSets2.size() - 1);
String actStr = actSet.getActionString();
if (actStr.startsWith("UPD SimpleName") || actStr.startsWith("INS SimpleName") || actStr.startsWith("DEL SimpleName")) {
String variableName = actSet.getNode().getLabel();
variableNames.add(variableName); // "SimpleName:" + variableName TODO: effect range
addToUselessActions(actionSet, uselessActions);
}
}
}
} else {
if (actionSet.getParent() != null) {
while (actionSet.getParent() != null) {
actionSet = actionSet.getParent();
}
if (uselessActions.contains(actionSet)) {
return uselessActions;
} else {
uselessActions.addAll(findoutUselessActions(actionSet.getSubActions()));
}
}
}
}
return uselessActions;
}
/**
* Identify the AST node of this ActionSet is VariableDeclarationFragment or not.
* And, whether the action is happened on the Variable name or not.
*
* @param actionSet
*/
private boolean identifyUpdateVDF(HierarchicalActionSet actionSet) {
String actStr = actionSet.getActionString();
if (actStr.startsWith("UPD VariableDeclarationFragment")
|| actStr.startsWith("INS VariableDeclarationFragment")
|| actStr.startsWith("DEL VariableDeclarationFragment")) {
List<HierarchicalActionSet> subActionSets = actionSet.getSubActions();
if (subActionSets == null || subActionSets.size() == 0) {
// modification of Dimension
return true;
}
HierarchicalActionSet actSet = subActionSets.get(0);
String actSetStr = actSet.getActionString();
if (actSetStr.startsWith("UPD SimpleName") || actSetStr.startsWith("INS SimpleName") || actSetStr.startsWith("DEL SimpleName")) {
String variableName = actSet.getNode().getLabel();
variableNames.add(variableName); // "SimpleName:" + variableName TODO: effect range
return true;
}
}
return false;
}
}
src/main/java/edu/lu/uni/serval/gumtree/regroup/HierarchicalActionSet.java
+14 -72
View File
@@ -1,12 +1,12 @@
package edu.lu.uni.serval.gumtree.regroup;
import com.github.gumtreediff.actions.model.Action;
import com.github.gumtreediff.tree.ITree;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
import com.github.gumtreediff.actions.model.Action;
import com.github.gumtreediff.tree.ITree;
/**
* Hierarchical-level results of GumTree results
*
@@ -29,10 +29,15 @@ public class HierarchicalActionSet implements Comparable<HierarchicalActionSet>,
private List<HierarchicalActionSet> subActions = new ArrayList<>();
private ITree node;
private SimpleTree abstractSimpleTree = null; // semi-source code tree. and AST node type tree
private SimpleTree abstractIdentifierTree = null; // abstract identifier tree
private SimpleTree simpleTree = null; // source code tree and AST node type tree
private SimpleTree originalTree = null; // source code tree.
// source code tree.
public int getBugEndPosition() {
return bugEndPosition;
}
public int getFixEndPosition() {
return fixEndPosition;
}
private int bugEndPosition;
private int fixEndPosition;
@@ -147,49 +152,12 @@ public class HierarchicalActionSet implements Comparable<HierarchicalActionSet>,
this.subActions = subActions;
}
public SimpleTree getAbstractSimpleTree() {
return abstractSimpleTree;
}
public void setAbstractSimpleTree(SimpleTree simpleTree) {
this.abstractSimpleTree = simpleTree;
}
public SimpleTree getAbstractIdentifierTree() {
return abstractIdentifierTree;
}
public void setAbstractIdentifierTree(SimpleTree abstractIdentifierTree) {
this.abstractIdentifierTree = abstractIdentifierTree;
}
public SimpleTree getSimpleTree() {
return simpleTree;
}
public void setSimpleTree(SimpleTree rawTokenTree) {
this.simpleTree = rawTokenTree;
}
public SimpleTree getOriginalTree() {
return originalTree;
}
public void setOriginalTree(SimpleTree originalTree) {
this.originalTree = originalTree;
}
public int getBugEndPosition() {
return bugEndPosition;
}
public void setBugEndPosition(int bugEndPosition) {
this.bugEndPosition = bugEndPosition;
}
public int getFixEndPosition() {
return fixEndPosition;
}
public void setFixEndPosition(int fixEndPosition) {
this.fixEndPosition = fixEndPosition;
@@ -197,10 +165,7 @@ public class HierarchicalActionSet implements Comparable<HierarchicalActionSet>,
@Override
public int compareTo(HierarchicalActionSet o) {
// int result = this.startPosition.compareTo(o.startPosition);
// if (result == 0) {
// result = this.length >= o.length ? -1 : 1;
// }
return this.startPosition.compareTo(o.startPosition);//this.action.compareTo(o.action);
}
@@ -238,28 +203,5 @@ public class HierarchicalActionSet implements Comparable<HierarchicalActionSet>,
return str;
}
public String toASTNodeLevelAction() {
if (strList.size() == 0) {
toString();
}
String astNodeStr = "";
for (String str : strList) {
astNodeStr += str.substring(0, str.indexOf("@@")) + "\n";
}
return astNodeStr;
}
public String toRawCodeLevelAction() {
if (strList.size() == 0) {
toString();
}
String astNodeStr = "";
for (String str : strList) {
str = str.substring(0, str.indexOf(" @AT@")) + "\n";
int index1 = str.indexOf(" ") + 1;
int index2 = str.indexOf("@@") + 2;
astNodeStr += str.substring(0, index1) + str.substring(index2);
}
return astNodeStr;
}
}
src/main/java/edu/lu/uni/serval/gumtree/regroup/HierarchicalRegrouper.java
+5 -19
View File
@@ -1,22 +1,13 @@
package edu.lu.uni.serval.gumtree.regroup;
import java.io.File;
import com.github.gumtreediff.actions.model.*;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.FixPattern.utils.ASTNodeMap;
import edu.lu.uni.serval.utils.ListSorter;
import java.util.ArrayList;
import java.util.List;
import com.github.gumtreediff.actions.model.Action;
import com.github.gumtreediff.actions.model.Addition;
import com.github.gumtreediff.actions.model.Delete;
import com.github.gumtreediff.actions.model.Insert;
import com.github.gumtreediff.actions.model.Move;
import com.github.gumtreediff.actions.model.Update;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.FixPattern.utils.ASTNodeMap;
import edu.lu.uni.serval.FixPattern.utils.Checker;
import edu.lu.uni.serval.gumtree.GumTreeComparer;
import edu.lu.uni.serval.utils.ListSorter;
/**
* Regroup GumTree results to a hierarchical construction.
*
@@ -209,11 +200,6 @@ public class HierarchicalRegrouper {
}
// } else if (parent.getType() == 31) { // method declaration
// int type = action.getNode().getType();
// if (Checker.isStatement(type)) {// statements
// return null;
// }
}
for (Action act : actions) {
src/main/java/edu/lu/uni/serval/gumtree/regroup/HierarchicalRegrouperForC.java
-300
View File
@@ -1,300 +0,0 @@
package edu.lu.uni.serval.gumtree.regroup;
import java.util.ArrayList;
import java.util.List;
import com.github.gumtreediff.actions.model.Action;
import com.github.gumtreediff.actions.model.Addition;
import com.github.gumtreediff.actions.model.Delete;
import com.github.gumtreediff.actions.model.Insert;
import com.github.gumtreediff.actions.model.Move;
import com.github.gumtreediff.actions.model.Update;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.FixPattern.utils.CNodeMap;
import edu.lu.uni.serval.utils.ListSorter;
/**
* Regroup GumTree results to a hierarchical construction.
*
* @author kui.liu
*
*/
public class HierarchicalRegrouperForC {
// public static void main(String[] args) {
// GumTreeComparer com = new GumTreeComparer();
// File cFile1 = new File("/Users/anilkoyuncu/bugStudy/dataset/GumTreeInput/linux-stable/prevFiles/prev_0a3d00_b404bc_drivers#pci#iov.c");
// File cFile2 = new File("/Users/anilkoyuncu/bugStudy/dataset/GumTreeInput/linux-stable/revFiles/0a3d00_b404bc_drivers#pci#iov.c");
// List<Action> action = com.compareTwoFilesWithGumTreeForCCode(cFile1, cFile2);
// List<HierarchicalActionSet> actionSet = new HierarchicalRegrouperForC().regroupGumTreeResults(action);
// System.out.println(actionSet);
// }
List<HierarchicalActionSet> actionSets = new ArrayList<>();
public List<HierarchicalActionSet> regroupGumTreeResults(List<Action> actions) {
/*
* First, sort actions by their positions.
*/
// List<Action> actions = new ListSorter<Action>(actionsArgu).sortAscending();
// if (actions == null) {
// actions = actionsArgu;
// }
/*
* Second, group actions by their positions.
*/
HierarchicalActionSet actionSet = null;
for(Action act : actions){
Action parentAct = findParentAction(act, actions);
if (parentAct == null) {
actionSet = createActionSet(act, parentAct, null);
actionSets.add(actionSet);
} else {
if (!addToAactionSet(act, parentAct, actionSets)) {
// The index of the parent action in the actions' list is larger than the index of this action.
actionSet = createActionSet(act, parentAct, null);
actionSets.add(actionSet);
}
}
}
/*
* Third, add the subActionSet to its parent ActionSet.
*/
List<HierarchicalActionSet> reActionSets = new ArrayList<>();
for (HierarchicalActionSet actSet : actionSets) {
Action parentAct = actSet.getParentAction();
if (parentAct != null) {
addToActionSets(actSet, parentAct, actionSets);
} else {
// TypeDeclaration, FieldDeclaration, MethodDeclaration, Statement.
// CatchClause, ConstructorInvocation, SuperConstructorInvocation, SwitchCase
// String astNodeType = actSet.getAstNodeType();
// if (astNodeType.endsWith("TypeDeclaration") || astNodeType.endsWith("FieldDeclaration") || astNodeType.endsWith("EnumDeclaration") ||
// astNodeType.endsWith("MethodDeclaration") || astNodeType.endsWith("Statement") ||
// astNodeType.endsWith("ConstructorInvocation") || astNodeType.endsWith("CatchClause") || astNodeType.endsWith("SwitchCase")) {
reActionSets.add(actSet);
// }
}
}
return reActionSets;
}
private HierarchicalActionSet createActionSet(Action act, Action parentAct, HierarchicalActionSet parent) {
HierarchicalActionSet actionSet = new HierarchicalActionSet();
actionSet.setAction(act);
String actStr = parseAction(act.toString());
actionSet.setActionString(actStr);
actionSet.setParentAction(parentAct);
actionSet.setNode(act.getNode());
actionSet.setParent(parent);
int bugStartLineNum = 0;
int bugEndLineNum = 0;
int fixStartLineNum = 0;
int fixEndLineNum = 0;
if (actStr.startsWith("INS")) {
Insert insert = (Insert) act;
// int pos = insert.getPos();//index position of the new node in the children array list of its corresponding old parent node.
ITree newTree = insert.getNode();
fixStartLineNum = newTree.getPos();
fixEndLineNum = newTree.getLength();
} else {
ITree tree = act.getNode();
bugStartLineNum = tree.getPos();
bugEndLineNum = tree.getLength();
if (actStr.startsWith("UPD")) {
Update update = (Update) act;
ITree newTree = update.getNewNode();
fixStartLineNum = newTree.getPos();
fixEndLineNum = newTree.getLength();
} else if (actStr.startsWith("MOV")) {
Move update = (Move) act;
ITree newTree = update.getNewNode();
fixStartLineNum = newTree.getPos();
fixEndLineNum = newTree.getLength();
}
}
actionSet.setBugStartLineNum(bugStartLineNum);
actionSet.setBugEndLineNum(bugEndLineNum);
actionSet.setFixStartLineNum(fixStartLineNum);
actionSet.setFixEndLineNum(fixEndLineNum);
return actionSet;
}
private String parseAction(String actStr1) {
// UPD 25@@!a from !a to isTrue(a) at 69
String[] actStrArrays = actStr1.split("@@");
String actStr = "";
int length = actStrArrays.length;
for (int i =0; i < length - 1; i ++) {
String actStrFrag = actStrArrays[i];
int index = actStrFrag.lastIndexOf(" ") + 1;
String nodeType = actStrFrag.substring(index);
if (!"".equals(nodeType)) {
if (Character.isDigit(nodeType.charAt(0)) || (nodeType.startsWith("-") && Character.isDigit(nodeType.charAt(1)))) {
try {
int typeInt = Integer.parseInt(nodeType);
if (CNodeMap.map.containsKey(typeInt)) {
String type = CNodeMap.map.get(Integer.parseInt(nodeType));
nodeType = type;
}
} catch (NumberFormatException e) {
nodeType = actStrFrag.substring(index);
}
}
}
actStrFrag = actStrFrag.substring(0, index) + nodeType + "@@";
actStr += actStrFrag;
}
actStr += actStrArrays[length - 1];
return actStr;
}
private void addToActionSets(HierarchicalActionSet actionSet, Action parentAct, List<HierarchicalActionSet> actionSets) {
Action act = actionSet.getAction();
for (HierarchicalActionSet actSet : actionSets) {
if (actSet.equals(actionSet)) continue;
Action action = actSet.getAction();
if (!areRelatedActions(action, act)) continue;
if (action.equals(parentAct)) { // actSet is the parent of actionSet.
actionSet.setParent(actSet);
actSet.getSubActions().add(actionSet);
sortSubActions(actSet);
break;
} else {
if (isPossibileSubAction(action, act)) {
// SubAction range： startPosition2 <= startPosition && length <= length2
addToActionSets(actionSet, parentAct, actSet.getSubActions());
}
}
}
}
private boolean isPossibileSubAction(Action parent, Action child) {
if ((parent instanceof Update && !(child instanceof Addition))
|| (parent instanceof Delete && child instanceof Delete)
|| (parent instanceof Insert && (child instanceof Insert))) {
int startPosition = child.getPosition();
int length = child.getLength();
int startPosition2 = parent.getPosition();
int length2 = parent.getLength();
if (!(startPosition2 <= startPosition && length <= length2)) {
// when act is not the sub-set of action.
return false;
}
}
return true;
}
private void sortSubActions(HierarchicalActionSet actionSet) {
ListSorter<HierarchicalActionSet> sorter = new ListSorter<HierarchicalActionSet>(actionSet.getSubActions());
List<HierarchicalActionSet> subActions = sorter.sortAscending();
if (subActions != null) {
actionSet.setSubActions(subActions);
}
}
private boolean addToAactionSet(Action act, Action parentAct, List<HierarchicalActionSet> actionSets) {
for(HierarchicalActionSet actionSet : actionSets) {
Action action = actionSet.getAction();
if (!areRelatedActions(action, act)) continue;
if (action.equals(parentAct)) { // actionSet is the parent of actSet.
HierarchicalActionSet actSet = createActionSet(act, actionSet.getAction(), actionSet);
actionSet.getSubActions().add(actSet);
sortSubActions(actionSet);
return true;
} else {
if (isPossibileSubAction(action, act)) {
// SubAction range: startPosition2 <= startPosition && startPosition + length <= startP + length2
List<HierarchicalActionSet> subActionSets = actionSet.getSubActions();
if (subActionSets.size() > 0) {
boolean added = addToAactionSet(act, parentAct, subActionSets);
if (added) {
return true;
} else {
continue;
}
}
}
}
}
return false;
}
List<Action> newParentActions = new ArrayList<>();
private Action findParentAction(Action action, List<Action> actions) {
ITree parent = action.getNode().getParent();
if (parent == null) return null;
if (action instanceof Addition) {
parent = ((Addition) action).getParent(); // parent in the fixed source code tree
}
for (Action act : actions) {
if (act.getNode().equals(parent)) {
if (areRelatedActions(act, action)) {
return act;
}
}
}
for (Action act : newParentActions) {
if (act.getNode().equals(parent)) {
if (areRelatedActions(act, action)) {
return act;
}
}
}
ITree tree = action.getNode();
Action parentAction = null;
if (!isStatement(tree)) {
parentAction = new Update(parent, action.getNode().getParent());
newParentActions.add(parentAction);
Action higherParentAct = findParentAction(parentAction, actions);
HierarchicalActionSet actionSet = null;
if (higherParentAct == null) {
actionSet = createActionSet(parentAction, higherParentAct, null);
actionSets.add(actionSet);
} else {
if (!addToAactionSet(parentAction, higherParentAct, actionSets)) {
// The index of the parent action in the actions' list is larger than the index of this action.
actionSet = createActionSet(parentAction, higherParentAct, null);
actionSets.add(actionSet);
}
}
}
return parentAction;
}
private boolean isStatement(ITree tree) {
int nodeType = tree.getType();
if (nodeType == 280002 || nodeType == 280001 || nodeType == 310200 || nodeType == 280100
|| nodeType == 300100 || nodeType == 280003 || nodeType == 310100 || nodeType == 310300
|| 260300 == nodeType || nodeType == 460000) {// TODO
return true;
}
return false;
}
private boolean areRelatedActions(Action parent, Action child) {
if (parent instanceof Move && !(child instanceof Move)) {// If action is MOV, its children must be MOV.
return false;
}
if (parent instanceof Delete && !(child instanceof Delete)) {// If action is DEL, its children must be DEL.
return false;
}
if (parent instanceof Insert && !(child instanceof Addition)) {// If action is INS, its children must be MOV or INS.
return false;
}
return true;
}
}
src/main/java/edu/lu/uni/serval/gumtree/regroup/HunkActionFilter.java
-625
View File
@@ -1,625 +0,0 @@
package edu.lu.uni.serval.gumtree.regroup;
import java.io.File;
import java.util.ArrayList;
import java.util.List;
import org.eclipse.jdt.core.dom.CompilationUnit;
import com.github.gumtreediff.actions.model.Move;
import com.github.gumtreediff.actions.model.Update;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.FixPattern.utils.Checker;
import edu.lu.uni.serval.FixPatternParser.CUCreator;
import edu.lu.uni.serval.FixPatternParser.violations.Violation;
import edu.lu.uni.serval.diffentry.DiffEntryHunk;
public class HunkActionFilter {
/**
* Filter out the modify actions, which are not in the DiffEntry hunks, without considering the same parent node.
*
* @param hunks
* @param actionSets
* @return
*/
public List<HierarchicalActionSet> filterActionsByDiffEntryHunk(List<DiffEntryHunk> hunks,
List<HierarchicalActionSet> actionSets, File revFile, File prevFile) {
List<HierarchicalActionSet> uselessActions = new ArrayList<>();
CUCreator cuCreator = new CUCreator();
CompilationUnit prevUnit = cuCreator.createCompilationUnit(prevFile);
CompilationUnit revUnit = cuCreator.createCompilationUnit(revFile);
if (prevUnit == null || revUnit == null) {
return uselessActions;
}
for (HierarchicalActionSet actionSet : actionSets) {
// position of buggy statements
int startPosition = 0;
int endPosition = 0;
int startLine = 0;
int endLine = 0;
// position of fixed statements
int startPosition2 = 0;
int endPosition2 = 0;
int startLine2 = 0;
int endLine2 = 0;
String actionStr = actionSet.getActionString();
if (actionStr.startsWith("INS")) {
startPosition2 = actionSet.getStartPosition();
endPosition2 = startPosition2 + actionSet.getLength();
List<Move> firstAndLastMov = getFirstAndLastMoveAction(actionSet);
if (firstAndLastMov != null) {
startPosition = firstAndLastMov.get(0).getNode().getPos();
ITree lastTree = firstAndLastMov.get(1).getNode();
endPosition = lastTree.getPos() + lastTree.getLength();
}
} else {
startPosition = actionSet.getStartPosition(); // range of actions
endPosition = startPosition + actionSet.getLength();
if (actionStr.startsWith("UPD")) {
Update update = (Update) actionSet.getAction();
ITree newNode = update.getNewNode();
startPosition2 = newNode.getPos();
endPosition2 = startPosition2 + newNode.getLength();
}
}
startLine = startPosition == 0 ? 0 : prevUnit.getLineNumber(startPosition);
endLine = endPosition == 0 ? 0 : prevUnit.getLineNumber(endPosition);
startLine2 = startPosition2 == 0 ? 0 : revUnit.getLineNumber(startPosition2);
endLine2 = endPosition2 == 0 ? 0 : revUnit.getLineNumber(endPosition2);
for (DiffEntryHunk hunk : hunks) {
int bugStartLine = hunk.getBugLineStartNum();
int bugRange = hunk.getBugRange();
int fixStartLine = hunk.getFixLineStartNum();
int fixRange = hunk.getFixRange();
if (actionStr.startsWith("INS")) {
if (fixStartLine + fixRange < startLine2) {
continue;
}
if (endLine2 < fixStartLine ) {
uselessActions.add(actionSet);
}
} else {
if (bugStartLine + bugRange < startLine) {
continue;
}
if (endLine < bugStartLine ) {
uselessActions.add(actionSet);
}
break;
}
}
actionSet.setBugStartLineNum(startLine);
actionSet.setBugEndLineNum(endLine);
actionSet.setFixStartLineNum(startLine2);
actionSet.setFixEndLineNum(endLine2);
}
actionSets.removeAll(uselessActions);
uselessActions.clear();
return actionSets;
}
/**
* Filter out the modify actions, which are not in the DiffEntry hunks, with considering the same parent node.
*
* @param hunks
* @param actionSets
* @return
*/
public List<HunkFixPattern> filterActionsByDiffEntryHunk2(List<DiffEntryHunk> hunks,
List<HierarchicalActionSet> actionSets, File revFile, File prevFile) {
List<HunkFixPattern> allHunkFixPatterns = new ArrayList<>();
CUCreator cuCreator = new CUCreator();
CompilationUnit prevUnit = cuCreator.createCompilationUnit(prevFile);
CompilationUnit revUnit = cuCreator.createCompilationUnit(revFile);
if (prevUnit == null || revUnit == null) {
return allHunkFixPatterns;
}
int i = 0;
int size = actionSets.size();
for (DiffEntryHunk hunk : hunks) {
int hunkBugStartLine = hunk.getBugLineStartNum();
int hunkBugRange = hunk.getBugRange();
int hunkFixStartLine = hunk.getFixLineStartNum();
int hunkFixRange = hunk.getFixRange();
for (; i < size; i ++) {
HierarchicalActionSet actionSet = actionSets.get(i);
int actionBugStartLine = actionSet.getBugStartLineNum();
if (actionBugStartLine == 0) {
actionBugStartLine = setLineNumbers(actionSet, prevUnit, revUnit);
}
int actionBugEndLine = actionSet.getBugEndLineNum();
int actionFixStartLine = actionSet.getFixStartLineNum();
int actionFixEndLine = actionSet.getFixEndLineNum();
String actionStr = actionSet.getActionString();
ITree previousParent = null;
List<HierarchicalActionSet> hunkActionSets = new ArrayList<>();
if (actionStr.startsWith("INS")) {
if (hunkFixStartLine + hunkFixRange < actionFixStartLine) {
addHunkActionSets(hunkActionSets, allHunkFixPatterns, hunk);// save the previous non-null hunkFixPattern.
break;
}
if (actionFixEndLine >= hunkFixStartLine ) {
ITree parent = addToHunkActionSets(actionSet, hunkActionSets, allHunkFixPatterns, previousParent, hunk);
if (parent != null) {
if (parent != previousParent) {
hunkActionSets = new ArrayList<>();
}
hunkActionSets.add(actionSet);
} else if (hunkActionSets.size() > 0) {
hunkActionSets = new ArrayList<>();
}
previousParent = parent;
}
} else { // UPD, DEL, MOV
if (hunkBugStartLine + hunkBugRange < actionBugStartLine) {
addHunkActionSets(hunkActionSets, allHunkFixPatterns, hunk);// save the previous non-null hunkFixPattern.
break;
}
if (actionBugEndLine >= hunkBugStartLine ) {
ITree parent = addToHunkActionSets(actionSet, hunkActionSets, allHunkFixPatterns, previousParent, hunk);
if (parent != null) { // same parent
if (parent != previousParent) {
hunkActionSets = new ArrayList<>();
}
hunkActionSets.add(actionSet);
} else if (hunkActionSets.size() > 0) {
hunkActionSets = new ArrayList<>();
}
previousParent = parent;
}
}
addHunkActionSets(hunkActionSets, allHunkFixPatterns, hunk);
}
}
return allHunkFixPatterns;
}
private int getEndPosition(List<ITree> children) {
int endPosition = 0;
for (int i = 0, size = children.size(); i < size; i ++) {
ITree child = children.get(i);
int type = child.getType();
if (Checker.isStatement2(type)) {
if ( i > 0) {
child = children.get(i - 1);
endPosition = child.getPos() + child.getLength();
} else {
endPosition = child.getPos() - 1;
}
break;
}
}
return endPosition;
}
private void addHunkActionSets(List<HierarchicalActionSet> hunkActionSets, List<HunkFixPattern> allHunkFixPatterns, DiffEntryHunk hunk) {
if (hunkActionSets.size() > 0) {
HunkFixPattern hunkFixPattern = new HunkFixPattern(hunk, hunkActionSets);
allHunkFixPatterns.add(hunkFixPattern);
}
}
private ITree addToHunkActionSets(HierarchicalActionSet actionSet, List<HierarchicalActionSet> hunkActionSets,
List<HunkFixPattern> allHunkFixPatterns, ITree previousParent, DiffEntryHunk hunk) {
String astNodeType = actionSet.getAstNodeType();
if ("FieldDeclaration".equals(astNodeType)) {
addHunkActionSets(hunkActionSets, allHunkFixPatterns, hunk);
hunkActionSets = new ArrayList<>();
hunkActionSets.add(actionSet);
HunkFixPattern hunkFixPattern = new HunkFixPattern(hunk, hunkActionSets);
allHunkFixPatterns.add(hunkFixPattern);
return null;
} else {
ITree currentParent = actionSet.getNode().getParent();
if (previousParent == null) {
previousParent = currentParent;
} else {
if (!previousParent.equals(currentParent)) {
HunkFixPattern hunkFixPattern = new HunkFixPattern(hunk, hunkActionSets);
allHunkFixPatterns.add(hunkFixPattern);
previousParent = currentParent;
}
}
return previousParent;
}
}
private List<Move> getFirstAndLastMoveAction(HierarchicalActionSet gumTreeResult) {
List<Move> firstAndLastMoveActions = new ArrayList<>();
List<HierarchicalActionSet> actions = new ArrayList<>();
actions.addAll(gumTreeResult.getSubActions());
if (actions.size() == 0) {
return null;
}
Move firstMoveAction = null;
Move lastMoveAction = null;
while (actions.size() > 0) {
List<HierarchicalActionSet> subActions = new ArrayList<>();
for (HierarchicalActionSet action : actions) {
subActions.addAll(action.getSubActions());
if (action.toString().startsWith("MOV")) {
if (firstMoveAction == null) {
firstMoveAction = (Move) action.getAction();
lastMoveAction = (Move) action.getAction();
} else {
int startPosition = action.getStartPosition();
int length = action.getLength();
int startPositionFirst = firstMoveAction.getPosition();
int startPositionLast = lastMoveAction.getPosition();
int lengthLast = lastMoveAction.getNode().getLength();
if (startPosition < startPositionFirst || (startPosition == startPositionFirst && length > firstMoveAction.getLength())) {
firstMoveAction = (Move) action.getAction();
}
if ((startPosition + length) > (startPositionLast + lengthLast)) {
lastMoveAction = (Move) action.getAction();
}
}
}
}
actions.clear();
actions.addAll(subActions);
}
if (firstMoveAction == null) {
return null;
}
firstAndLastMoveActions.add(firstMoveAction);
firstAndLastMoveActions.add(lastMoveAction);
return firstAndLastMoveActions;
}
/**
* Filter out the modify actions, which are not in the DiffEntry hunks, without considering the same parent node.
*
* @param violations
* @param actionSets
* @param revFile
* @param prevFile
* @return
*/
public List<Violation> filterActionsByModifiedRange(List<Violation> violations,
List<HierarchicalActionSet> actionSets, File revFile, File prevFile) {
List<Violation> selectedViolations = new ArrayList<>();
CUCreator cuCreator = new CUCreator();
CompilationUnit prevUnit = cuCreator.createCompilationUnit(prevFile);
CompilationUnit revUnit = cuCreator.createCompilationUnit(revFile);
if (prevUnit == null || revUnit == null) {
return selectedViolations;
}
for (Violation violation : violations) {
int startLine = violation.getStartLineNum();
int endLine = violation.getEndLineNum();
int bugStartLine = violation.getBugStartLineNum();
int bugEndLine = violation.getBugEndLineNum();
int fixStartLine = violation.getFixStartLineNum();
int fixEndLine = violation.getFixEndLineNum();
for (HierarchicalActionSet actionSet : actionSets) {
int actionBugStartLine = actionSet.getBugStartLineNum();
if (actionBugStartLine == 0) {
actionBugStartLine = setLineNumbers(actionSet, prevUnit, revUnit);
}
int actionBugEndLine = actionSet.getBugEndLineNum();
int actionFixStartLine = actionSet.getFixStartLineNum();
int actionFixEndLine = actionSet.getFixEndLineNum();
String actionStr = actionSet.getActionString();
if (actionStr.startsWith("INS")) {
if (fixStartLine <= actionFixStartLine && actionFixEndLine <= fixEndLine) {
if (actionBugStartLine != 0) {
if (startLine <= actionBugEndLine && endLine >= actionBugStartLine) {
violation.getActionSets().add(actionSet);
}
} else {
violation.getActionSets().add(actionSet);
}
}
} else {
// if (bugEndLine < actionBugStartLine) {
// break;
// }
if (bugStartLine <= actionBugStartLine && actionBugEndLine <= bugEndLine) {
if (startLine <= actionBugEndLine && endLine >= actionBugStartLine) {
violation.getActionSets().add(actionSet);
}
}
}
}
if (violation.getActionSets().size() > 0) {
selectedViolations.add(violation);
}
}
return selectedViolations;
}
/**
* Match hierarchical actions with code change diffs for C code.
*
* @param diffentryHunks
* @param actionSets
* @param revFile
* @param prevFile
* @return
*/
public List<DiffEntryHunk> matchActionsByDiffEntryForC(List<DiffEntryHunk> diffentryHunks, List<HierarchicalActionSet> actionSets) {
List<DiffEntryHunk> selectedViolations = new ArrayList<>();
for (DiffEntryHunk diffentryHunk : diffentryHunks) {
int bugHunkStartLine = diffentryHunk.getBugLineStartNum();
int bugHunkEndLine = bugHunkStartLine + diffentryHunk.getBugRange() - 1;
int fixHunkStartLine = diffentryHunk.getFixLineStartNum();
int fixHunkEndLine = fixHunkStartLine + diffentryHunk.getFixedHunkSize() - 1;
for (HierarchicalActionSet actionSet : actionSets) {
String actionStr = actionSet.getActionString();
if (actionStr.startsWith("INS")) {
int actionFixStartLine = actionSet.getFixStartLineNum();
int actionFixEndLine = actionSet.getFixEndLineNum();
if (fixHunkStartLine <= actionFixEndLine && actionFixStartLine <= fixHunkEndLine ) {
diffentryHunk.getActionSets().add(actionSet);
}
} else {
int actionBugStartLine = actionSet.getBugStartLineNum();
int actionBugEndLine = actionSet.getBugEndLineNum();
if (bugHunkStartLine <= actionBugEndLine && actionBugStartLine <= bugHunkEndLine) {
diffentryHunk.getActionSets().add(actionSet);
}
}
}
if (diffentryHunk.getActionSets().size() > 0) {
selectedViolations.add(diffentryHunk);
}
}
return selectedViolations;
}
/**
* Filter out the modify actions, which are not in the DiffEntry hunks, with considering the same parent node.
*
* @param violations
* @param actionSets
* @param revFile
* @param prevFile
* @return
*/
public List<DiffEntryHunk> filterActionsByModifiedRange2(List<DiffEntryHunk> diffentryHunks,
List<HierarchicalActionSet> actionSets, File revFile, File prevFile) {
List<DiffEntryHunk> selectedViolations = new ArrayList<>();
CUCreator cuCreator = new CUCreator();
CompilationUnit prevUnit = cuCreator.createCompilationUnit(prevFile);
CompilationUnit revUnit = cuCreator.createCompilationUnit(revFile);
if (prevUnit == null || revUnit == null) {
return selectedViolations;
}
for (DiffEntryHunk diffentryHunk : diffentryHunks) {
// int violationEndLine = violationStartLine + diffentryHunk.getBugRange();
int bugHunkStartLine = diffentryHunk.getBugLineStartNum();
int bugHunkEndLine = bugHunkStartLine + diffentryHunk.getBugRange() - 1;
int fixHunkStartLine = diffentryHunk.getFixLineStartNum();
int fixHunkEndLine = fixHunkStartLine + diffentryHunk.getFixedHunkSize() - 1;
for (HierarchicalActionSet actionSet : actionSets) {
int actionBugStartLine = actionSet.getBugStartLineNum();
if (actionBugStartLine == 0) {
actionBugStartLine = setLineNumbers(actionSet, prevUnit, revUnit);
}
int actionBugEndLine = actionSet.getBugEndLineNum();
int actionFixStartLine = actionSet.getFixStartLineNum();
int actionFixEndLine = actionSet.getFixEndLineNum();
String actionStr = actionSet.getActionString();
if (actionStr.startsWith("INS")) {
if (fixHunkStartLine <= actionFixEndLine && fixHunkEndLine >= actionFixStartLine ) {
if (actionBugStartLine != 0) {
diffentryHunk.getActionSets().add(actionSet);
}
}
} else {
if (bugHunkStartLine <= actionBugEndLine && bugHunkEndLine >= actionBugStartLine) {
diffentryHunk.getActionSets().add(actionSet);
}
}
}
if (diffentryHunk.getActionSets().size() > 0) {
selectedViolations.add(diffentryHunk);
}
}
return selectedViolations;
}
/**
* Filter out the modify actions, which are not in the DiffEntry hunks, without considering DiffEntry hunks.
*
* @param violations
* @param actionSets
* @param revFile
* @param prevFile
* @return
*/
public List<Violation> filterActionsByModifiedRange3(List<Violation> violations,
List<HierarchicalActionSet> actionSets, File revFile, File prevFile) {
List<Violation> selectedViolations = new ArrayList<>();
CUCreator cuCreator = new CUCreator();
CompilationUnit prevUnit = cuCreator.createCompilationUnit(prevFile);
CompilationUnit revUnit = cuCreator.createCompilationUnit(revFile);
if (prevUnit == null || revUnit == null) {
return selectedViolations;
}
for (Violation violation : violations) {
int startLine = violation.getStartLineNum();
int endLine = violation.getEndLineNum();
// ITree parent = null;
// List<HierarchicalActionSet> actionSetsWithSameParent = new ArrayList<>(); //TODO
for (HierarchicalActionSet actionSet : actionSets) {
int actionBugStartLine = actionSet.getStartPosition();
int actionBugEndLine = actionSet.getLength();
int actionFixStartLine = actionSet.getFixStartLineNum();
int actionFixEndLine = actionSet.getFixEndLineNum();
String actionStr = actionSet.getActionString();
if (actionStr.startsWith("INS")) { // FIXME It is impossible to locate the INS action by the buggy line range.
if (startLine <= actionFixStartLine && actionFixEndLine <= endLine) {
if (actionBugStartLine != 0) {
if (startLine <= actionBugEndLine && endLine >= actionBugStartLine) {
violation.getActionSets().add(actionSet);
}
} else {
violation.getActionSets().add(actionSet);
}
}
} else {
// if (endLine < actionBugStartLine) {
// break;
// }
if (startLine <= actionBugStartLine && actionBugEndLine <= endLine) {
if (startLine <= actionBugEndLine && endLine >= actionBugStartLine) {
violation.getActionSets().add(actionSet);
}
}
}
}
if (violation.getActionSets().size() > 0) {
selectedViolations.add(violation);
}
}
return selectedViolations;
}
private int setLineNumbers(HierarchicalActionSet actionSet, CompilationUnit prevUnit, CompilationUnit revUnit) {
int actionBugStartLine;
int actionBugEndLine;
int actionFixStartLine;
int actionFixEndLine;
// position of buggy statements
int bugStartPosition = 0;
int bugEndPosition = 0;
// position of fixed statements
int fixStartPosition = 0;
int fixEndPosition = 0;
String actionStr = actionSet.getActionString();
if (actionStr.startsWith("INS")) {
fixStartPosition = actionSet.getStartPosition();
fixEndPosition = fixStartPosition + actionSet.getLength();
List<Move> firstAndLastMov = getFirstAndLastMoveAction(actionSet);
if (firstAndLastMov != null) {
bugStartPosition = firstAndLastMov.get(0).getNode().getPos();
ITree lastTree = firstAndLastMov.get(1).getNode();
bugEndPosition = lastTree.getPos() + lastTree.getLength();
}
} else {
bugStartPosition = actionSet.getStartPosition(); // range of actions
bugEndPosition = bugStartPosition + actionSet.getLength();
if (actionStr.startsWith("UPD")) {
Update update = (Update) actionSet.getAction();
ITree newNode = update.getNewNode();
fixStartPosition = newNode.getPos();
fixEndPosition = fixStartPosition + newNode.getLength();
String astNodeType = actionSet.getAstNodeType();
// if (Checker.withBlockStatement(newNode.getType())) {
//// List<ITree> children = update.getNode().getChildren();
//// bugEndPosition = getEndPosition(children);
//// List<ITree> newChildren = newNode.getChildren();
//// fixEndPosition = getEndPosition(newChildren);
// } else
if ("TypeDeclaration".equals(astNodeType)) {
bugEndPosition = getClassBodyStartPosition(update.getNode());
fixEndPosition = getClassBodyStartPosition(newNode);
} else if ("MethodDeclaration".equals(astNodeType)) {
List<ITree> children = update.getNode().getChildren();
bugEndPosition = getEndPosition(children);
List<ITree> newChildren = newNode.getChildren();
fixEndPosition = getEndPosition(newChildren);
}
if (fixEndPosition == 0) {
fixEndPosition = fixStartPosition + newNode.getLength();
}
} else if (actionStr.startsWith("DEL")) {
ITree buggyTree = actionSet.getNode();
int type = buggyTree.getType();
if (type == 55) { // TypeDeclaration
bugEndPosition = getClassBodyStartPosition(buggyTree);
} else if (type == 31 || Checker.withBlockStatement(type)) {//MethodDeclaration && Block-Statements
List<ITree> children = buggyTree.getChildren();
bugEndPosition = getEndPosition(children);
}
}
if (bugEndPosition == 0) {
bugEndPosition = bugStartPosition + actionSet.getLength();
}
}
actionBugStartLine = bugStartPosition == 0 ? 0 : prevUnit.getLineNumber(bugStartPosition);
actionBugEndLine = bugEndPosition == 0 ? 0 : prevUnit.getLineNumber(bugEndPosition);
actionFixStartLine = fixStartPosition == 0 ? 0 : revUnit.getLineNumber(fixStartPosition);
actionFixEndLine = fixEndPosition == 0 ? 0 : revUnit.getLineNumber(fixEndPosition);
actionSet.setBugStartLineNum(actionBugStartLine);
actionSet.setBugEndLineNum(actionBugEndLine);
actionSet.setFixStartLineNum(actionFixStartLine);
actionSet.setFixEndLineNum(actionFixEndLine);
actionSet.setBugEndPosition(bugEndPosition);
actionSet.setFixEndPosition(fixEndPosition);
return actionBugStartLine;
}
private int getClassBodyStartPosition(ITree tree) {
List<ITree> children = tree.getChildren();
for (int i = 0, size = children.size(); i < size; i ++) {
ITree child = children.get(i);
int type = child.getType();
// Modifier, NormalAnnotation, MarkerAnnotation, SingleMemberAnnotation
if (type != 83 && type != 77 && type != 78 && type != 79
&& type != 5 && type != 39 && type != 43 && type != 74 && type != 75
&& type != 76 && type != 84 && type != 87 && type != 88 && type != 42) {
// ArrayType, PrimitiveType, SimpleType, ParameterizedType,
// QualifiedType, WildcardType, UnionType, IntersectionType, NameQualifiedType, SimpleName
if (i > 0) {
child = children.get(i - 1);
return child.getPos() + child.getLength() + 1;
} else {
return child.getPos() - 1;
}
}
}
return 0;
}
}
src/main/java/edu/lu/uni/serval/gumtree/regroup/HunkFixPattern.java
-27
View File
@@ -1,27 +0,0 @@
package edu.lu.uni.serval.gumtree.regroup;
import java.util.List;
import edu.lu.uni.serval.diffentry.DiffEntryHunk;
public class HunkFixPattern {
private DiffEntryHunk hunk;
private List<HierarchicalActionSet> hunkActionSets;
public HunkFixPattern(DiffEntryHunk hunk, List<HierarchicalActionSet> hunkActionSets) {
super();
this.hunk = hunk;
this.hunkActionSets = hunkActionSets;
}
public DiffEntryHunk getHunk() {
return hunk;
}
public List<HierarchicalActionSet> getHunkActionSets() {
return hunkActionSets;
}
}
src/main/java/edu/lu/uni/serval/gumtree/regroup/SimpleTree.java
-68
View File
@@ -1,68 +0,0 @@
package edu.lu.uni.serval.gumtree.regroup;
import java.util.ArrayList;
import java.util.List;
public class SimpleTree {
private String nodeType;
private String label;
private SimpleTree parent;
private List<SimpleTree> children = new ArrayList<>();
public String getNodeType() {
return nodeType;
}
public void setNodeType(String nodeType) {
this.nodeType = nodeType;
}
public String getLabel() {
return label;
}
public void setLabel(String label) {
this.label = label;
}
public SimpleTree getParent() {
return parent;
}
public void setParent(SimpleTree parent) {
this.parent = parent;
}
public List<SimpleTree> getChildren() {
return children;
}
public void setChildren(List<SimpleTree> children) {
this.children = children;
}
private List<String> strList = new ArrayList<>();
@Override
public String toString() {
String str = this.nodeType + "@@" + this.label;
if (strList.size() == 0) {
strList.add(str);
for (SimpleTree child : children) {
child.toString();
List<String> strList1 = child.strList;
for (String str1 : strList1) {
strList.add("------" + str1);
}
}
}
str = "";
for (String str1 : strList) {
str += str1 + "\n";
}
return str;
}
}
src/main/java/edu/lu/uni/serval/gumtree/regroup/SimplifyTree.java
-902
View File
@@ -1,902 +0,0 @@
package edu.lu.uni.serval.gumtree.regroup;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.github.gumtreediff.actions.model.Action;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.FixPattern.utils.ASTNodeMap;
import edu.lu.uni.serval.FixPattern.utils.Checker;
import edu.lu.uni.serval.utils.ListSorter;
/**
* Simplify the ITree of source code into a simple tree.
*
* @author kui.liu
*
*/
public class SimplifyTree {
private static final String ABSTRACT_TYPE = "T";
private static final String ABSTRACT_NAME = "N";
private static final String ABSTRACT_METHOD = "m";
private static final String ABSTRACT_VARIABLE = "v";
private Map<String, String> abstractTypeIdentifiers = new HashMap<>();
private Map<String, String> abstractMethodIdentifiers = new HashMap<>();
private Map<String, String> abstractNameIdentifiers = new HashMap<>();
private Map<String, String> abstractVariableIdentifiers = new HashMap<>();
/**
* Convert ITree to a source code simple tree, an abstract identifier simple tree, and a semi-source code simple tree.
*
* @param actionSet
*/
public void abstractTree(HierarchicalActionSet actionSet) {
SimpleTree sourceCodeSimpleTree = null; // source code tree and AST node type tree
SimpleTree abstractIdentifierTree = null; // abstract identifier tree
SimpleTree abstractSimpleTree = null; // semi-source code tree. and AST node type tree
SimpleTree simpleTree = null; // source code tree with canonical variable names.
if (actionSet.getActionString().startsWith("INS")) {
List<Action> allMoveActions = getAllMoveActions(actionSet);
if (allMoveActions != null) {
List<Action> actions = new ArrayList<>();
for (Action action : allMoveActions) {
boolean hasParent = false;
ITree parent = action.getNode().getParent();
for (Action act : allMoveActions) {
if (act == action) continue;
ITree actNode = act.getNode();
if (actNode.equals(parent)) {
hasParent = true;
break;
}
}
if (!hasParent) {
actions.add(action);
}
}
// sourceCodeSimpleTree = sourceCodeTree(actions);
simpleTree = canonicalizeSourceCodeTree(actions, null);
}
} else {
ITree tree = actionSet.getNode();
String astNodeType = actionSet.getAstNodeType();
if (Checker.containsBodyBlock(astNodeType)) {
// delete the body block.
List<ITree> children = tree.getChildren();
List<ITree> newChildren = new ArrayList<>();
for (ITree child : children) {
if (!child.getLabel().endsWith("Body")) {
newChildren.add(child);
}
}
tree.setChildren(newChildren);
}
// sourceCodeSimpleTree = originalSourceCodeTree(tree, null);
// abstractIdentifierTree = abstractIdentifierTree(actionSet, tree, null);
// abstractSimpleTree = semiSourceCodeTree(actionSet, tree, null);
simpleTree = canonicalizeSourceCodeTree(tree, null);
}
actionSet.setAbstractSimpleTree(abstractSimpleTree);
actionSet.setAbstractIdentifierTree(abstractIdentifierTree);
actionSet.setSimpleTree(simpleTree);
actionSet.setOriginalTree(sourceCodeSimpleTree);
}
/**
* Convert ITree to a source code simple tree, an abstract identifier simple tree, and a semi-source code simple tree.
*
* @param actionSet
*/
public void abstractTree(HierarchicalActionSet actionSet, int bugEndPosition) {
SimpleTree sourceCodeSimpleTree = null; // source code tree and AST node type tree
SimpleTree abstractIdentifierTree = null; // abstract identifier tree
SimpleTree abstractSimpleTree = null; // semi-source code tree. and AST node type tree
SimpleTree simpleTree = null; // source code tree with canonical variable names.
if (actionSet.getActionString().startsWith("INS")) {
List<Action> allMoveActions = getAllMoveActions(actionSet);
if (allMoveActions != null) {
List<Action> actions = new ArrayList<>();
for (Action action : allMoveActions) {
boolean hasParent = false;
ITree parent = action.getNode().getParent();
for (Action act : allMoveActions) {
if (act == action) continue;
ITree actNode = act.getNode();
if (actNode.equals(parent)) {
hasParent = true;
break;
}
}
if (!hasParent) {
actions.add(action);
}
}
// sourceCodeSimpleTree = sourceCodeTree(actions);
simpleTree = canonicalizeSourceCodeTree(actions, null);
}
} else {
ITree tree = actionSet.getNode();
// String astNodeType = actionSet.getAstNodeType();
// if (Checker.containsBodyBlock(astNodeType)) {
// // delete the body block.
// List<ITree> children = tree.getChildren();
// List<ITree> newChildren = new ArrayList<>();
// for (ITree child : children) {
// if (!child.getLabel().endsWith("Body")) {
// newChildren.add(child);
// }
// }
// tree.setChildren(newChildren);
// }
// sourceCodeSimpleTree = originalSourceCodeTree(tree, null);
// abstractIdentifierTree = abstractIdentifierTree(actionSet, tree, null);
// abstractSimpleTree = semiSourceCodeTree(actionSet, tree, null);
// if (actionSet.getActionString().startsWith("UPD")) {
// simpleTree = canonicalizeSourceCodeTree(tree, null, bugEndPosition);
// } else {
// simpleTree = canonicalizeSourceCodeTree(tree, null);
// }
simpleTree = canonicalizeSourceCodeTree(tree, null, bugEndPosition);
}
actionSet.setAbstractSimpleTree(abstractSimpleTree);
actionSet.setAbstractIdentifierTree(abstractIdentifierTree);
actionSet.setSimpleTree(simpleTree);
actionSet.setOriginalTree(sourceCodeSimpleTree);
}
private SimpleTree canonicalizeSourceCodeTree(List<Action> actions, SimpleTree parent) {
SimpleTree simpleTree = new SimpleTree();
simpleTree.setLabel("Block");
simpleTree.setNodeType("Block");
simpleTree.setParent(parent);
List<SimpleTree> children = new ArrayList<>();
for (Action action : actions) {
ITree node = action.getNode();
children.add(canonicalizeSourceCodeTree(node, simpleTree));
}
simpleTree.setChildren(children);
return simpleTree;
}
public SimpleTree canonicalizeSourceCodeTree(ITree tree, SimpleTree parent) {
SimpleTree simpleTree = new SimpleTree();
String label = tree.getLabel();
String astNode = ASTNodeMap.map.get(tree.getType());
List<ITree> children = tree.getChildren();
if (children.size() > 0) {
simpleTree.setNodeType(astNode);
if (astNode.endsWith("Type")) {
simpleTree.setLabel(canonicalizeTypeStr(label).replaceAll(" ", ""));
} else {
if ((astNode.equals("SimpleName") || astNode.equals("MethodInvocation"))) {
if (label.startsWith("MethodName:")) {
simpleTree.setNodeType("MethodName");
label = label.substring(11);
int argusIndex = label.indexOf(":[");
if (argusIndex > 0) {
label = label.substring(0, argusIndex);
}
} else if (label.startsWith("ClassName:")) {
simpleTree.setNodeType("ClassName");
label = label.substring(10);
}
simpleTree.setLabel(label);
} else {
simpleTree.setLabel(astNode);
}
List<SimpleTree> subTrees = new ArrayList<>();
for (ITree child : children) {
subTrees.add(canonicalizeSourceCodeTree(child, simpleTree));
}
simpleTree.setChildren(subTrees);
}
} else {
if (astNode.endsWith("Name")) {
// variableName, methodName, QualifiedName
if (label.startsWith("MethodName:")) { // <MethodName, name>
simpleTree.setNodeType("MethodName");
label = label.substring(11);
int argusIndex = label.indexOf(":[");
if (argusIndex > 0) {
label = label.substring(0, argusIndex);
}
simpleTree.setLabel(label);
} else if (label.startsWith("ClassName:")) {
simpleTree.setNodeType("ClassName");
label = label.substring(10);
simpleTree.setLabel(label);
} else if (label.startsWith("Name:")) {
label = label.substring(5);
char firstChar = label.charAt(0);
if (Character.isUpperCase(firstChar)) {
simpleTree.setNodeType("Name");
simpleTree.setLabel(label);
} else {// variableName: <VariableName, canonicalName>
simpleTree.setNodeType("VariableName");
simpleTree.setLabel(canonicalVariableName(label, tree));
}
} else {// variableName: <VariableName, canonicalName>
simpleTree.setNodeType("VariableName");
simpleTree.setLabel(canonicalVariableName(label, tree));
}
} else {
simpleTree.setNodeType(astNode);
if (astNode.endsWith("Type")) {
simpleTree.setLabel(canonicalizeTypeStr(label).replaceAll(" ", ""));
} else if (astNode.startsWith("Type")) {
simpleTree.setLabel(canonicalizeTypeStr(label).replaceAll(" ", ""));
} else if ((astNode.equals("SimpleName") || astNode.equals("MethodInvocation")) && label.startsWith("MethodName:")) {
simpleTree.setNodeType("MethodName");
label = label.substring(11);
int argusIndex = label.indexOf(":[");
if (argusIndex > 0) {
label = label.substring(0, argusIndex);
}
simpleTree.setLabel(label);
} else {
simpleTree.setLabel(label.replaceAll(" ", ""));
}
}
}
simpleTree.setParent(parent);
return simpleTree;
}
public void canonicalizeSourceCodeTree(ITree tree) {
String label = tree.getLabel();
String astNode = ASTNodeMap.map.get(tree.getType());
List<ITree> children = tree.getChildren();
if (children.size() > 0) {
if (astNode.endsWith("Type")) {
} else {
if ((astNode.equals("SimpleName") || astNode.equals("MethodInvocation"))) {
if (label.startsWith("MethodName:")) {
} else if (label.startsWith("ClassName:")) {
}
} else {
}
for (ITree child : children) {
canonicalizeSourceCodeTree(child);
}
}
} else {
if (astNode.endsWith("Name")) {
// variableName, methodName, QualifiedName
if (label.startsWith("MethodName:")) { // <MethodName, name>
} else if (label.startsWith("ClassName:")) {
} else if (label.startsWith("Name:")) {
label = label.substring(5);
char firstChar = label.charAt(0);
if (Character.isUpperCase(firstChar)) {
// simpleTree.setNodeType("Name");
} else {// variableName: <VariableName, canonicalName>
canonicalVariableName(label, tree);
}
} else {// variableName: <VariableName, canonicalName>
canonicalVariableName(label, tree);
}
}
}
}
public SimpleTree canonicalizeSourceCodeTree(ITree tree, SimpleTree parent, int bugEndPosition) {
SimpleTree simpleTree = new SimpleTree();
String label = tree.getLabel();
String astNode = ASTNodeMap.map.get(tree.getType());
List<ITree> children = tree.getChildren();
if (children.size() > 0) {
simpleTree.setNodeType(astNode);
if (astNode.endsWith("Type")) {
simpleTree.setLabel(canonicalizeTypeStr(label).replaceAll(" ", ""));
} else {
if ((astNode.equals("SimpleName") || astNode.equals("MethodInvocation"))) {
if (label.startsWith("MethodName:")) {
simpleTree.setNodeType("MethodName");
label = label.substring(11);
int argusIndex = label.indexOf(":[");
if (argusIndex > 0) {
label = label.substring(0, argusIndex);
}
} else if (label.startsWith("ClassName:")) {
simpleTree.setNodeType("ClassName");
label = label.substring(10);
}
simpleTree.setLabel(label);
} else {
simpleTree.setLabel(astNode);
}
List<SimpleTree> subTrees = new ArrayList<>();
for (ITree child : children) {
if (child.getPos() > bugEndPosition) continue;
subTrees.add(canonicalizeSourceCodeTree(child, simpleTree));
}
simpleTree.setChildren(subTrees);
}
} else {
if (astNode.endsWith("Name")) {
// variableName, methodName, QualifiedName
if (label.startsWith("MethodName:")) { // <MethodName, name>
simpleTree.setNodeType("MethodName");
label = label.substring(11);
int argusIndex = label.indexOf(":[");
if (argusIndex > 0) {
label = label.substring(0, argusIndex);
}
simpleTree.setLabel(label);
} else if (label.startsWith("ClassName:")) {
simpleTree.setNodeType("ClassName");
label = label.substring(10);
simpleTree.setLabel(label);
} else if (label.startsWith("Name:")) {
label = label.substring(5);
char firstChar = label.charAt(0);
if (Character.isUpperCase(firstChar)) {
simpleTree.setNodeType("Name");
simpleTree.setLabel(label);
} else {// variableName: <VariableName, canonicalName>
simpleTree.setNodeType("VariableName");
simpleTree.setLabel(canonicalVariableName(label, tree));
}
} else {// variableName: <VariableName, canonicalName>
simpleTree.setNodeType("VariableName");
simpleTree.setLabel(canonicalVariableName(label, tree));
}
} else {
simpleTree.setNodeType(astNode);
if (astNode.endsWith("Type")) {
simpleTree.setLabel(canonicalizeTypeStr(label).replaceAll(" ", ""));
} else if (astNode.startsWith("Type")) {
simpleTree.setLabel(canonicalizeTypeStr(label).replaceAll(" ", ""));
} else if ((astNode.equals("SimpleName") || astNode.equals("MethodInvocation")) && label.startsWith("MethodName:")) {
simpleTree.setNodeType("MethodName");
label = label.substring(11);
int argusIndex = label.indexOf(":[");
if (argusIndex > 0) {
label = label.substring(0, argusIndex);
}
simpleTree.setLabel(label);
} else {
simpleTree.setLabel(label.replaceAll(" ", ""));
}
}
}
simpleTree.setParent(parent);
return simpleTree;
}
public String canonicalVariableName(String label, ITree tree) {
ITree parent = tree.getParent();
if (parent == null) {
return label;
} else {
String matchStr = null;
int parentType = parent.getType();
if (parentType == 44) { // SingleVariableDeclaration
matchStr = matchSingleVariableDeclaration(parent, label);
} else if (parentType == 23 || parentType == 58 || parentType == 60) {
//FieldDeclaration, VariableDeclarationExpression, VariableDeclarationStatement
matchStr = matchVariableDeclarationExpression(parent, label);
} else if (parentType == 31) { // MethodDeclaration
List<ITree> children = parent.getChildren();
int index = children.indexOf(tree);
for (int i = index - 1; i >=0; i --) {
ITree child = children.get(i);
int childType = child.getType();
if (childType == 60) { // VariableDeclarationStatement
matchStr = matchVariableDeclarationExpression(child, label);
} else if (childType == 44) { // SingleVariableDeclaration
matchStr = matchSingleVariableDeclaration(child, label);
}
if (matchStr != null) break;
}
} else if (parentType ==70 || parentType == 24 ||parentType == 12 || parentType == 54) {
// EnhancedForStatement, ForStatement, CatchClause, TryStatement
matchStr = matchStatements(parentType, parent, label);
} else if (parentType == 55) { // TypeDeclaration: Class Declaration
List<ITree> children = parent.getChildren();
int index = children.indexOf(tree);
for (int i = index - 1; i >=0; i --) {
ITree child = children.get(i);
if (child.getType() == 23) { // FieldDeclaration
matchStr = matchVariableDeclarationExpression(child, label);
if (matchStr != null) break;
}
}
} else if (parentType == 8) { // Block body
List<ITree> children = parent.getChildren();
int index = children.indexOf(tree);
for (int i = index - 1; i >=0; i --) {
ITree child = children.get(i);
if (child.getType() == 60) { // VariableDeclarationStatement
matchStr = matchVariableDeclarationExpression(child, label);
if (matchStr != null) break;
}
}
}
if (matchStr != null) {
return matchStr;
} else {
return canonicalVariableName(label, parent);
}
}
}
private String matchStatements(int typeInt, ITree tree, String label) {
String matchStr = null;
if (typeInt == 70) { // EnhancedForStatement
matchStr = matchSingleVariableDeclaration(tree.getChild(0), label);
} else if (typeInt == 24) { // ForStatement
List<ITree> children = tree.getChildren();
for (ITree child : children) {
if (child.getType() == 58) {
matchStr = matchVariableDeclarationExpression(child, label);
if (matchStr != null) break;
} else {
break;
}
}
} else if (typeInt == 12) { // CatchClause
matchStr = matchSingleVariableDeclaration(tree.getChild(0), label);
} else if (typeInt == 54) { // TryStatement
List<ITree> children = tree.getChildren();
for (ITree child : children) {
if (child.getType() == 58) { //VariableDeclarationExpression
matchStr = matchVariableDeclarationExpression(tree, label);
if (matchStr != null) break;
} else {
break;
}
}
}
return null;
}
public Map<String, String> canonicalVariableMap = new HashMap<>();
public Map<String, Integer> canonicalVariables = new HashMap<>();
private String matchVariableDeclarationExpression(ITree variable, String label) {
List<ITree> children = variable.getChildren();
ITree type = null;
for (int i = 0, size = children.size(); i < size; i ++) {
ITree child = children.get(i);
if (child.getType() == 59) {// VariableDeclarationFragment
if (type == null) {
type = children.get(i - 1);
}
ITree simpleName = child.getChild(0);
if (simpleName.getLabel().equals(label)) {
String typeStr = canonicalizeTypeStr(type.getLabel());
return canonicalizeVariable(label, typeStr);
}
}
}
return null;
}
private String matchSingleVariableDeclaration(ITree singleVariable, String label) {
List<ITree> children = singleVariable.getChildren();
for (int i = 0, size = children.size(); i < size; i ++) {
ITree child = children.get(i);
if (child.getType() == 42) { // SimpleName
if (child.getLabel().equals(label)) {
ITree type = children.get(i - 1);
String typeStr = canonicalizeTypeStr(type.getLabel());
return canonicalizeVariable(label, typeStr);
}
break;
}
}
return null;
}
private String canonicalizeVariable(String label, String typeStr) {
String key = label + ":" + typeStr;
if (canonicalVariableMap.containsKey(key)) {
label = canonicalVariableMap.get(key);
} else {
label = typeStr.toLowerCase() + "Var";
Integer num = canonicalVariables.get(label);
if (num == null) {
num = 0;
}
num ++;
canonicalVariables.put(label, num);
label += "" + num;
canonicalVariableMap.put(key, label);
}
return label;
}
private String canonicalizeTypeStr(String label) {
String typeStr = label;
int index1 = typeStr.indexOf("<");
if (index1 != -1) {
typeStr = typeStr.substring(0, index1);
}
index1 = typeStr.lastIndexOf(".");
if (index1 != -1) {
typeStr = typeStr.substring(index1 + 1);
}
return typeStr;
}
// public static String addPrefixByType(Type type) {
// String newName = "";
// if (type instanceof PrimitiveType) {
// // byte,short,char,int,long,float,double,boolean,void
// newName = type.toString().toLowerCase();
// } else if (type instanceof ArrayType) {
// // Type [ ]
// ArrayType at = (ArrayType) type;
// type = at.getElementType();
// if (type instanceof SimpleType || type instanceof PrimitiveType) {
// newName = getNewName(type);
// } else {
// newName = addPrefixByType(type);
// }
// } else if (type instanceof SimpleType) {
// // TypeName
// if (type.toString().equals("Integer")) {
// newName = "int";
// } else {
// newName = getNewName(type);
// }
// } else if (type instanceof QualifiedType) {
// // Type.SimpleName
// newName = ((QualifiedType) type).getName().toString().toLowerCase();
// } else if (type instanceof ParameterizedType) {
// // Type < Type { , Type } > 泛型
// ParameterizedType t = (ParameterizedType) type;
// newName = getNewName(t.getType());
// } else if (type instanceof WildcardType) {
// newName = "object";
// }
// return newName;
// }
//
// private static String getNewName(Type type) {
// String newName = "";
// String typeName = type.toString();
// int dot = typeName.lastIndexOf(".");
// if (dot > 0) {
// newName = typeName.substring(dot + 1).toString().toLowerCase();
// } else {
// newName = typeName.toString().toLowerCase();
// }
// return newName;
// }
/**
* Convert the Move actions of an INS action into a simple tree with AST nodes and leaf labels.
*
* @param actions
* @return
*/
private SimpleTree sourceCodeTree(List<Action> actions) {
if (actions.size() > 0) {
SimpleTree simpleTree = new SimpleTree();
simpleTree.setNodeType("Block");
simpleTree.setLabel("Block");
simpleTree.setParent(null);
List<SimpleTree> subTrees = new ArrayList<>();
for (Action action : actions) {
ITree node = action.getNode();
subTrees.add(sourceCodeTree(node, simpleTree));
}
simpleTree.setChildren(subTrees);
return simpleTree;
}
return null;
}
/**
* Convert a Move action into a simple tree with AST nodes and leaf labels.
*
* @param tree
* @param parent
* @return
*/
private SimpleTree sourceCodeTree(ITree tree, SimpleTree parent) {
SimpleTree simpleTree = new SimpleTree();
String astNode = ASTNodeMap.map.get(tree.getType());
do {
if (astNode.endsWith("Statement") || astNode.equals("FieldDeclaration")) break;
tree = tree.getParent();
astNode = ASTNodeMap.map.get(tree.getType());// FIXME if the ASTNode is a method declaration or class declaration?
} while (!astNode.endsWith("Statement") && !astNode.equals("FieldDeclaration"));
String label = tree.getLabel();
List<ITree> children = tree.getChildren();
if (children.size() > 0) {
List<SimpleTree> subTrees = new ArrayList<>();
for (ITree child : children) {
subTrees.add(sourceCodeTree(child, simpleTree));
}
simpleTree.setChildren(subTrees);
simpleTree.setLabel(astNode);
} else {
simpleTree.setLabel(label);
}
simpleTree.setNodeType(astNode);
simpleTree.setParent(parent);
return simpleTree;
}
/**
* Convert an UPD/DEL/MOV action into a simple tree with AST nodes and leaf labels.
*
* @param tree
* @param parent
* @return
*/
private SimpleTree originalSourceCodeTree(ITree tree, SimpleTree parent) {
SimpleTree simpleTree = new SimpleTree();
String label = tree.getLabel();
String astNode = ASTNodeMap.map.get(tree.getType());
simpleTree.setNodeType(astNode);
List<ITree> children = tree.getChildren();
if (children.size() > 0) {
List<SimpleTree> subTrees = new ArrayList<>();
for (ITree child : children) {
subTrees.add(originalSourceCodeTree(child, simpleTree));
}
simpleTree.setChildren(subTrees);
simpleTree.setLabel(astNode);
} else {
simpleTree.setLabel(label);
}
simpleTree.setParent(parent);
return simpleTree;
}
/**
* Convert an UPD/DEL/MOV action into a simple tree with abstract identifiers of AST nodes and abstract identifiers of leaf labels.
*
* @param actionSet
* @param tree
* @param parent
* @return
*/
private SimpleTree abstractIdentifierTree(ITree tree, SimpleTree parent) {
SimpleTree simpleTree = new SimpleTree();
String label = tree.getLabel();
String astNode = ASTNodeMap.map.get(tree.getType());
simpleTree.setNodeType(astNode);
List<ITree> children = tree.getChildren();
if (children.size() > 0) {
if (astNode.endsWith("Type")) {
simpleTree.setNodeType("Type");
simpleTree.setLabel(getAbstractLabel(abstractTypeIdentifiers, label, ABSTRACT_TYPE)); // abstract Type identifier
} else {
List<SimpleTree> subTrees = new ArrayList<>();
for (ITree child : children) {
subTrees.add(abstractIdentifierTree(child, simpleTree));
}
simpleTree.setChildren(subTrees);
simpleTree.setLabel(astNode);
}
} else {
if (astNode.endsWith("Type")) {
simpleTree.setNodeType("Type");
if (astNode.equals("WildcardType")) {
simpleTree.setLabel("?");
} else {
simpleTree.setLabel(getAbstractLabel(abstractTypeIdentifiers, label, ABSTRACT_TYPE)); // abstract Type identifier
}
} else if (astNode.endsWith("Name")) {
// variableName, methodName, QualifiedName
if (label.startsWith("MethodName:")) { // <Method, name>
label = label.substring(11);
simpleTree.setNodeType("Method");
simpleTree.setLabel(getAbstractLabel(abstractMethodIdentifiers, label, ABSTRACT_METHOD)); // abstract method identifier
} else if (label.startsWith("Name:")) {
label = label.substring(5);
String firstChar = label.substring(0, 1);
if (firstChar.equals(firstChar.toUpperCase())) {
simpleTree.setNodeType("Name");
simpleTree.setLabel(getAbstractLabel(abstractNameIdentifiers, label, ABSTRACT_NAME)); // abstract Name identifier
} else {// variableName: <Variable, var>
simpleTree.setNodeType("Variable");
simpleTree.setLabel(getAbstractLabel(abstractVariableIdentifiers, label, ABSTRACT_VARIABLE));// abstract Variable identifier
}
} else {// variableName: <Variable, var>
simpleTree.setNodeType("Variable");
simpleTree.setLabel(getAbstractLabel(abstractVariableIdentifiers, label, ABSTRACT_VARIABLE));// abstract Variable identifier
}
} else if (astNode.equals("BooleanLiteral") || astNode.equals("CharacterLiteral") || astNode.equals("NullLiteral")
|| astNode.equals("NumberLiteral") || astNode.equals("StringLiteral") || astNode.equals("ThisExpression")
|| astNode.equals("Modifier") || astNode.equals("Operator")) {
simpleTree.setNodeType(astNode);
simpleTree.setLabel(label);
}
}
simpleTree.setParent(parent);
return simpleTree;
}
/**
* Convert an UPD/DEL/MOV action into a semi-source code simple tree by abstracting the non-buggy code.
*
* @param actionSet
* @param tree
* @param parent
* @return
*/
private SimpleTree semiSourceCodeTree(HierarchicalActionSet actionSet, ITree tree, SimpleTree parent) {
SimpleTree simpleTree = new SimpleTree();
simpleTree.setParent(parent);
// deep first
abstractBuggyTreeDeepFirst(actionSet, tree, simpleTree);
return simpleTree;
}
private void abstractBuggyTreeDeepFirst(HierarchicalActionSet actionSet, ITree tree, SimpleTree simpleTree) {
List<ITree> children = tree.getChildren();
HierarchicalActionSet modifyAction = findHierarchicalActionSet(tree.getPos(), tree.getLength(), actionSet);
String label = tree.getLabel();
String astNode = ASTNodeMap.map.get(tree.getType());
if (Checker.isExpressionType(astNode)) {
if (modifyAction == null || !modifyAction.getActionString().contains("@@" + label)) {
simpleTree.setNodeType("Expression");
simpleTree.setLabel("EXP"); // astNode
}
} else {
if (astNode.endsWith("Type")) { // <Type, ?> TODO: sub Type
simpleTree.setNodeType("Type");
// simpleTree.setLabel("?");
if (astNode.equals("WildcardType")) {
simpleTree.setLabel("?");
} else { // ArrayType, PrimitiveType, SimpleType, ParameterizedType, QualifiedType, WildcardType, UnionType,NameQualifiedType, IntersectionType
simpleTree.setLabel(astNode + "@@" + label);
}
} else if (astNode.endsWith("Name")) { // variableName, methodName, QualifiedName
if (label.startsWith("MethodName:")) { // <Method, name>
label = label.substring(11);
simpleTree.setNodeType("Method");
simpleTree.setLabel(label);
} else if (label.startsWith("Name:")) {
label = label.substring(5);
String firstChar = label.substring(0, 1);
if (firstChar.equals(firstChar.toUpperCase())) {
simpleTree.setNodeType("Name");
simpleTree.setLabel(label); // <Name, name>
} else {// variableName: <Variable, var>
simpleTree.setNodeType("Variable");
simpleTree.setLabel(getAbstractLabel(abstractVariableIdentifiers, label, ABSTRACT_VARIABLE));
}
} else {// variableName: <Variable, var>
simpleTree.setNodeType("Variable");
simpleTree.setLabel(getAbstractLabel(abstractVariableIdentifiers, label, ABSTRACT_VARIABLE));
}
} else if (astNode.equals("BooleanLiteral") ||astNode.equals("CharacterLiteral") || astNode.equals("ThisExpression")
|| astNode.equals("NullLiteral") || astNode.equals("NumberLiteral") || astNode.equals("StringLiteral")
|| astNode.equals("Modifier") || astNode.equals("Operator")) {
simpleTree.setNodeType(astNode);
simpleTree.setLabel(label);
} else {
simpleTree.setNodeType(astNode);
simpleTree.setLabel(astNode);
}
}
List<SimpleTree> simpleChildren = new ArrayList<>();
if (children != null && !astNode.endsWith("Type")) {
for (ITree child : children) {
simpleChildren.add(semiSourceCodeTree(actionSet, child, simpleTree));
}
}
simpleTree.setChildren(simpleChildren);
}
private List<Action> getAllMoveActions(HierarchicalActionSet actionSet) {
String astNodeType = actionSet.getAstNodeType();
if (Checker.containsBodyBlock(astNodeType)) {
List<Action> allMoveActions = getAllMoveActions2(actionSet);
if (allMoveActions != null && allMoveActions.size() > 0) {
ListSorter<Action> sorter = new ListSorter<Action>(allMoveActions);
List<Action> moveActions = sorter.sortAscending();
if (moveActions != null) {
allMoveActions = moveActions;
}
return allMoveActions;
} else {// FIXME: pure INS actions.
return null;
}
} else {// FIXME: pure INS actions.
return null;
}
/**
* Variables, non-new and used in the inserted statements, could be selected to localize buggy code
*/
}
private List<Action> getAllMoveActions2(HierarchicalActionSet actionSet) {
List<Action> allMoveActions = new ArrayList<>();
List<HierarchicalActionSet> actions = new ArrayList<>();
actions.addAll(actionSet.getSubActions());
if (actions.size() == 0) {
return null;
}
while (actions.size() > 0) {
List<HierarchicalActionSet> subActions = new ArrayList<>();
for (HierarchicalActionSet action : actions) {
subActions.addAll(action.getSubActions());
if (action.toString().startsWith("MOV")) {
allMoveActions.add(action.getAction());
}
}
actions.clear();
actions.addAll(subActions);
}
return allMoveActions;
}
private String getAbstractLabel(Map<String, String> map, String label, String nameType) {
if (map.containsKey(label)) {
return map.get(label);
} else {
String name = nameType + map.size();
map.put(label, name);
return name;
}
}
private HierarchicalActionSet findHierarchicalActionSet(int position, int length, HierarchicalActionSet actionSet) {
if (actionSet.getStartPosition() == position && actionSet.getLength() == length && !actionSet.getActionString().startsWith("INS")) {
return actionSet;
} else {
for (HierarchicalActionSet subActionSet : actionSet.getSubActions()) {
HierarchicalActionSet actSet = findHierarchicalActionSet(position, length, subActionSet);
if (actSet != null) {
return actSet;
}
}
}
return null;
}
public Map<String, String> getAbstractTypeIdentifiers() {
return abstractTypeIdentifiers;
}
public Map<String, String> getAbstractMethodIdentifiers() {
return abstractMethodIdentifiers;
}
public Map<String, String> getAbstractNameIdentifiers() {
return abstractNameIdentifiers;
}
public Map<String, String> getAbstractVariableIdentifiers() {
return abstractVariableIdentifiers;
}
}
src/main/java/edu/lu/uni/serval/gumtree/regroup/Traveler.java
-140
View File
@@ -1,140 +0,0 @@
package edu.lu.uni.serval.gumtree.regroup;
import java.util.ArrayList;
import java.util.List;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.FixPattern.utils.ASTNodeMap;
/**
* A traveler to travel a tree-constructed object.
*
* @author kui.liu
*
*/
public class Traveler {
public List<List<String>> list = new ArrayList<>();
/**
* Get all action string by traveling HierarchicalActionSet in a deep-first way.
*
* @param actionSet
* @param astNodeTypeActionQueue
*/
public void travelActionSetDeepFirstToASTNodeQueue(HierarchicalActionSet actionSet, List<String> astNodeTypeActionQueue) {
if (actionSet == null) {
System.err.println("Null Action set!");
} else {
if (astNodeTypeActionQueue == null) {
astNodeTypeActionQueue = new ArrayList<>();
}
String actionStr = actionSet.getActionString();
actionStr = actionStr.substring(0, actionStr.indexOf("@@"));
astNodeTypeActionQueue.add(actionStr); // RawToken: TODO
if (actionStr.startsWith("DEL")) {
list.add(astNodeTypeActionQueue); // FIXME BUG: Change AST node type 1 to AST node type 2. Solve method: a list is one pattern.
} else {
List<HierarchicalActionSet> subActionSet = actionSet.getSubActions();
int size = subActionSet.size();
if (size > 0) {
for (HierarchicalActionSet subAction : subActionSet) {
List<String> astNodeTypeActionQueue_ = new ArrayList<>();
astNodeTypeActionQueue_.addAll(astNodeTypeActionQueue);
travelActionSetDeepFirstToASTNodeQueue(subAction, astNodeTypeActionQueue_);
}
} else {
list.add(astNodeTypeActionQueue);
}
}
}
}
/**
* Get all AST node types of a root tree by traveling the root tree in a deep-first way.
*
* @param root
* @return
*/
public static List<String> travelTreeDeepFirstToASTNodeQueue(ITree root) {
if (root == null) {
System.err.println("Null tree!");
return null;
}
List<String> astNodeTypeQueue = new ArrayList<>();
astNodeTypeQueue.add(ASTNodeMap.map.get(root.getType())); // RawToken: root.getLabel();
List<ITree> childrenTreeList = root.getChildren();
if (childrenTreeList != null && childrenTreeList.size() > 0) {
for (ITree childTree : childrenTreeList) {
astNodeTypeQueue.addAll(travelTreeDeepFirstToASTNodeQueue(childTree));
}
}
return astNodeTypeQueue;
}
/**
* Get all AST node types of a root tree by traveling the root tree in a breadth-first way.
*
* @param root
* @return
*/
public static List<String> travelTreeBreadthFirstToASTNodeQueue(ITree root) {
if (root == null) {
System.err.println("Null tree.");
return null;
}
List<String> astNodeTypeQueue = new ArrayList<>();
astNodeTypeQueue.add(ASTNodeMap.map.get(root.getType())); // RawToken: root.getLabel();
List<ITree> treeList = new ArrayList<>();
treeList.add(root);
while (!treeList.isEmpty()) {
List<ITree> childrenTreeList = new ArrayList<>();
for (ITree tree : treeList) {
astNodeTypeQueue.addAll(travelTreeBreadthFirstToASTNodeQueue(tree));
childrenTreeList.addAll(tree.getChildren());
}
treeList.clear();
treeList.addAll(childrenTreeList);
}
return astNodeTypeQueue;
}
/**
* Convert a root ITree into a SimpleTree by traveling the root tree in a deep-first way.
*
* SimpleTree node label is root.toShortString().
*
* @param root
* @param parent
* @return
*/
public static SimpleTree travelITreeDeepFirstToSimpleTree(ITree root, SimpleTree parent) {
if (root == null) {
System.err.println("Null tree!");
return null;
}
SimpleTree simpleTree = new SimpleTree();
simpleTree.setLabel(root.toShortString());
simpleTree.setParent(parent);
List<SimpleTree> children = new ArrayList<>();
List<ITree> childrenTreeList = root.getChildren();
if (childrenTreeList != null && childrenTreeList.size() > 0) {
for (ITree childTree : childrenTreeList) {
children.add(travelITreeDeepFirstToSimpleTree(childTree, simpleTree));
}
}
simpleTree.setChildren(children);
return simpleTree;
}
}
src/main/java/edu/lu/uni/serval/livestudy/Alarm.java
-33
View File
@@ -1,33 +0,0 @@
package edu.lu.uni.serval.livestudy;
public class Alarm {
private String alarmType;
private String fileName;
private int startLine;
private int endLine;
public String getAlarmType() {
return alarmType;
}
public String getFileName() {
return fileName;
}
public int getStartLine() {
return startLine;
}
public int getEndLine() {
return endLine;
}
public Alarm(String alarmType, String fileName, int startLine, int endLine) {
super();
this.alarmType = alarmType;
this.fileName = fileName;
this.startLine = startLine;
this.endLine = endLine;
}
}
src/main/java/edu/lu/uni/serval/livestudy/ParseViolations.java
-212
View File
@@ -1,212 +0,0 @@
package edu.lu.uni.serval.livestudy;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Scanner;
import edu.lu.uni.serval.FixPatternParser.Tokenizer;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.gumtree.regroup.SimpleTree;
import edu.lu.uni.serval.utils.FileHelper;
public class ParseViolations {
private static final String OUTPUT_PATH = Configuration.ROOT_PATH + "LiveStudy/BugsInfo/";
private static int i = 0;
private static int j = 0;
public static void main(String[] args) {
// output path
// FileHelper.deleteDirectory(OUTPUT_PATH);
List<String> projects = new ArrayList<>();
File[] files = new File(OUTPUT_PATH).listFiles();
for (File file : files) {
if (file.isDirectory()) {
projects.add(file.getName());
}
}
String projectsPath = Configuration.ROOT_PATH + "LiveStudy/projects/";
String bugsListPath = Configuration.ROOT_PATH + "LiveStudy/BugsList/";
List<File> bugsListFiles = FileHelper.getAllFiles(bugsListPath, ".list");
for (File bugsListFile : bugsListFiles) {
String fileName = FileHelper.getFileNameWithoutExtension(bugsListFile);
if (projects.contains(fileName)) continue;
ParseViolations parser = new ParseViolations();
Map<String, Violation> violations = new HashMap<>(); // <ProjectName, Violations>, Violation: <projectName, List<Alarm>>.
violations = parser.readViolations(bugsListFile);
parser.parseViolationToTokens(violations, projectsPath);
}
System.out.println(i);
System.out.println(j);
}
/**
* Get the source code tokens of all violation instances by visiting each java project.
*
* @param violations
* @param projectsPath
*/
public void parseViolationToTokens(Map<String, Violation> violations, String projectsPath) {
for (Map.Entry<String, Violation> entry : violations.entrySet()) {
String projectName = entry.getKey();
List<Alarm> alarms = entry.getValue().getAlarms();
List<File> javaFiles = FileHelper.getAllFiles(projectsPath + projectName, ".java");
//Each violation: commons-math : DLS_DEAD_LOCAL_STORE : org/apache/commons/math4/dfp/Dfp.java : 2049 : 2049
/**
* @ProjectName
* @ViolationType
* @FileName
* @LineNumber
* @SourceCode
* @Tokens
* @NumberOfTokens
*/
// sizes file, tokens file, and bugs-info file
StringBuilder sizesBuilder = new StringBuilder();
StringBuilder tokensBuilder = new StringBuilder();
StringBuilder bugsInfoBuilder = new StringBuilder();
for (Alarm alarm : alarms) {
String fileName = alarm.getFileName();
// if (fileName.toLowerCase(Locale.ENGLISH).contains("test")) continue;
File sourceCodeFile = locateSourceCodeFile(javaFiles, fileName);
if (sourceCodeFile == null) {
j ++;
continue;
}
Parser parser = new Parser(alarm, sourceCodeFile);
parser.parse();
SimpleTree simpleTree = parser.getSimpleTree();
if (simpleTree != null) {
int finalStartLine = parser.getFinalStartLine();
int finalEndLine = parser.getFinalEndLine();
String violationType = alarm.getAlarmType();
String tokens = Tokenizer.getTokensDeepFirst(simpleTree);
String[] tokensArray = tokens.split(" ");
int length = tokensArray.length;
StringBuilder sourceCode = readSourceCode(sourceCodeFile, finalStartLine, finalEndLine);
sizesBuilder.append(length).append("\n");
tokensBuilder.append(tokens).append("\n");
bugsInfoBuilder.append("###BugInstance###\n##Info:");
bugsInfoBuilder.append(violationType).append(":");
bugsInfoBuilder.append(projectName).append(":");
bugsInfoBuilder.append(fileName).append(":");
bugsInfoBuilder.append(finalStartLine).append(":");
bugsInfoBuilder.append(finalEndLine).append("\n");
bugsInfoBuilder.append(sourceCode).append("\n");
i ++;
} else {
j ++;
}
}
FileHelper.outputToFile(OUTPUT_PATH + "sizes.list", sizesBuilder, true);
FileHelper.outputToFile(OUTPUT_PATH + "tokens.list", tokensBuilder, true);
FileHelper.outputToFile(OUTPUT_PATH + "bugsInfo.list", bugsInfoBuilder, true);
}
}
private StringBuilder readSourceCode(File javaFile, int startLine, int endLine) {
StringBuilder sourceCode = new StringBuilder("##Source_Code:\n");
FileInputStream fis = null;
Scanner scanner = null;
try {
fis = new FileInputStream(javaFile);
scanner = new Scanner(fis);
int counter = 0;
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
counter ++;
if (startLine <= counter && counter <= endLine) {
sourceCode.append(line + "\n");
}
if (counter == endLine) break;
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} finally {
try {
scanner.close();
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sourceCode;
}
private File locateSourceCodeFile(List<File> javaFiles, String fileName) {
for (File javaFile : javaFiles) {
if (javaFile.getPath().endsWith(fileName)) return javaFile;
}
return null;
}
public Map<String, Violation> readViolations(String violationFile) {
return readViolations(new File(violationFile));
}
public Map<String, Violation> readViolations(File violationFile) {
Map<String, Violation> violations = new HashMap<>();
FileInputStream fis = null;
Scanner scanner = null;
try {
fis = new FileInputStream(violationFile);
scanner = new Scanner(fis);
while (scanner.hasNextLine()) {
//commons-math : DLS_DEAD_LOCAL_STORE : org/apache/commons/math4/dfp/Dfp.java : 2049 : 2049
String line = scanner.nextLine();
String[] elements = line.split(" : ");
String projectName = elements[0];
String alarmType = elements[1];
String fileName = elements[2];
int startLine = Integer.parseInt(elements[3]);
int endLine = Integer.parseInt(elements[4]);
Alarm alarm = new Alarm(alarmType, fileName, startLine, endLine);
addAlarmToViolations(projectName, alarm, violations);
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} finally {
try {
scanner.close();
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return violations;
}
private void addAlarmToViolations(String projectName, Alarm alarm, Map<String, Violation> violations) {
Violation violation = null;
if (violations.containsKey(projectName)) {
violation = violations.get(projectName);
} else {
violation = new Violation(projectName);
violations.put(projectName, violation);
}
List<Alarm> alarms = violation.getAlarms();
alarms.add(alarm);
}
}
src/main/java/edu/lu/uni/serval/livestudy/Parser.java
-84
View File
@@ -1,84 +0,0 @@
package edu.lu.uni.serval.livestudy;
import java.io.File;
import java.util.ArrayList;
import java.util.List;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.gumtree.regroup.SimpleTree;
import edu.lu.uni.serval.gumtree.regroup.SimplifyTree;
import edu.lu.uni.serval.violation.code.parser.ViolationSourceCodeTree;
public class Parser {
// input
private Alarm alarm;
private File sourceCodeFile;
// output
private int finalStartLine = 0;
private int finalEndLine = 0;
private SimpleTree simpleTree = null;
public Parser(Alarm alarm, File sourceCodeFile) {
this.alarm = alarm;
this.sourceCodeFile = sourceCodeFile;
}
public int getFinalStartLine() {
return finalStartLine;
}
public int getFinalEndLine() {
return finalEndLine;
}
public SimpleTree getSimpleTree() {
return simpleTree;
}
public void parse() {
int startLine = this.alarm.getStartLine();
int endLine = this.alarm.getEndLine();
String violationType = this.alarm.getAlarmType();
ViolationSourceCodeTree treeParser = new ViolationSourceCodeTree(this.sourceCodeFile, startLine, endLine);
if ("EQ_DOESNT_OVERRIDE_EQUALS".equals(violationType)|| "HE_EQUALS_USE_HASHCODE".equals(violationType)
|| "HE_INHERITS_EQUALS_USE_HASHCODE".equals(violationType)|| "SE_NO_SUITABLE_CONSTRUCTOR".equals(violationType)
|| "RI_REDUNDANT_INTERFACES".equals(violationType) || "CN_IDIOM".equals(violationType)
|| "SE_NO_SERIALVERSIONID".equals(violationType) || "SE_COMPARATOR_SHOULD_BE_SERIALIZABLE".equals(violationType)) {
// Class name level, tokens
// classStartP <= vS <= vE <= classEndP
ITree classNameTree = treeParser.getClassNameTokens();
if (classNameTree != null) {
this.simpleTree = new SimplifyTree().canonicalizeSourceCodeTree(classNameTree, null);
} else {
System.err.println("#Null_Violation_Hunk: " + this.alarm.getAlarmType() + ":" + this.sourceCodeFile.getPath() + ":" + startLine + ":" + endLine);
}
} else {
treeParser.extract();
List<ITree> matchedTrees = treeParser.getViolationSourceCodeTrees();
if (matchedTrees.size() == 0) {
System.err.println("#Null_Violation_Hunk: " + this.alarm.getAlarmType() + ":" + this.sourceCodeFile.getPath() + ":" + startLine + ":" + endLine);
} else {
this.simpleTree = new SimpleTree();
this.simpleTree.setLabel("Block");
this.simpleTree.setNodeType("Block");
List<SimpleTree> children = new ArrayList<>();
for (ITree matchedTree : matchedTrees) {
SimpleTree simpleT = new SimplifyTree().canonicalizeSourceCodeTree(matchedTree, this.simpleTree);
children.add(simpleT);
}
this.simpleTree.setChildren(children);
}
}
if (this.simpleTree != null) {
this.finalStartLine = treeParser.getViolationFinalStartLine();
this.finalEndLine = treeParser.getViolationFinalEndLine();
}
}
}
src/main/java/edu/lu/uni/serval/livestudy/README.md
-6
View File
@@ -1,6 +0,0 @@
## Parse the violations of Live Study ##
1. Localize bugs with FindBugs.
2. Parse results of FindBugs. (bug-type project.)
3. Parse source code of violations. (ParseViolation.java)
src/main/java/edu/lu/uni/serval/livestudy/Violation.java
-25
View File
@@ -1,25 +0,0 @@
package edu.lu.uni.serval.livestudy;
import java.util.ArrayList;
import java.util.List;
public class Violation {
private String project;
private List<Alarm> alarms;
public String getProject() {
return project;
}
public List<Alarm> getAlarms() {
return alarms;
}
public Violation(String project) {
super();
this.project = project;
this.alarms = new ArrayList<>();
}
}
src/main/java/edu/lu/uni/serval/parameters/ParseAlarms.java
-145
View File
@@ -1,145 +0,0 @@
package edu.lu.uni.serval.parameters;
import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.StringReader;
import java.util.ArrayList;
import java.util.List;
import com.github.gumtreediff.tree.ITree;
import edu.lu.uni.serval.FixPatternParser.Tokenizer;
import edu.lu.uni.serval.FixPatternParser.violations.Violation;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.gumtree.regroup.SimpleTree;
import edu.lu.uni.serval.gumtree.regroup.SimplifyTree;
import edu.lu.uni.serval.utils.FileHelper;
import edu.lu.uni.serval.violation.code.parser.ViolationSourceCodeTree;
/**
* Prepare data for tuning parameters of deep learning models.
*
* @author kui.liu
*
*/
public class ParseAlarms {
public static void main(String[] args) {
ParseAlarms parser = new ParseAlarms();
String outputPath = Configuration.ROOT_PATH + "TuneParameters/fixedAlarmTokens.list";
String outputPath2 = Configuration.ROOT_PATH + "TuneParameters/EmptyStatement/fixedAlarmTokens.list";
FileHelper.deleteFile(outputPath);
FileHelper.deleteFile(outputPath2);
int subIndex = 5;
String fixedAlarmFilesPath = Configuration.GUM_TREE_INPUT + "prevFiles/";
String positionsFilePath = Configuration.GUM_TREE_INPUT + "positions/";
parser.dataPreparation(fixedAlarmFilesPath, positionsFilePath, subIndex, outputPath, outputPath2);
outputPath = Configuration.ROOT_PATH + "TuneParameters/unfixedAlarmTokens.list";
outputPath2 = Configuration.ROOT_PATH + "TuneParameters/EmptyStatement/unfixedAlarmTokens.list";
FileHelper.deleteFile(outputPath);
FileHelper.deleteFile(outputPath2);
subIndex = 8;
String unfixedAlarmFilesPath = Configuration.GUM_TREE_INPUT + "unfixAlarms/";
String unfixedPositionsFilePath = Configuration.GUM_TREE_INPUT + "un_positions/";
parser.dataPreparation(unfixedAlarmFilesPath, unfixedPositionsFilePath, subIndex, outputPath, outputPath2);
}
public void dataPreparation(String sourceCodeFilePath, String positionFilePath, int subIndex, String outputPath, String outputPath2) {
StringBuilder tokensBuilder = new StringBuilder();
List<File> javaFiles = FileHelper.getAllFilesInCurrentDiectory(sourceCodeFilePath, ".java");
int counter = 0;
int a = 0;
StringBuilder emptyStatements = new StringBuilder();
StringBuilder sizes = new StringBuilder();
for (File javaFile : javaFiles) {
String fileName = javaFile.getName().replace(".java", ".txt");
fileName = fileName.substring(subIndex);
// if (fileName.endsWith("apache-commons-configuration_8c42aa_8b26e6src#java#org#apache#commons#configuration#plist#XMLPropertyListConfiguration.txt")) {
// System.out.println();
// }
List<Violation> violations = readViolations(positionFilePath + fileName);
for (Violation violation : violations) {
int startLine = violation.getStartLineNum();
int endLine = violation.getEndLineNum();
String alarmType = violation.getViolationType();
if (endLine > startLine + 5) continue;
ViolationSourceCodeTree alarmTree = new ViolationSourceCodeTree(javaFile, startLine, endLine);
alarmTree.extract(alarmType);
List<ITree> matchedTrees = alarmTree.getViolationSourceCodeTrees();
if (matchedTrees.size() == 0) {
emptyStatements.append(alarmType + "," + fileName + "," + startLine + "," + endLine + "\n");
a ++;
continue;
}
SimpleTree simpleTree = new SimpleTree();
simpleTree.setLabel("Block");
simpleTree.setNodeType("Block");
List<SimpleTree> children = new ArrayList<>();
for (ITree matchedTree : matchedTrees) {
SimpleTree simpleT = new SimplifyTree().canonicalizeSourceCodeTree(matchedTree, null);
children.add(simpleT);
}
simpleTree.setChildren(children);
String tokens = Tokenizer.getTokensDeepFirst(simpleTree);
String[] tokensArray = tokens.split(" ");
int length = tokensArray.length;
sizes.append(length + "\n");
tokensBuilder.append(alarmType + ":" + fileName + ":" + alarmTree.getViolationFinalStartLine() + ":" + alarmTree.getViolationFinalEndLine() + ":" + tokens + "\n");
counter ++;
if (counter % 10000 == 0) {
FileHelper.outputToFile(outputPath, tokensBuilder, true);
tokensBuilder.setLength(0);
}
}
}
System.out.println("Volidated Instances: " + counter);
System.out.println("Empty Instances: " + a);
FileHelper.outputToFile(outputPath2, emptyStatements, false);
FileHelper.outputToFile(outputPath.replace(".list", "Sizes.csv"), sizes, false);
FileHelper.outputToFile(outputPath, tokensBuilder, true);
}
private List<Violation> readViolations(String file) {
List<Violation> violations = new ArrayList<>();
String fileContent = FileHelper.readFile(file);
BufferedReader reader = null;
reader = new BufferedReader(new StringReader(fileContent));
String line = null;
try {
while ((line = reader.readLine()) != null) {
String[] positionStr = line.split(":");
int startLine = Integer.parseInt(positionStr[1]);
int endLine = Integer.parseInt(positionStr[2]);
if (startLine == -1 || endLine == -1) {
continue;
}
String alarmType = positionStr[0];
Violation violation = new Violation(startLine, endLine, alarmType);
violations.add(violation);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return violations;
}
}
src/main/java/edu/lu/uni/serval/statistics/.Rapp.history
View File
src/main/java/edu/lu/uni/serval/statistics/Categories.java
-75
View File
@@ -1,75 +0,0 @@
package edu.lu.uni.serval.statistics;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.StringReader;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.utils.FileHelper;
public class Categories {
public static void main(String[] args) throws IOException {
String content = FileHelper.readFile(Configuration.ROOT_PATH + "RQ1/Quantity-per-Fixed_1.0V-Type.csv");//"RQ1/Quantity-per-V-Type.csv");
BufferedReader reader = new BufferedReader(new StringReader(content));
String line = reader.readLine();
int counter = 0;
// int size = 400;
Map<String, Integer> map = new HashMap<>();
map.put("Dodgy code", 0);
map.put("Experimental", 0);
map.put("Internationalization", 0);
map.put("Multithreaded correctness", 0);
map.put("Malicious code vulnerability", 0);
map.put("Performance", 0);
map.put("Correctness", 0);
map.put("Security", 0);
map.put("Bad practice", 0);
map.put("Other", 0);
// List<Integer> list = new ArrayList<>();
Map<String, List<Integer>> map2 = new HashMap<>();
map2.put("Dodgy code", new ArrayList<Integer>());
map2.put("Experimental", new ArrayList<Integer>());
map2.put("Internationalization", new ArrayList<Integer>());
map2.put("Multithreaded correctness", new ArrayList<Integer>());
map2.put("Malicious code vulnerability", new ArrayList<Integer>());
map2.put("Performance", new ArrayList<Integer>());
map2.put("Correctness", new ArrayList<Integer>());
map2.put("Security", new ArrayList<Integer>());
map2.put("Bad practice", new ArrayList<Integer>());
map2.put("Other", new ArrayList<Integer>());
while ((line = reader.readLine()) != null) {
String type = line.substring(line.lastIndexOf(",") + 1);
counter ++;
if (map.containsKey(type)) {
map.put(type, map.get(type) + 1);
} else {
map.put(type, 1);
}
if (counter % 10 == 0) {
for (Map.Entry<String, List<Integer>> entry : map2.entrySet()) {
String type2 = entry.getKey();
Integer integ = map.get(type2);
if (integ == null) {
map2.get(type2).add(0);
} else {
map2.get(type2).add(integ);
}
}
// System.out.println();
}
}
reader.close();
for (Map.Entry<String, Integer> entry : map.entrySet()) {
System.out.println(entry.getKey() + "," + entry.getValue());
}
for (Map.Entry<String, List<Integer>> entry : map2.entrySet()) {
System.out.println(entry.getKey() + "," + entry.getValue());
}
}
}
src/main/java/edu/lu/uni/serval/statistics/FixedProjects.java
-26
View File
@@ -1,26 +0,0 @@
package edu.lu.uni.serval.statistics;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.StringReader;
import edu.lu.uni.serval.utils.FileHelper;
public class FixedProjects {
public static void main(String[] args) throws IOException {
String content = FileHelper.readFile("../FPM_Violations/RQ1/Quantity-per-FixedProj.csv");
BufferedReader reader = new BufferedReader(new StringReader(content));
String line = reader.readLine();
StringBuilder builder = new StringBuilder();
while ((line = reader.readLine()) != null) {
String[] elements = line.split(",");
// String projectName = elements[0];
builder.append(elements[0] + "\n");
}
reader.close();
FileHelper.outputToFile("../FPM_Violations/RQ1/fixedProjects.list", builder, false);
}
}
src/main/java/edu/lu/uni/serval/statistics/FixedVilations.java
-47
View File
@@ -1,47 +0,0 @@
package edu.lu.uni.serval.statistics;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import java.util.Scanner;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.utils.FileHelper;
public class FixedVilations {
public static void main(String[] args) throws IOException {
// projectName,ViolationType,Number.
FileInputStream fis = new FileInputStream(Configuration.ROOT_PATH + "RQ1/fixed-alarms-v1.0.list");
Scanner scanner = new Scanner(fis);
Map<String, Integer> map = new HashMap<>();
int counter = 0;
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
String[] elements = line.split(":");
// String violationType = elements[0];
// String projectName = elements[1];
String key = elements[1] + "," + elements[0];
if (map.containsKey(key)) {
map.put(key, map.get(key)+ 1);
} else {
map.put(key, 1);
}
counter ++;
}
scanner.close();
fis.close();
System.out.println(counter);
StringBuilder builder = new StringBuilder();
for (Map.Entry<String, Integer> entry : map.entrySet()) {
builder.append(entry.getKey()).append(",").append(entry.getValue()).append("\n");
counter -= entry.getValue();
}
FileHelper.outputToFile(Configuration.ROOT_PATH + "RQ1/fixedViolations-v-1.0.csv", builder, false);
System.out.println(counter);
}
}
src/main/java/edu/lu/uni/serval/statistics/HunkSizeComputer.java
-106
View File
@@ -1,106 +0,0 @@
package edu.lu.uni.serval.statistics;
import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.StringReader;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.utils.FileHelper;
import edu.lu.uni.serval.utils.ListSorter;
public class HunkSizeComputer {
public static void main(String[] args) throws IOException {
String violationsFilePath = Configuration.ROOT_PATH + "unFixedInstances/";
sizes(violationsFilePath, ".list");
}
public static void sizes(String filePath, String fileType) throws NumberFormatException, IOException {
List<File> unfixedAlarmFiles = FileHelper.getAllFilesInCurrentDiectory(filePath, fileType);
System.out.println(unfixedAlarmFiles.size());
StringBuilder builder = new StringBuilder();
int counter = 0;
int size = 0;
int i = 0;
List<Integer> sizes = new ArrayList<>();
Map<Integer, Integer> map = new HashMap<>();// <size, amount>
for (File file : unfixedAlarmFiles) {
String content = FileHelper.readFile(file);
BufferedReader reader = new BufferedReader(new StringReader(content));
String line = null;
while ((line = reader.readLine()) != null) {
String[] elements = line.split(",");
int startLine = Integer.parseInt(elements[4]);
int endLine = Integer.parseInt(elements[5]);
i ++;
if (startLine == -1 || endLine == -1) continue;
int range = endLine - startLine + 1;
builder.append(range + "\n");
if (range != 1) {
counter ++;
}
size ++;
sizes.add(range);
if (map.containsKey(range)) {
map.put(range, map.get(range) + 1);
} else {
map.put(range, 1);
}
}
reader.close();
}
System.out.println(i);
String content = FileHelper.readFile("Dataset/fixed-alarms-v1.0.list");
BufferedReader reader = new BufferedReader(new StringReader(content));
String line = null;
while ((line = reader.readLine()) != null) {
int arrowIndex = line.indexOf("=>");
String buggyInfo = line.substring(0, arrowIndex);
String[] buggyElements = buggyInfo.split(":");
int startLine = Integer.parseInt(buggyElements[4]);
int endLine = Integer.parseInt(buggyElements[5]);
if (startLine == -1 || endLine == -1) continue;
int range = endLine - startLine + 1;
if (range != 1) {
counter ++;
}
builder.append(range + "\n");
size ++;
sizes.add(range);
if (map.containsKey(range)) {
map.put(range, map.get(range) + 1);
} else {
map.put(range, 1);
}
}
reader.close();
FileHelper.outputToFile("Dataset/sizes.csv", builder, false);
System.out.println(size);
System.out.println(counter);
ListSorter<Integer> sorter = new ListSorter<>(sizes);
sizes = sorter.sortAscending();
System.out.println(sizes.get((int) (sizes.size() * 0.7)));
System.out.println(sizes.get((int) (sizes.size() * 0.8)));
System.out.println(sizes.get((int) (sizes.size() * 0.9)));
System.out.println(sizes.get((int) (sizes.size() * 0.95)));
int sum = 0;
for (i = 1; i <= 100; i ++) {
sum += map.get(i);
System.out.println(i + "," + String.format("%.2f", ((double) sum / size * 100)));
}
}
}
src/main/java/edu/lu/uni/serval/statistics/NumberOfFilesAndCodeLines.java
-74
View File
@@ -1,74 +0,0 @@
package edu.lu.uni.serval.statistics;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.List;
import java.util.Locale;
import java.util.Scanner;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.utils.FileHelper;
/**
* Statistics of files and code lines of projects in Live Study.
*
* @author kui.liu
*
*/
public class NumberOfFilesAndCodeLines {
public static void main(String[] args) {
String projectsPath = Configuration.ROOT_PATH + "LiveStudy/BugsInfo/";
File[] projects = new File(projectsPath).listFiles();
for (File project : projects) {
if (project.isDirectory()) {
if (project.getName().equals("poi"))
new NumberOfFilesAndCodeLines().statistic(project.getName());
}
}
}
public void statistic(String project) {
List<File> javaFiles = FileHelper.getAllFiles(Configuration.ROOT_PATH + "LiveStudy/projects/" + project, ".java");
int numberOfFiles = 0;
int LOC = 0;
for (File javaFile : javaFiles) {
if (!javaFile.getPath().toLowerCase(Locale.ENGLISH).contains("test")) {
numberOfFiles ++;
LOC += readLinesOfCode(javaFile);
}
// numberOfFiles ++;
// LOC += readLinesOfCode(javaFile);
}
System.out.println(project + " : Files = " + numberOfFiles + ", LOC = " + LOC);
}
private int readLinesOfCode(File javaFile) {
int LOC = 0;
Scanner scanner = null;
FileInputStream fis = null;
try {
fis = new FileInputStream(javaFile);
scanner = new Scanner(fis);
while (scanner.hasNextLine()) {
scanner.nextLine();
LOC ++;
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} finally {
try {
scanner.close();
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return LOC;
}
}
src/main/java/edu/lu/uni/serval/statistics/ReadSpearmanResults.java
-104
View File
@@ -1,104 +0,0 @@
package edu.lu.uni.serval.statistics;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.StringReader;
import edu.lu.uni.serval.utils.FileHelper;
public class ReadSpearmanResults {
public static void main(String[] args) throws IOException {
String fileName = "../FPM_Violations/RQ1/SpearmanResults.list";
String content = FileHelper.readFile(fileName);
BufferedReader reader = new BufferedReader(new StringReader(content));
String line = null;
String str1 = "1";
String str2 = "2";
String str3 = "3";
String str4 = "4";
String str5 = "5";
String str6 = "6";
String str7 = "7";
String str8 = "8";
String str9 = "9";
String str10 = "10";
int counter = 0;
while ((line = reader.readLine()) != null) {
// String[] elements = line.split("");
counter ++;
if ((counter / 10 == 2 || counter / 10 == 4) && counter % 10 == 1) {
str1 += "@@@@1";
str2 += "@@@@2";
str3 += "@@@@3";
str4 += "@@@@4";
str5 += "@@@@5";
str6 += "@@@@6";
str7 += "@@@@7";
str8 += "@@@@8";
str9 += "@@@@9";
str10 += "@@@@10";
}
int n = counter % 10;
switch (n) {
case 1:
str1 += "@" + line;
break;
case 2:
str2 += "@" + line;
break;
case 3:
str3 += "@" + line;
break;
case 4:
str4 += "@" + line;
break;
case 5:
str5 += "@" + line;
break;
case 6:
str6 += "@" + line;
break;
case 7:
str7 += "@" + line;
break;
case 8:
str8 += "@" + line;
break;
case 9:
str9 += "@" + line;
break;
case 0:
str10 += "@" + line;
break;
}
}
reader.close();
str1 = str1.replace(".", ",");
str2 = str2.replace(".", ",");
str3 = str3.replace(".", ",");
str4 = str4.replace(".", ",");
str5 = str5.replace(".", ",");
str6 = str6.replace(".", ",");
str7 = str7.replace(".", ",");
str8 = str8.replace(".", ",");
str9 = str9.replace(".", ",");
str10 = str10.replace(".", ",");
System.out.println(str1);
System.out.println(str2);
System.out.println(str3);
System.out.println(str4);
System.out.println(str5);
System.out.println(str6);
System.out.println(str7);
System.out.println(str8);
System.out.println(str9);
System.out.println(str10);
}
}
src/main/java/edu/lu/uni/serval/statistics/Spearman.R
-362
View File
@@ -1,362 +0,0 @@
a <- read.csv("~/Public/git/FPM_Violations/RQ1/TenFolds-Q/Ten-fold-all-fixed-violation-type1-70.csv", header=T)
d<-cor.test( ~ X0 + X0.T,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = FALSE, sep = "@")
d<-cor.test( ~ X1 + X1.T,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X2 + X2.T,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X3 + X3.T,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X4 + X4.T,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X5 + X5.T,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X6 + X6.T,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X7 + X7.T,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X8 + X8.T,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X9 + X9.T,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X0.R + X0.T.R,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X1.R + X1.T.R,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X2.R + X2.T.R,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X3.R + X3.T.R,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X4.R + X4.T.R,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X5.R + X5.T.R,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X6.R + X6.T.R,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X7.R + X7.T.R,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X8.R + X8.T.R,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X9.R + X9.T.R,
data=a, method = "spearman",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X0 + X0.T,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X1 + X1.T,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X2 + X2.T,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X3 + X3.T,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X4 + X4.T,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X5 + X5.T,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X6 + X6.T,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X7 + X7.T,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X8 + X8.T,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X9 + X9.T,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X0.R + X0.T.R,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X1.R + X1.T.R,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X2.R + X2.T.R,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X3.R + X3.T.R,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X4.R + X4.T.R,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X5.R + X5.T.R,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X6.R + X6.T.R,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X7.R + X7.T.R,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X8.R + X8.T.R,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X9.R + X9.T.R,
data=a, method = "pearson",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X0 + X0.T,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X1 + X1.T,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X2 + X2.T,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X3 + X3.T,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X4 + X4.T,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X5 + X5.T,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X6 + X6.T,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X7 + X7.T,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X8 + X8.T,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X9 + X9.T,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X0.R + X0.T.R,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X1.R + X1.T.R,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X2.R + X2.T.R,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X3.R + X3.T.R,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X4.R + X4.T.R,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X5.R + X5.T.R,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X6.R + X6.T.R,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X7.R + X7.T.R,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X8.R + X8.T.R,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
d<-cor.test( ~ X9.R + X9.T.R,
data=a, method = "kendall",
continuity = FALSE,
conf.level = 0.95)
write(c(d$estimate, d$p.value), file = "~/Public/git/FPM_Violations/RQ1/SpearmanResults.list",append = TRUE, sep = "@")
a=1
src/main/java/edu/lu/uni/serval/statistics/SpearmanCodeGenerator.java
-81
View File
@@ -1,81 +0,0 @@
package edu.lu.uni.serval.statistics;
public class SpearmanCodeGenerator {
public static void main(String[] args) {
generateSpearmanCode();
}
public static void generateSpearmanCode() {
for (int i = 0; i < 10; i ++) {
String s = "d<-cor.test( ~ X" + i + " + X" + i + ".T, \n";
s += " data=a,";
s += " method = \"spearman\",\n";
s += " continuity = FALSE,\n";
s += " conf.level = 0.95)\n";
if (i == 0) {
s += "write(c(d$estimate, d$p.value), file = \"~/Public/git/FPM_Violations/RQ1/SpearmanResults.list\"," +
"append = FALSE, sep = \"@\")\n";
} else {
s += "write(c(d$estimate, d$p.value), file = \"~/Public/git/FPM_Violations/RQ1/SpearmanResults.list\"," +
"append = TRUE, sep = \"@\")\n";
}
System.out.println(s);
}
for (int i = 0; i < 10; i ++) {
String s = "d<-cor.test( ~ X" + i + ".R + X" + i + ".T.R, \n";
s += " data=a,";
s += " method = \"spearman\",\n";
s += " continuity = FALSE,\n";
s += " conf.level = 0.95)\n";
s += "write(c(d$estimate, d$p.value), file = \"~/Public/git/FPM_Violations/RQ1/SpearmanResults.list\"," +
"append = TRUE, sep = \"@\")\n";
System.out.println(s);
}
for (int i = 0; i < 10; i ++) {
String s = "d<-cor.test( ~ X" + i + " + X" + i + ".T, \n";
s += " data=a,";
s += " method = \"pearson\",\n";
s += " continuity = FALSE,\n";
s += " conf.level = 0.95)\n";
s += "write(c(d$estimate, d$p.value), file = \"~/Public/git/FPM_Violations/RQ1/SpearmanResults.list\"," +
"append = TRUE, sep = \"@\")\n";
System.out.println(s);
}
for (int i = 0; i < 10; i ++) {
String s = "d<-cor.test( ~ X" + i + ".R + X" + i + ".T.R, \n";
s += " data=a,";
s += " method = \"pearson\",\n";
s += " continuity = FALSE,\n";
s += " conf.level = 0.95)\n";
s += "write(c(d$estimate, d$p.value), file = \"~/Public/git/FPM_Violations/RQ1/SpearmanResults.list\"," +
"append = TRUE, sep = \"@\")\n";
System.out.println(s);
}
for (int i = 0; i < 10; i ++) {
String s = "d<-cor.test( ~ X" + i + " + X" + i + ".T, \n";
s += " data=a,";
s += " method = \"kendall\",\n";
s += " continuity = FALSE,\n";
s += " conf.level = 0.95)\n";
s += "write(c(d$estimate, d$p.value), file = \"~/Public/git/FPM_Violations/RQ1/SpearmanResults.list\"," +
"append = TRUE, sep = \"@\")\n";
System.out.println(s);
}
for (int i = 0; i < 10; i ++) {
String s = "d<-cor.test( ~ X" + i + ".R + X" + i + ".T.R, \n";
s += " data=a,";
s += " method = \"kendall\",\n";
s += " continuity = FALSE,\n";
s += " conf.level = 0.95)\n";
s += "write(c(d$estimate, d$p.value), file = \"~/Public/git/FPM_Violations/RQ1/SpearmanResults.list\"," +
"append = TRUE, sep = \"@\")\n";
System.out.println(s);
}
}
}
src/main/java/edu/lu/uni/serval/statistics/Statistic.java
-947
View File
@@ -1,947 +0,0 @@
package edu.lu.uni.serval.statistics;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.StringReader;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Scanner;
import edu.lu.uni.serval.config.Configuration;
import edu.lu.uni.serval.utils.FileHelper;
import edu.lu.uni.serval.utils.ListSorter;
import edu.lu.uni.serval.utils.MapSorter;
public class Statistic {
private static Map<String, Integer> map1 = new HashMap<String, Integer>();
private static Map<String, Integer> map2 = new HashMap<>();
public static void main(String[] args) throws IOException {
/*
* DM_DEFAULT_ENCODING
* NP_NONNULL_RETURN_VIOLATION
* NP_NULL_ON_SOME_PATH_FROM_RETURN_VALUE
* NP_PARAMETER_MUST_BE_NONNULL_BUT_MARKED_AS_NULLABLE
* ODR_OPEN_DATABASE_RESOURCE
* PZLA_PREFER_ZERO_LENGTH_ARRAYS
* RI_REDUNDANT_INTERFACES
* RV_RETURN_VALUE_IGNORED_BAD_PRACTICE
* RV_RETURN_VALUE_IGNORED_NO_SIDE_EFFECT
* SE_NO_SERIALVERSIONID
* SF_SWITCH_NO_DEFAULT
* SIC_INNER_SHOULD_BE_STATIC
* SQL_PREPARED_STATEMENT_GENERATED_FROM_NONCONSTANT_STRING
* UC_USELESS_CONDITION
* UC_USELESS_OBJECT
* UCF_USELESS_CONTROL_FLOW
* WMI_WRONG_MAP_ITERATOR
*/
// /*
// * Quantities' distribution of all violation types.
// */
// quantityOfEachViolationType();
// /*
// * Widespread of each violation type.
// */
// widespreadOfEachViolationType();
// /*
// * Statistics of categories of all violation types.
// */
// statisticWithCategories();
//
// /*
// * Quantity of each violation type in each project.
// */
// reloadData();
//
// /*
// * Quantities' distribution of all fixed violation types.
// * Widespread of each fixed violation type.
// * Statistics of categories of all fixed violation types.
// * Quantity of each fixed violation type in each project.
// */
// statisticOfFixedViolations();
// fixedVSunfixed();
/**
* Do statistics from two files:
*/
statistics("../FPM_Violations/RQ1/all-leafnodes-per-project-vtype.csv", "", map1, 16918530, 730);
// statistics("../FPM_Violations/RQ1/distinct-fixed-summary-per-project-vtype.csv", "Fixed");
statistics("../FPM_Violations/RQ1/fixedViolations-v-1.0.csv", "Fixed_1.0", map2, 88927, 548);
// fixedVSunfixed();
StringBuilder builder = new StringBuilder();
for (Map.Entry<String, Integer> entry : map1.entrySet()) {
String key = entry.getKey();
builder.append(key + "," + entry.getValue() + "," + (map2.containsKey(key) ? map2.get(key) : 0) + "\n");
}
FileHelper.outputToFile(Configuration.ROOT_PATH + "RQ2/quantity-ratios.csv", builder, false);
// statisticsOfFixedViolations();
//rsync -avP gaia-cluster:/work/users/kliu/FixPattern/FPM_Violations/UnfixedViolations/BC_UNCONFIRMED_CAST/Sizes.list Sizes/Sizes1.list
// String s = "rsync -avP gaia-cluster:/work/users/kliu/FixPattern/FPM_Violations/UnfixedViolations_RQ3/";
// File[] files = new File(Configuration.ROOT_PATH + "RQ3_1/UnfixedInstances/").listFiles();
// int i = 0;
// for (File file : files) {
// if (file.getName().endsWith(".list")) {
// i ++;
// System.out.println(s + FileHelper.getFileNameWithoutExtension(file) + "/Sizes.list Sizes/Sizes" + i + ".list");
// }
// }
// tokenSizes();
}
public static void tokenSizes() {
List<Integer> sizes = new ArrayList<>();
List<File> files = FileHelper.getAllFilesInCurrentDiectory(Configuration.ROOT_PATH + "RQ3_1/Sizes/", ".list");
StringBuilder builder = new StringBuilder();
for (File file : files) {
try {
String content = FileHelper.readFile(file);
BufferedReader reader = new BufferedReader(new StringReader(content));
String line = null;
while ((line = reader.readLine()) != null) {
sizes.add(Integer.parseInt(line));
builder.append("1,").append(line).append("\n");
}
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
System.out.println(sizes.size());
FileHelper.outputToFile(Configuration.ROOT_PATH + "RQ3_1/Sizes.csv", builder, false);
}
public static void statisticsOfFixedViolations() {
String statistic = "../FPM_Violations/OUTPUT3/";
List<File> files = FileHelper.getAllFiles(statistic, ".list");
int positions = 0;
int numV = 0;
int testAlarms = 0;
int nullGumTreeResults = 0;
int nullMappingGumTreeResults = 0;
int pureDeletion = 0;
int timeout = 0;
int noSourceCodeChagnes = 0;
int largeHunk = 0;
int nullSourceCode = 0;
int noStatementChanges = 0;
int nullDiffentry = 0;
int TestingInfo = 0;
int i = 0;
Map<String, Integer> types1 = new HashMap<>();
for (File file : files) {
if (file.getName().startsWith("statistic")) {
String content = FileHelper.readFile(file);
BufferedReader reader = new BufferedReader(new StringReader(content));
String line = null;
i ++;
try {
while ((line = reader.readLine()) != null) {
if (line.startsWith("Positions")) {
positions += Integer.parseInt(line.substring(line.lastIndexOf(":") + 1).trim());
} else if (line.startsWith("NumViolations")) {
numV += Integer.parseInt(line.substring(line.lastIndexOf(":") + 1).trim());
} else
if (line.startsWith("TestViolations")) {
testAlarms += Integer.parseInt(line.substring(line.lastIndexOf(":") + 1).trim());
} else if (line.startsWith("NullGumTreeResults")) {
nullGumTreeResults += Integer.parseInt(line.substring(line.lastIndexOf(":") + 1).trim());
} else if (line.startsWith("NoSourceCodeChanges")) {
noSourceCodeChagnes += Integer.parseInt(line.substring(line.lastIndexOf(":") + 1).trim());
} else if (line.startsWith("NoStatementChanges")) {
noStatementChanges += Integer.parseInt(line.substring(line.lastIndexOf(":") + 1).trim());
} else if (line.startsWith("NullDiffEntry")) {
nullDiffentry += Integer.parseInt(line.substring(line.lastIndexOf(":") + 1).trim());
} else if (line.startsWith("NullMatchedGumTreeResults")) {
nullMappingGumTreeResults += Integer.parseInt(line.substring(line.lastIndexOf(":") + 1).trim());
} else if (line.startsWith("PureDeletion")) {
pureDeletion += Integer.parseInt(line.substring(line.lastIndexOf(":") + 1).trim());
} else if (line.startsWith("LargeHunk")) {
largeHunk += Integer.parseInt(line.substring(line.lastIndexOf(":") + 1).trim());
} else if (line.startsWith("NullSourceCode")) {
nullSourceCode += Integer.parseInt(line.substring(line.lastIndexOf(":") + 1).trim());
} else if (line.startsWith("Timeout")) {
timeout += Integer.parseInt(line.substring(line.lastIndexOf(":") + 1).trim());
} else if (line.startsWith("TestingInfo")) {
TestingInfo +=Integer.parseInt(line.substring(line.lastIndexOf(":") + 1).trim());
}
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
if (file.getName().startsWith("UnfixedV")) {
String content = FileHelper.readFile(file);
BufferedReader reader = new BufferedReader(new StringReader(content));
String line = null;
try {
while ((line = reader.readLine()) != null) {
// types1
if (line.startsWith("## OAR [")) break;
String type = line.substring(0, line.indexOf(":"));
if (types1.containsKey(type)) {
types1.put(type, types1.get(type) + 1);
} else {
types1.put(type, 1);
}
if (line.startsWith("#NullSourceCode:")) {
System.out.println(line);
}
}
} catch(IOException e) {
e.printStackTrace();
} finally {
try {
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
Map<String, Integer> types = new HashMap<>();
// FileInputStream fis = new FileInputStream("../FPM_Violations/OAR.FPM.4222208.stderr");
// Scanner scanner = new Scanner(fis);
// while (scanner.hasNextLine()) {
// String line = scanner.nextLine();
// if (line.startsWith("## OAR [")) break;
// String type = line.substring(0, line.indexOf(":"));
// if (types.containsKey(type)) {
// types.put(type, types.get(type) + 1);
// } else {
// types.put(type, 1);
// }
// }
// scanner.close();
// fis.close();
// int sum = 0;
// int sum2 = 0;
// for (Map.Entry<String, Integer> entry : types.entrySet()) {
// System.out.println(entry.getKey() + ": " + entry.getValue());
// if (!entry.getKey().startsWith("#PureDeletion")) {
// if (!entry.getKey().startsWith("#Timeout") && !entry.getKey().startsWith("#TestViolation"))
// sum += entry.getValue();
// else sum2 += entry.getValue();
// }
// }
int sum3 = 0;
int sum4 = 0;
int sum5 = 0;
for (Map.Entry<String, Integer> entry : types1.entrySet()) {
System.out.println(entry.getKey() + ": " + entry.getValue());
if (!entry.getKey().startsWith("#PureDeletion")) {
if (!entry.getKey().startsWith("#Timeout") && !entry.getKey().startsWith("#TestViolation"))
sum3 += entry.getValue();
else sum4 += entry.getValue();
sum5+= entry.getValue();
}
}
System.out.println(sum5);
System.out.println(i);
System.out.println("\n\nStatistics:\nPositions: " + positions);
System.out.println("NumViolations: " + numV);
System.out.println("\nTestViolation: " + testAlarms + " :: " + types.get("#TestViolation") + " :: " + types1.get("#TestViolation"));
System.out.println("NullGumTreeResults: " + nullGumTreeResults + " :: " + types.get("#NullGumTreeResults") + " :: " + types1.get("#NullGumTreeResults"));
System.out.println("NoSourceCodeChange: " + noSourceCodeChagnes + " :: " + types.get("#NoSourceCodeChange") + " :: " + types1.get("#NoSourceCodeChange"));
System.out.println("NoStatementChange: " + noStatementChanges + " :: " + types.get("#NoStatementChange") + " :: " + types1.get("#NoStatementChange"));
System.out.println("NullDiffEntry: " + nullDiffentry + " :: " + types.get("#NullDiffEntry") + " :: " + types1.get("#NullDiffEntry"));
System.out.println("NullMatchedGumTreeResult: " + nullMappingGumTreeResults + " :: " + types.get("#NullMatchedGumTreeResult") + " :: " + types1.get("#NullMatchedGumTreeResult"));
System.out.println("PureDeletion: " + pureDeletion + " :: " + types.get("#PureDeletion") + " :: " + types1.get("#PureDeletion"));
System.out.println("LargeHunk: " + largeHunk + " :: " + types.get("#LargeHunk") + " :: " + types1.get("#LargeHunk"));
System.out.println("NullSourceCode: " + nullSourceCode + " :: " + types.get("#NullSourceCode") + " :: " + types1.get("#NullSourceCode"));
System.out.println("Timeout: " + timeout + " :: " + types.get("#Timeout") + " :: " + types1.get("#Timeout"));
System.out.println("TestingInfo: " + TestingInfo + " :: " + types.get("#TestingInfo") + " :: " + types1.get("#TestingInfo"));
// System.out.println("A: " + (positions + testAlarms + timeout));
// System.out.println("B: " + (numV + testAlarms + timeout));
System.out.println(testAlarms + nullGumTreeResults + noSourceCodeChagnes + noStatementChanges +
nullDiffentry + nullMappingGumTreeResults + nullSourceCode + timeout + TestingInfo);
// System.out.println(nullGumTreeResults + noSourceCodeChagnes + noStatementChanges +
// nullDiffentry + nullMappingGumTreeResults + nullSourceCode + TestingInfo);
// System.out.println(testAlarms + timeout);
// System.out.println(sum);
// System.out.println(sum2);
System.out.println(sum3);
System.out.println(sum4);
System.out.println(types1);
/*32690 56237 ,, 88927 31782
* Statistics:
TestViolation: 4682 :: null :: null
NullGumTreeResults: 0 :: null :: null
NoSourceCodeChange: 7010 :: null :: 6984
NoStatementChange: 163 :: null :: 163
NullDiffEntry: 6943 :: null :: 6943
NullMatchedGumTreeResult: 25747 :: null :: 25747
PureDeletion: 10815 :: null :: 10815
LargeHunk: 6318 :: null :: 6318
NullSourceCode: 1025 :: null :: 1025
Timeout: 0 :: null :: null
TestingInfo: 0 :: null :: null
A: 4682
B: 4682
51888
47206
4682
47180
0
*/
}
public static void statistics(String fileName, String type, Map<String, Integer> map, int totalQ, int totalS) throws IOException {
FileInputStream fis = new FileInputStream(fileName);
Scanner scanner = new Scanner(fis);
Map<String, Integer> violationTypesMap = new HashMap<>();
Map<String, Integer> projectsMap = new HashMap<>();
Map<String, Integer> perVperProjMap = new HashMap<>();
Map<String, List<String>> perVProjs = new HashMap<>();
Map<String, List<String>> widespreadViolationsMap = new HashMap<>();
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
String[] elements = line.split(",");
String projectName = elements[0];
String violationType = elements[1];
int quantity = Integer.parseInt(elements[2]);
addToMap(projectsMap, projectName, quantity);
addToMap(violationTypesMap, violationType, quantity);
addToMap(perVperProjMap, projectName + "," + violationType, quantity);
if (widespreadViolationsMap.containsKey(violationType)) {
List<String> projectList = widespreadViolationsMap.get(violationType);
if (!projectList.contains(projectName)) {
projectList.add(projectName);
}
} else {
List<String> projectList = new ArrayList<>();
projectList.add(projectName);
widespreadViolationsMap.put(violationType, projectList);
}
if (perVProjs.containsKey(violationType)) {
perVProjs.get(violationType).add(projectName);
} else {
List<String> projs = new ArrayList<>();
projs.add(projectName);
perVProjs.put(violationType, projs);
}
}
scanner.close();
fis.close();
// Category
MapSorter<String, Integer> sorter = new MapSorter<String, Integer>();
violationTypesMap = sorter.sortByValueDescending(violationTypesMap);
projectsMap = sorter.sortByValueDescending(projectsMap);
Map<String, String> categories = new HashMap<>();
Map<String, List<String>> categoryVList = new HashMap<>();
String content = FileHelper.readFile("../FPM_Violations/RQ1/ViolationCategory.list");
BufferedReader reader = new BufferedReader(new StringReader(content));
String line = null;
while ((line = reader.readLine()) != null) {
String[] elements = line.split("@@");
categories.put(elements[1], elements[0]); // Violation type, category
}
reader.close();
// Sort Violation types by widespread.
Map<String, Integer> widespreadOfAllViolations = new HashMap<>();
for (Map.Entry<String, List<String>> entry : widespreadViolationsMap.entrySet()) {
widespreadOfAllViolations.put(entry.getKey(), entry.getValue().size());
}
widespreadOfAllViolations = sorter.sortByValueDescending(widespreadOfAllViolations);
StringBuilder wbuilder = new StringBuilder("Type,Identifier,Quantity,Category\n");
int identifier1 = 0;
for (Map.Entry<String, Integer> entry : widespreadOfAllViolations.entrySet()) {
identifier1 ++;
String violationType = entry.getKey();
String category = categories.get(violationType);
if (category == null || category.equals("null")) {
category = "Other";
}
wbuilder.append(violationType + "," + identifier1 + "," + entry.getValue() + "," + category + "\n");
if (categoryVList.containsKey(category)) {
categoryVList.get(category).add(violationType);
} else {
List<String> list = new ArrayList<>();
list.add(violationType);
categoryVList.put(category, list);
}
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Widespread-per-" + type + "V-Type.csv", wbuilder, false);
// output statistics
List<String> sortedViolationTypes = new ArrayList<>();
Map<String, Integer> quantityOfCategory = new HashMap<>();
Map<String, List<String>> violationTypesOfCategory = new HashMap<>();
Map<String, List<String>> projectsOfCategory = new HashMap<>();
StringBuilder violationsBuilder = new StringBuilder("Type,Identifier,Quantity,Ratio,Widespread,Ratio2, Categories\n");
int identifier = 0;
for (Map.Entry<String, Integer> entry : violationTypesMap.entrySet()) {
String violationType = entry.getKey();
int quantity = entry.getValue();
identifier ++;
String category = categories.get(violationType);
if (category == null || category.equals("null")) {
category = "Other";
}
int spread = widespreadViolationsMap.get(violationType).size();
violationsBuilder.append(violationType + "," + identifier + "," + quantity + "," + ((double) quantity / totalQ * 100) + ","
+ spread + "," + ((double) spread / totalS * 100) + "," + category + "\n");
if (quantityOfCategory.containsKey(category)) {
quantityOfCategory.put(category, quantityOfCategory.get(category) + quantity);
} else {
quantityOfCategory.put(category, quantity);
}
if (violationTypesOfCategory.containsKey(category)) {
List<String> violationTypes = violationTypesOfCategory.get(category);
if (!violationTypes.contains(violationType)) {
violationTypes.add(violationType);
}
} else {
List<String> violationTypes = new ArrayList<>();
violationTypes.add(violationType);
violationTypesOfCategory.put(category, violationTypes);
}
if (projectsOfCategory.containsKey(category)) {
List<String> projs = projectsOfCategory.get(category);
List<String> projs2 = perVProjs.get(violationType);
for (String proj : projs2) {
if (!projs.contains(proj)) {
projs.add(proj);
}
}
} else {
projectsOfCategory.put(category, perVProjs.get(violationType));
}
sortedViolationTypes.add(violationType);
map.put(violationType, quantity);
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Quantity-per-" + type + "V-Type.csv", violationsBuilder, false);
StringBuilder pBuilder = new StringBuilder("Project,Quantity\n");
List<String> projectNames = new ArrayList<>();
for (Map.Entry<String, Integer> entry : projectsMap.entrySet()) {
String project = entry.getKey();
pBuilder.append(project + "," + entry.getValue() + "\n");
projectNames.add(project);
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Quantity-per-" + type + "Proj.csv", pBuilder, false);
StringBuilder categoryBuilder = new StringBuilder("Category,Quantity,Types,Projects\n");
for (Map.Entry<String, Integer> entry : quantityOfCategory.entrySet()) {
String key = entry.getKey();
categoryBuilder.append(key + "," + entry.getValue() + "," + violationTypesOfCategory.get(key).size() + "," + projectsOfCategory.get(key).size() + "\n");
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Quantity-per-" + type + "Category.csv", categoryBuilder, false);
StringBuilder builder = new StringBuilder("Projects");
String a = "";
String b = "";
for (int i = 0; i < sortedViolationTypes.size(); i ++) {
builder.append("," + sortedViolationTypes.get(i));
if (i < 50) {
a += "a$" + sortedViolationTypes.get(i) + ",";
b += "'" + (i + 1) + "',";
}
}
builder.append("\n");
System.out.println(a);
System.out.println(b);
// for (int i = 0; i < sortedViolationTypes.size(); i ++) {
// builder.append("," + i);
// }
// builder.append("\n");
StringBuilder bui = new StringBuilder();
for (int i = 0; i < projectNames.size(); i++) {
String projectName = projectNames.get(i);
builder.append(projectName);
for (int j = 0; j < sortedViolationTypes.size(); j ++) {
String violationType = sortedViolationTypes.get(j);
String key = projectName + "," + violationType;
Integer value = perVperProjMap.get(key);
if (value == null) {
value = 0;
} else {
bui.append(projectName).append(",").append(violationType).append(",").append(value).append(",").append(categories.get(violationType) == null ? "Other" : categories.get(violationType)).append("\n");
}
builder.append("," + value);
}
builder.append("\n");
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Distribution-per-project-per-" + type + "type.csv", builder, false);
FileHelper.outputToFile("../FPM_Violations/RQ1/Distribution-per-project-per-" + type + "type___.csv", bui, false);
StringBuilder ssbuilder = new StringBuilder("Type, Identifier, Quantity, Category\n");
Map<String, Integer> perTypePerProj = new HashMap<>();
Map<String, List<String>> categoryProjects = new HashMap<>();
List<String> others = new ArrayList<>();
for (int j = 0; j < sortedViolationTypes.size(); j ++) {
String violationType = sortedViolationTypes.get(j);
List<Integer> projs = new ArrayList<>();
for (int i = 0; i < projectNames.size(); i++) {
String projectName = projectNames.get(i);
String key = projectName + "," + violationType;
Integer value = perVperProjMap.get(key);
if (value != null) {
String category = categories.get(violationType);
if (category == null || category.equals("null")) {
category = "Other";
if (!others.contains(violationType)) {
others.add(violationType);
}
}
ssbuilder.append(violationType + "," + (j + 1) + "," + value + "," + category + "\n");
projs.add(value);
if (categoryProjects.containsKey(category)) {
List<String> projects = categoryProjects.get(category);
if (!projects.contains(projectName)) {
projects.add(projectName);
}
} else {
List<String> projects = new ArrayList<>();
projects.add(projectName);
categoryProjects.put(category, projects);
}
}
}
ListSorter<Integer> sorter2 = new ListSorter<Integer>(projs);
projs = sorter2.sortAscending();
int index = projs.size() % 2 == 0 ? projs.size() / 2 - 1 : projs.size() / 2;
perTypePerProj.put(violationType, projs.get(index));
}
System.out.println(others);
for (Map.Entry<String, List<String>> entry : categoryVList.entrySet()) {
System.out.println(entry.getKey() + ":" + entry.getValue().size() + ":" + categoryProjects.get(entry.getKey()).size());
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Distribution-per-project-per-" + type + "type2.csv", ssbuilder, false);
StringBuilder ssbuilder2 = new StringBuilder("Type, Identifier, Quantity, Category\n");
perTypePerProj = sorter.sortByValueDescending(perTypePerProj);
int j = 0;
for (Map.Entry<String, Integer> entry : perTypePerProj.entrySet()) {
String violationType = entry.getKey();
// System.out.println(entry.getValue());
j ++;
for (int i = 0; i < projectNames.size(); i++) {
String projectName = projectNames.get(i);
String key = projectName + "," + violationType;
Integer value = perVperProjMap.get(key);
if (value != null) {
String category = categories.get(violationType);
if (category == null || category.equals("null")) {
category = "Other";
}
ssbuilder2.append(violationType + "," + j + "," + value + "," + category + "\n");
}
}
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Distribution-per-project-per-" + type + "type3.csv", ssbuilder2, false);
// StringBuilder pVpPBuilder = new StringBuilder();
// for (Map.Entry<String, Integer> entry : perVperProjMap.entrySet()) {
// pVpPBuilder.append(entry.getKey() + "," + entry.getValue() + "\n");
// }
// FileHelper.outputToFile("../FPM_Violations/RQ1/Per-project-per-" + type + "type.csv", pVpPBuilder, false);
}
private static void addToMap(Map<String, Integer> map, String key, int value) {
if (map.containsKey(key)) {
map.put(key, map.get(key) + value);
} else {
map.put(key, value);
}
}
public static void quantityOfEachViolationType() {
Map<String, Integer> violationQuantities = readTypeQuantityMap("../FPM_Violations/RQ1/distinct-per-vtype.csv");
Map<String, Integer> violationWidespread = readWidespread("../FPM_Violations/RQ1/distinct-per-project-vtype.csv");
StringBuilder buidler = new StringBuilder("Violation Type, Identifier, Quantity, Quantity of Projects, Ratio\n");
int identifier = 0;
double totality = 15961605d;
int sum = 0;
for (Map.Entry<String, Integer> entry : violationQuantities.entrySet()) {
String key = entry.getKey();
identifier ++;
int quantity = entry.getValue();
sum += quantity;
buidler.append(key + "," + identifier + "," + quantity + "," + violationWidespread.get(key) + "," + (sum / totality) + "\n");
// if (identifier >= 50) break;
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Quantity-per-V-type.csv", buidler, false);
}
public static void widespreadOfEachViolationType() {
Map<String, Integer> violationWidespread = readWidespread("../FPM_Violations/RQ1/distinct-per-project-vtype.csv");
System.out.println("Violation types: " + violationWidespread.size());
StringBuilder buidler = new StringBuilder("Violation Type, Identifier, Quantity of Projects\n");
int identifier = 0;
for (Map.Entry<String, Integer> entry : violationWidespread.entrySet()) {
identifier ++;
buidler.append(entry.getKey() + "," + identifier + "," + entry.getValue() + "\n");
// if (identifier >= 50) break;
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Widespread-per-V-type.csv", buidler, false);
}
public static void statisticWithCategories() throws IOException {
Map<String, String> categories = new HashMap<>();
String content = FileHelper.readFile("../FPM_Violations/RQ1/ViolationCategory.list");
BufferedReader reader = new BufferedReader(new StringReader(content));
String line = null;
while ((line = reader.readLine()) != null) {
String[] elements = line.split("@@");
categories.put(elements[1], elements[0]); // Violation type, category
}
reader.close();
Map<String, Integer> quantityOfCategory = new HashMap<>();
String content2 = FileHelper.readFile("../FPM_Violations/RQ1/distinct-per-vtype.csv");
reader = new BufferedReader(new StringReader(content2));
line = reader.readLine();
while ((line = reader.readLine()) != null) {
String[] elements = line.split(",");
String violationType = elements[0];
int quantity = Integer.parseInt(elements[1]);
String category = categories.get(violationType);
if (quantityOfCategory.containsKey(category)) {
quantityOfCategory.put(category, quantityOfCategory.get(category) + quantity);
} else {
quantityOfCategory.put(category, quantity);
}
}
reader.close();
String categoryQuantity = "Category Type, Quantity\n";
for (Map.Entry<String, Integer> entry : quantityOfCategory.entrySet()) {
categoryQuantity += entry.getKey() + "," + entry.getValue() + "\n";
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Quantity-per-Category.csv", categoryQuantity, false);
}
public static void reloadData() {
// ordered projects by quantity of violations
Map<String, Integer> projectQuantities = readTypeQuantityMap("../FPM_Violations/RQ1/distinct-per-project.csv");
List<String> projectNames = new ArrayList<>();
for (Map.Entry<String, Integer> entry : projectQuantities.entrySet()) {
projectNames.add(entry.getKey());
}
// get ordered types by quantity of violations.
Map<String, Integer> violations = readTypeQuantityMap("../FPM_Violations/RQ1/distinct-per-vtype.csv");
List<String> violationTypes = new ArrayList<>();
for (Map.Entry<String, Integer> entry : violations.entrySet()) {
violationTypes.add(entry.getKey());
}
Map<String, String> perVperProjs = readData("../FPM_Violations/RQ1/distinct-per-project-vtype.csv");
StringBuilder builder = new StringBuilder("Projects");
for (int i = 0; i < violationTypes.size(); i ++) {
builder.append("," + violationTypes.get(i));
}
builder.append("\n");
for (int i = 0; i < projectNames.size(); i++) {
String projectName = projectNames.get(i);
builder.append(projectName);
for (int j = 0; j < violationTypes.size(); j ++) {
String violationType = violationTypes.get(j);
String key = projectName + "," + violationType;
String value = perVperProjs.get(key);
if (value == null) {
value = "0";
}
builder.append("," + value);
}
builder.append("\n");
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Distribution-per-project-per-type.csv", builder, false);
}
public static void statisticOfFixedViolations() throws IOException {
String fileName = "../FPM_Violations/RQ1/fixed-alarms-v1.0.list";
FileInputStream fis = new FileInputStream(fileName);
Scanner scanner = new Scanner(fis);
Map<String, Integer> violations = new HashMap<>();
Map<String, Integer> projects = new HashMap<>();
Map<String, Integer> perVperPro = new HashMap<>();
Map<String, List<String>> widespreadFixedV = new HashMap<>();
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
String[] elements = line.split(":");
String violationType = elements[0];
String projectName = elements[1];
if (violations.containsKey(violationType)) {
violations.put(violationType, violations.get(violationType) + 1);
} else {
violations.put(violationType, 1);
}
if (projects.containsKey(projectName)) {
projects.put(projectName, projects.get(projectName) + 1);
} else {
projects.put(projectName, 1);
}
String perVperProj = projectName + "," + violationType;
if (perVperPro.containsKey(perVperProj)) {
perVperPro.put(perVperProj, perVperPro.get(perVperProj) + 1);
} else {
perVperPro.put(perVperProj, 1);
}
if (widespreadFixedV.containsKey(violationType)) {
List<String> projectList = widespreadFixedV.get(violationType);
if (!projectList.contains(projectName)) {
projectList.add(projectName);
}
} else {
List<String> projectList = new ArrayList<>();
projectList.add(projectName);
widespreadFixedV.put(violationType, projectList);
}
}
scanner.close();
fis.close();
MapSorter<String, Integer> sorter = new MapSorter<String, Integer>();
violations = sorter.sortByValueDescending(violations);
projects = sorter.sortByValueDescending(projects);
Map<String, String> categories = new HashMap<>();
String content = FileHelper.readFile("../FPM_Violations/RQ1/ViolationCategory.list");
BufferedReader reader = new BufferedReader(new StringReader(content));
String line = null;
while ((line = reader.readLine()) != null) {
String[] elements = line.split("@@");
categories.put(elements[1], elements[0]); // Violation type, category
}
reader.close();
List<String> violationTypes = new ArrayList<>();
Map<String, Integer> quantityOfCategory = new HashMap<>();
StringBuilder violationsBuilder = new StringBuilder();
for (Map.Entry<String, Integer> entry : violations.entrySet()) {
String violationType = entry.getKey();
int quantity = entry.getValue();
violationsBuilder.append(violationType + "," + quantity + "," + widespreadFixedV.get(violationType).size() + "\n");
String category = categories.get(violationType);
if (quantityOfCategory.containsKey(category)) {
quantityOfCategory.put(category, quantityOfCategory.get(category) + quantity);
} else {
quantityOfCategory.put(category, quantity);
}
violationTypes.add(violationType);
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Quantity-per-FixedV-Type.csv", violationsBuilder, false);
StringBuilder pBuilder = new StringBuilder();
List<String> projectNames = new ArrayList<>();
for (Map.Entry<String, Integer> entry : projects.entrySet()) {
String project = entry.getKey();
pBuilder.append(project + "," + entry.getValue() + "\n");
projectNames.add(project);
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Quantity-per-FixedV-Proj.csv", pBuilder, false);
StringBuilder categoryBuilder = new StringBuilder();
for (Map.Entry<String, Integer> entry : quantityOfCategory.entrySet()) {
categoryBuilder.append(entry.getKey() + "," + entry.getValue() + "\n");
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Quantity-per-Fixed-Category.csv", categoryBuilder, false);
StringBuilder builder = new StringBuilder("Projects");
for (int i = 0; i < violationTypes.size(); i ++) {
builder.append("," + violationTypes.get(i));
if (i < 500) {
}
}
builder.append("\n");
for (int i = 0; i < projectNames.size(); i++) {
String projectName = projectNames.get(i);
builder.append(projectName);
for (int j = 0; j < violationTypes.size(); j ++) {
String violationType = violationTypes.get(j);
String key = projectName + "," + violationType;
Integer value = perVperPro.get(key);
if (value == null) {
value = 0;
}
builder.append("," + value);
}
builder.append("\n");
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Distribution-per-project-per-Fixed-type.csv", builder, false);
StringBuilder pVpPBuilder = new StringBuilder();
for (Map.Entry<String, Integer> entry : perVperPro.entrySet()) {
pVpPBuilder.append(entry.getKey() + "," + entry.getValue() + "\n");
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Per-project-per-Fixed-type.csv", pVpPBuilder, false);
}
public static void fixedVSunfixed() throws IOException {
String fileName = "../FPM_Violations/RQ1/Quantity-per-V-type.csv"; // all violations
Map<String, Integer> allViolations = readTypeQuantityMap(fileName);
String file = "../FPM_Violations/RQ1/Quantity-per-FixedV-Type.csv"; // fixed violations
Map<String, Integer> fixedViolations = readTypeQuantityMap(file);
MapSorter<String, Integer> sorter = new MapSorter<>();
allViolations = sorter.sortByValueDescending(allViolations);
StringBuilder builder = new StringBuilder("Type,fixed,unfixed,all,fixed Ratio,unfixed Ratio\n");
for (Map.Entry<String, Integer> entry : allViolations.entrySet()) {
String violationType = entry.getKey();
int quantity = entry.getValue();
Integer fixedQuantity = fixedViolations.get(violationType);
if (fixedQuantity == null) {
builder.append(violationType + "," + 0 + "," + quantity + "," + quantity + ",0.0,1\n");
} else {
builder.append(violationType + "," + fixedQuantity + "," + (quantity)
+ "," + quantity + "," + ((double)fixedQuantity) / ((double) quantity)
+ "," + ((double)(quantity)) / ((double) quantity) + "\n");
}
}
FileHelper.outputToFile("../FPM_Violations/RQ1/Distribution-per-Fixed-type-VS-per-unFixed-type.csv", builder, false);
}
private static Map<String, String> readData(String fileName) {
Map<String, String> perVperPros = new HashMap<>();
FileInputStream fis = null;
Scanner scanner = null;
try {
fis = new FileInputStream(fileName);
scanner = new Scanner(fis);
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
String[] elements = line.split(",");
String key = elements[0] + "," + elements[1];
String value = elements[2];
perVperPros.put(key, value);
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} finally {
try {
scanner.close();
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return perVperPros;
}
private static Map<String, Integer> readTypeQuantityMap(String fileName) {
Map<String, Integer> map = new HashMap<>();
String fileContent = FileHelper.readFile(fileName);
BufferedReader reader = new BufferedReader(new StringReader(fileContent));
try {
String line= reader.readLine();
while ((line = reader.readLine()) != null) {
String[] elements = line.split(",");
map.put(elements[0], Integer.valueOf(elements[2]));
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
MapSorter<String, Integer> sorter = new MapSorter<>();
map = sorter.sortByValueDescending(map);
return map;
}
private static Map<String, Integer> readWidespread(String fileName) {
Map<String, Integer> violationWidespread = new HashMap<String, Integer>();
String fileContent = FileHelper.readFile(fileName);
BufferedReader reader = new BufferedReader(new StringReader(fileContent));
try {
String line = null;
while ((line = reader.readLine()) != null) {
String[] strArray = line.split(",");
String key = strArray[1];
if (violationWidespread.containsKey(key)) {
violationWidespread.put(key, violationWidespread.get(key) + 1);
} else {
violationWidespread.put(key, 1);
}
// String[] elements = line.split(",");
// violationWidespread.put(elements[0], Integer.valueOf(elements[1]));
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
MapSorter<String, Integer> sorter = new MapSorter<>();
violationWidespread = sorter.sortByValueDescending(violationWidespread);
return violationWidespread;
}
}

Some files were not shown because too many files have changed in this diff Show More