changes for jar
This commit is contained in:
fixminer
Generated
+1
-1
@@ -21,7 +21,7 @@
|
||||
<module name="fixminer_source" target="1.8" />
|
||||
<module name="gen.jdt" target="1.8" />
|
||||
<module name="gen.srcml" target="1.8" />
|
||||
<module name="gumtree (1) (com.github.gumtreediff)" target="1.8" />
|
||||
<module name="gumtree" target="1.8" />
|
||||
<module name="RichEdit" target="1.8" />
|
||||
</bytecodeTargetLevel>
|
||||
</component>
|
||||
|
||||
Generated
+790
-64
File diff suppressed because it is too large
Load Diff
@@ -24,7 +24,7 @@ Fixminer is a systematic and automated approach to mine relevant and actionable
|
||||
## II. Environment setup
|
||||
|
||||
* OS: macOS Mojave (10.14.3)
|
||||
* JDK8: Oracle jdk1.8 (**important!**)
|
||||
* JDK8: (**important!**)
|
||||
* Download and configure Anaconda
|
||||
* Create an python environment using the [environment file](environment.yml)
|
||||
```powershell
|
||||
@@ -32,7 +32,12 @@ Fixminer is a systematic and automated approach to mine relevant and actionable
|
||||
```
|
||||
* After creating the environment, activate it. It is containing necessary dependencies for redis, and python.
|
||||
```powershell
|
||||
source activate redisEnv
|
||||
source activate fixminerEnv
|
||||
```
|
||||
|
||||
* Update the config.yml file with the corresponding paths in your computer. An example config.yml file could be found under
|
||||
```powershell
|
||||
fixminer_source/src/main/resources/config.yml
|
||||
```
|
||||
<!---
|
||||
[fixminer.sh](python/fixminer.sh)
|
||||
@@ -46,92 +51,49 @@ In order to launch FixMiner, execute [fixminer.sh](python/fixminer.sh)
|
||||
bash fixminer.sh /Users/..../enhancedASTDiff/python/ stats
|
||||
--->
|
||||
|
||||
## III. Replication Data
|
||||
Replication Data:
|
||||
|
||||
[singleBR.pickle](python/data/singleBR.pickle)
|
||||
|
||||
This pickle contains the list bug reports (i.e. bid) with the their corresponding fixes (i.e. commit) for each project in the dataset (i.e. project).
|
||||
|
||||
[bugReports.7z.00X](python/data/bugReports.7z.001)
|
||||
|
||||
This is the dump of the bug reports archive extracted from each commit. These bug reports are not necessarily considered as BUG,CLOSED; this archive is the contins initial bug reports before identifying the fixes.
|
||||
|
||||
[gumInput.7z.001](python/data/gumInput.7z.001)
|
||||
|
||||
This archive contains all the patches in our dataset, formatted in a way that can be processed by GumTree (i.e DiffEntries, prevFiles, revFiles)
|
||||
|
||||
[ALLbugReportsComplete.pickle](python/data/ALLbugReportsComplete.pickle)
|
||||
|
||||
The pickle object that represents the bug reports under the following columns 'bugReport', 'summary', 'description', 'created', 'updated', 'resolved', 'reporterDN', 'reporterEmail','hasAttachment', 'attachmentTime', 'hasPR', 'commentsCount'
|
||||
|
||||
#### Data Viewer
|
||||
|
||||
The data provided with replication package is listed in directory [python/data](python/data)
|
||||
The data is stored in different formats. (e.g. pickle, db, csv, etc..)
|
||||
|
||||
The see content of the .pickle file the following script could be used.
|
||||
|
||||
```python
|
||||
import pickle as p
|
||||
import gzip
|
||||
def load_zipped_pickle(filename):
|
||||
with gzip.open(filename, 'rb') as f:
|
||||
loaded_object = p.load(f)
|
||||
return loaded_object
|
||||
```
|
||||
Usage
|
||||
|
||||
```python
|
||||
result = load_zipped_pickle('code/LANGbugReportsComplete.pickle')
|
||||
# Result is pandas object which can be exported to several formats
|
||||
# Details on how to export is listed in offical library documentation
|
||||
# https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html
|
||||
|
||||
```
|
||||
|
||||
|
||||
## IV. Step-by-Step execution
|
||||
|
||||
#### Before running
|
||||
|
||||
* Update [config file](python/config.yml) with corresponding user paths.
|
||||
* Update [config file](src/main/resources/config.yml) with corresponding user paths.
|
||||
|
||||
* Active the conda environment from shell
|
||||
```powershell
|
||||
source activate redisEnv
|
||||
source activate fixminerEnv
|
||||
```
|
||||
|
||||
In order to launch FixMiner, execute [fixminer.sh](python/fixminer.sh)
|
||||
|
||||
bash fixminer.sh [PATH_TO_PYTHON_FOLDER] [OPTIONS]
|
||||
e.g. bash fixminer.sh Users/fixminer-core/python/ stats
|
||||
bash fixminer.sh [JOB] [CONFIG_FILE]
|
||||
e.g. bash fixminer.sh dataset4c /Users/projects/release/fixminer_source/src/main/resources/config.yml
|
||||
|
||||
#### Running Options
|
||||
|
||||
*FixMiner* needs to specify an option to run.
|
||||
#### Job Types
|
||||
|
||||
1. 'dataset': Create a mining dataset from the project speficied in [subjects.csv](python/data/subjects.csv)
|
||||
Eventually dataset option is the execution of the following steps, which are merged under 'dataset' option
|
||||
for this demo. Eventually single options can be activated by commenting out the corresponding option in [main.py](python/main.py)
|
||||
*FixMiner* needs to specify a job to run.
|
||||
|
||||
`clone` : Clone target project repository.
|
||||
1. __dataset4j__ / __dataset4c__: Create a java/c mining dataset from the projects listed in [subjects.csv](python/data/subjects.csv) or [datasets.csv](python/data/datasets.csv) for c
|
||||
|
||||
`collect` : Collect all commit from repository.
|
||||
|
||||
`fix` : Collect commits linked to a bug report.
|
||||
2. __richEditScript__: Calls the jar file produced as the results as maven package to compute Rich edit scripts.
|
||||
This step can be invoke natively from java or using the [Launcher](src/main/java/edu/lu/uni/serval/richedit/Launcher.java) with appropriate arguments.
|
||||
|
||||
`bugPoints` : Identify the snapshot of the repository before the bug fixing commit introducted.
|
||||
```powershell
|
||||
java -jar FixPatternMiner-1.0.0-jar-with-dependencies.jar /Users/projects/release/fixminer_source/src/main/resources/config.yml RICHEDITSCRIPT
|
||||
|
||||
`brDownload` : Download bug reports recovered from commit log
|
||||
```
|
||||
|
||||
`brParser` : Parse bug reports to select the bug report where type labelled as BUG and status as RESOLVED or CLOSED
|
||||
|
||||
2. 'richEditScript': Rich edit script computation step.
|
||||
|
||||
3. 'shapeSI': Search index creation for shapes. The output of this step is written to [pairs](python/data/pairs)
|
||||
3. __shapeSI__: Search index creation for shapes. The output of this step is written to __pairs__ which will be generated under __datapath__ in [config file](src/main/resources/config.yml)
|
||||
|
||||
4. 'compareShapes' : ShapeTree comparison
|
||||
4. __compareShapes__ : Calls the jar file produced as the results as maven package to compare the trees.
|
||||
This step can be invoke natively from java or using the [Launcher](src/main/java/edu/lu/uni/serval/richedit/Launcher.java) with appropriate arguments.
|
||||
|
||||
```powershell
|
||||
java -jar FixPatternMiner-1.0.0-jar-with-dependencies.jar /Users/projects/release/fixminer_source/src/main/resources/config.yml COMPARETREES
|
||||
|
||||
```
|
||||
|
||||
5. 'cluster': Forms clusters of identical shapetree. The output of this step is written to [shapes](python/data/shapes)
|
||||
|
||||
@@ -234,5 +196,50 @@ It is necessary to run the FixMiner, following the order.
|
||||
|
||||
|
||||
|
||||
## III. Replication Data
|
||||
Replication Data:
|
||||
|
||||
[singleBR.pickle](python/data/singleBR.pickle)
|
||||
|
||||
This pickle contains the list bug reports (i.e. bid) with the their corresponding fixes (i.e. commit) for each project in the dataset (i.e. project).
|
||||
|
||||
[bugReports.7z.00X](python/data/bugReports.7z.001)
|
||||
|
||||
This is the dump of the bug reports archive extracted from each commit. These bug reports are not necessarily considered as BUG,CLOSED; this archive is the contins initial bug reports before identifying the fixes.
|
||||
|
||||
[gumInput.7z.001](python/data/gumInput.7z.001)
|
||||
|
||||
This archive contains all the patches in our dataset, formatted in a way that can be processed by GumTree (i.e DiffEntries, prevFiles, revFiles)
|
||||
|
||||
[ALLbugReportsComplete.pickle](python/data/ALLbugReportsComplete.pickle)
|
||||
|
||||
The pickle object that represents the bug reports under the following columns 'bugReport', 'summary', 'description', 'created', 'updated', 'resolved', 'reporterDN', 'reporterEmail','hasAttachment', 'attachmentTime', 'hasPR', 'commentsCount'
|
||||
|
||||
#### Data Viewer
|
||||
|
||||
The data provided with replication package is listed in directory [python/data](python/data)
|
||||
The data is stored in different formats. (e.g. pickle, db, csv, etc..)
|
||||
|
||||
The see content of the .pickle file the following script could be used.
|
||||
|
||||
```python
|
||||
import pickle as p
|
||||
import gzip
|
||||
def load_zipped_pickle(filename):
|
||||
with gzip.open(filename, 'rb') as f:
|
||||
loaded_object = p.load(f)
|
||||
return loaded_object
|
||||
```
|
||||
Usage
|
||||
|
||||
```python
|
||||
result = load_zipped_pickle('code/LANGbugReportsComplete.pickle')
|
||||
# Result is pandas object which can be exported to several formats
|
||||
# Details on how to export is listed in offical library documentation
|
||||
# https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -12,7 +12,12 @@
|
||||
<groupId>edu.lu.uni.serval</groupId>
|
||||
<artifactId>RichEdit</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<scope>compile</scope>
|
||||
<!-- <scope>compile</scope>-->
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.yaml</groupId>
|
||||
<artifactId>snakeyaml</artifactId>
|
||||
<version>1.21</version>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
<packaging>pom</packaging>
|
||||
@@ -24,6 +29,7 @@
|
||||
|
||||
|
||||
<build>
|
||||
<sourceDirectory>richedit/src/main/java/</sourceDirectory>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
@@ -47,10 +53,8 @@
|
||||
<configuration>
|
||||
<archive>
|
||||
<manifest>
|
||||
<mainClass>
|
||||
<!--edu.lu.uni.serval.fixminer.jobs.CompareTrees-->
|
||||
edu.lu.uni.serval.fixminer.Launcher
|
||||
</mainClass>
|
||||
<addClasspath>true</addClasspath>
|
||||
<mainClass>edu.lu.uni.serval.Launcher</mainClass>
|
||||
</manifest>
|
||||
</archive>
|
||||
<descriptorRefs>
|
||||
|
||||
@@ -3,14 +3,14 @@ java:
|
||||
spinfer:
|
||||
home: /Users/anil.koyuncu/projects/spinfer/spinfer.native
|
||||
dataset:
|
||||
home: /Users/anil.koyuncu/projects/test/fixminer-data/patches/
|
||||
repo: /Users/anil.koyuncu/projects/test/fixminer-data/datasets
|
||||
# home: /data/fixminer-core/python/data/gumInputLinux/
|
||||
home: /Users/anil.koyuncu/projects/test/richedit-data/patches/
|
||||
repo: /Users/anil.koyuncu/projects/test/richedit-data/datasets
|
||||
# home: /data/richedit-core/python/data/gumInputLinux/
|
||||
|
||||
coccinelle:
|
||||
home: /Users/anil.koyuncu/projects/spinfer/statics
|
||||
|
||||
fixminer:
|
||||
datapath: /Users/anil.koyuncu/projects/test/fixminer-data/
|
||||
datapath: /Users/anil.koyuncu/projects/test/richedit-data/
|
||||
projectType : c
|
||||
# datapath: /data/fixminer-core/python/data/
|
||||
# datapath: /data/richedit-core/python/data/
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -179,7 +179,7 @@ def clusterCore(clusterPath, level, match, pairsPath, root, s,action ,token=''):
|
||||
parallelRun(dumpFilesCore,workList)
|
||||
# for wl in workList:
|
||||
# dumpFilesCore(wl)
|
||||
# dumpFilesCore(('hive_d65d5c_96c1dc_ql#src#gen#protobuf#gen-java#org#apache#hadoop#hive#ql#io#orc#OrcProto.txt_31', 'ReturnStatement', 'tokens', '/Users/anil.koyuncu/projects/fixminer-all/enhancedASTDiff/python/data/tokens', '3', '1', '0', 10))
|
||||
# dumpFilesCore(('hive_d65d5c_96c1dc_ql#src#gen#protobuf#gen-java#org#apache#hadoop#hive#ql#io#orc#OrcProto.txt_31', 'ReturnStatement', 'tokens', '/Users/anil.koyuncu/projects/richedit-all/enhancedASTDiff/python/data/tokens', '3', '1', '0', 10))
|
||||
|
||||
|
||||
def dumpFilesCore(t):
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
2020-04-07 17:31:35,222 - 31200 - INFO - commons.py:setEnv - ROOT_DIR : /Users/anilkoyuncu/projects/release/test/fixminer_source/python
|
||||
2020-04-07 17:31:35,224 - 31200 - INFO - commons.py:setEnv - REPO_PATH : /Users/anilkoyuncu/projects/test/richedit-data/datasets
|
||||
2020-04-07 17:31:35,224 - 31200 - INFO - commons.py:setEnv - CODE_PATH : /Users/anilkoyuncu/projects/release/test/fixminer_source/python/code/
|
||||
2020-04-07 17:31:35,224 - 31200 - INFO - commons.py:setEnv - COMMIT_DFS : /Users/anilkoyuncu/projects/test/richedit-data/commitsDF/
|
||||
2020-04-07 17:31:35,224 - 31200 - INFO - commons.py:setEnv - BUG_POINT : /Users/anilkoyuncu/projects/test/richedit-data/bugPoints/
|
||||
2020-04-07 17:31:35,224 - 31200 - INFO - commons.py:setEnv - COMMIT_FOLDER : /Users/anilkoyuncu/projects/test/richedit-data/commits/
|
||||
2020-04-07 17:31:35,225 - 31200 - INFO - commons.py:setEnv - FEATURE_DIR : /Users/anilkoyuncu/projects/test/richedit-data/features/
|
||||
2020-04-07 17:31:35,225 - 31200 - INFO - commons.py:setEnv - CLASSIFIER_DIR : /Users/anilkoyuncu/projects/test/richedit-data/classifiers/
|
||||
2020-04-07 17:31:35,225 - 31200 - INFO - commons.py:setEnv - PREDICTION_DIR : /Users/anilkoyuncu/projects/test/richedit-data/predictions/
|
||||
2020-04-07 17:31:35,225 - 31200 - INFO - commons.py:setEnv - DATASET_DIR : /Users/anilkoyuncu/projects/test/richedit-data/datasets/
|
||||
2020-04-07 17:31:41,346 - 31200 - INFO - commons.py:shellCallTemplate - JAVA_HOME='/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home' java -jar /Users/anilkoyuncu/projects/release/test/fixminer_source/target/FixPatternMiner-1.0.0-jar-with-dependencies.jar /Users/anilkoyuncu/projects/release/test/fixminer_source/src/main/resources/config.yml RICHEDITSCRIPT
|
||||
2020-04-07 17:31:41,551 - 31200 - ERROR - commons.py:shellCallTemplate - Error: Could not find or load main class edu.lu.uni.serval.richedit.Launcher
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -26,6 +26,8 @@ from collections import Counter
|
||||
|
||||
import datetime
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
|
||||
sourceCodeColumns = ['packageName', 'className', 'methodNames', 'formalParameter',
|
||||
@@ -72,13 +74,15 @@ def setEnv(args):
|
||||
|
||||
|
||||
import yaml
|
||||
if os.uname().nodename != '':
|
||||
with open(join(os.environ["ROOT_DIR"], os.uname().nodename + ".config.yml"), 'r') as ymlfile:
|
||||
cfg = yaml.load(ymlfile)
|
||||
else:
|
||||
with open(join(os.environ["ROOT_DIR"], "config.yml"), 'r') as ymlfile:
|
||||
cfg = yaml.load(ymlfile)
|
||||
# if os.uname().nodename != '':
|
||||
# with open(join(os.environ["ROOT_DIR"], os.uname().nodename + ".config.yml"), 'r') as ymlfile:
|
||||
# cfg = yaml.load(ymlfile)
|
||||
# else:
|
||||
# with open(join(os.environ["ROOT_DIR"], "config.yml"), 'r') as ymlfile:
|
||||
# cfg = yaml.load(ymlfile)
|
||||
|
||||
with open(args.prop, 'r') as ymlfile:
|
||||
cfg = yaml.load(ymlfile)
|
||||
# for section in cfg:
|
||||
# print(section)
|
||||
# print(cfg['mysql'])
|
||||
@@ -88,10 +92,11 @@ def setEnv(args):
|
||||
os.environ["JDK8"] = cfg['java']['8home']
|
||||
os.environ["spinfer"] = cfg['spinfer']['home']
|
||||
os.environ["coccinelle"] = cfg['coccinelle']['home']
|
||||
os.environ["dataset"] = cfg['dataset']['home']
|
||||
os.environ["dataset"] = cfg['dataset']['inputPath']
|
||||
os.environ["REPO_PATH"] = cfg['dataset']['repo']
|
||||
os.environ["DATA_PATH"] = cfg['fixminer']['datapath']
|
||||
os.environ["PROJECT_TYPE"] = cfg['fixminer']['projectType']
|
||||
os.environ["PROJECT_LIST"] = cfg['fixminer']['projectList']
|
||||
|
||||
# import yaml
|
||||
#
|
||||
@@ -160,11 +165,12 @@ def getRun():
|
||||
# parser.add_argument('-subject', dest='subject', help='Environment')
|
||||
parser.add_argument('-root', dest='root', help='root folder')
|
||||
parser.add_argument('-job',dest='job',help='job name')
|
||||
parser.add_argument('-prop',dest='prop',help='property file')
|
||||
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.root is None or args.job is None:
|
||||
if args.root is None or args.job is None or args.prop is None:
|
||||
parser.print_help()
|
||||
raise AttributeError
|
||||
return args
|
||||
|
||||
+6
-6
@@ -1,17 +1,17 @@
|
||||
java:
|
||||
8home: /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home
|
||||
spinfer:
|
||||
home: /Users/anil.koyuncu/projects/fixminer/spinfer/spinfer.native
|
||||
home: /Users/anil.koyuncu/projects/richedit/spinfer/spinfer.native
|
||||
#home: /Users/anil.koyuncu/projects/spinfer/spinfer.native
|
||||
dataset:
|
||||
# home: /Users/anil.koyuncu/projects/test/fixminer-core/python/data/gumInputLinux/
|
||||
home: /data/fixminer-core/python/data/gumInputLinux/
|
||||
# home: /Users/anil.koyuncu/projects/test/richedit-core/python/data/gumInputLinux/
|
||||
home: /data/richedit-core/python/data/gumInputLinux/
|
||||
|
||||
coccinelle:
|
||||
home: /Users/anil.koyuncu/projects/fixminer/spinfer/statics
|
||||
home: /Users/anil.koyuncu/projects/richedit/spinfer/statics
|
||||
#home: /Users/anil.koyuncu/projects/spinfer/statics
|
||||
|
||||
|
||||
fixminer:
|
||||
# datapath: /Users/anil.koyuncu/projects/test/fixminer-core/python/data/
|
||||
datapath: /data/fixminer-core/python/data/
|
||||
# datapath: /Users/anil.koyuncu/projects/test/richedit-core/python/data/
|
||||
datapath: /data/richedit-core/python/data/
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,46 @@
|
||||
Group,Subject,Repo,GitRepo,Branch
|
||||
Apache,CAMEL,camel,https://github.com/apache/camel.git,master
|
||||
Apache,HBASE,hbase,https://github.com/apache/hbase.git,master
|
||||
Apache,HIVE,hive,https://github.com/apache/hive.git,master
|
||||
Commons,CODEC,commons-codec,https://github.com/apache/commons-codec.git,trunk
|
||||
Commons,COLLECTIONS,commons-collections,https://github.com/apache/commons-collections.git,master
|
||||
Commons,COMPRESS,commons-compress,https://github.com/apache/commons-compress.git,master
|
||||
Commons,CONFIGURATION,commons-configuration,https://github.com/apache/commons-configuration.git,trunk
|
||||
Commons,CRYPTO,commons-crypto,https://github.com/apache/commons-crypto.git,master
|
||||
Commons,CSV,commons-csv,https://github.com/apache/commons-csv.git,master
|
||||
Commons,IO,commons-io,https://github.com/apache/commons-io.git,master
|
||||
Commons,WEAVER,commons-weaver,https://github.com/apache/commons-weaver.git,master
|
||||
JBoss,ENTESB,fuse,https://github.com/jboss-fuse/fuse.git,6.3.0.redhat
|
||||
JBoss,JBMETA,metadata,https://github.com/jboss/metadata.git,master
|
||||
Wildfly,ELY,wildfly-elytron,https://github.com/wildfly-security/wildfly-elytron.git,master
|
||||
Wildfly,SWARM,wildfly-swarm,https://github.com/wildfly-swarm/wildfly-swarm.git,master
|
||||
Wildfly,WFARQ,wildfly-arquillian,https://github.com/wildfly/wildfly-arquillian.git,master
|
||||
Wildfly,WFCORE,wildfly-core,https://github.com/wildfly/wildfly-core.git,master
|
||||
Wildfly,WFLY,wildfly,https://github.com/wildfly/wildfly.git,master
|
||||
Wildfly,WFMP,wildfly-maven-plugin,https://github.com/wildfly/wildfly-maven-plugin.git,master
|
||||
Spring,AMQP,spring-amqp,https://github.com/spring-projects/spring-amqp,master
|
||||
Spring,ANDROID,spring-android,https://github.com/spring-projects/spring-android,master
|
||||
Spring,BATCH,spring-batch,https://github.com/spring-projects/spring-batch,master
|
||||
Spring,BATCHADM,spring-batch-admin,https://github.com/spring-projects/spring-batch-admin,master
|
||||
Spring,DATACMNS,spring-data-commons,https://github.com/spring-projects/spring-data-commons,master
|
||||
Spring,DATAGRAPH,spring-data-neo4j,https://github.com/spring-projects/spring-data-neo4j,master
|
||||
Spring,DATAJPA,spring-data-jpa,https://github.com/spring-projects/spring-data-jpa,master
|
||||
Spring,DATAMONGO,spring-data-mongodb,https://github.com/spring-projects/spring-data-mongodb,master
|
||||
Spring,DATAREDIS,spring-data-redis,https://github.com/spring-projects/spring-data-redis,master
|
||||
Spring,DATAREST,spring-data-rest,https://github.com/spring-projects/spring-data-rest,master
|
||||
Spring,LDAP,spring-ldap,https://github.com/spring-projects/spring-ldap,master
|
||||
Spring,MOBILE,spring-mobile,https://github.com/spring-projects/spring-mobile,master
|
||||
Spring,ROO,spring-roo,https://github.com/spring-projects/spring-roo,master
|
||||
Spring,SEC,spring-security,https://github.com/spring-projects/spring-security,master
|
||||
Spring,SECOAUTH,spring-security-oauth,https://github.com/spring-projects/spring-security-oauth,master
|
||||
Spring,SGF,spring-data-gemfire,https://github.com/spring-projects/spring-data-gemfire,master
|
||||
Spring,SHDP,spring-hadoop,https://github.com/spring-projects/spring-hadoop,master
|
||||
Spring,SHL,spring-shell,https://github.com/spring-projects/spring-shell,master
|
||||
Spring,SOCIAL,spring-social,https://github.com/spring-projects/spring-social,master
|
||||
Spring,SOCIALFB,spring-social-facebook,https://github.com/spring-projects/spring-social-facebook,master
|
||||
Spring,SOCIALLI,spring-social-linkedin,https://github.com/spring-projects/spring-social-linkedin,master
|
||||
Spring,SOCIALTW,spring-social-twitter,https://github.com/spring-projects/spring-social-twitter,master
|
||||
Spring,SPR,spring-framework,https://github.com/spring-projects/spring-framework,master
|
||||
Spring,SWF,spring-webflow,https://github.com/spring-projects/spring-webflow,master
|
||||
Spring,SWS,spring-ws,https://github.com/spring-projects/spring-ws,master
|
||||
|
||||
|
+2
-2
@@ -1,6 +1,6 @@
|
||||
#!/bin/bash
|
||||
|
||||
|
||||
source activate redisEnv
|
||||
source activate fixminerEnv
|
||||
|
||||
PYTHONPATH=$(pwd) python -u main.py -root $(pwd) -job $1
|
||||
PYTHONPATH=$(pwd) python -u main.py -root $(pwd) -job $1 -prop $2
|
||||
|
||||
+20
-21
@@ -15,7 +15,6 @@ if __name__ == '__main__':
|
||||
setEnv(args)
|
||||
|
||||
job = args.job
|
||||
job = 'indexClusters'
|
||||
ROOT_DIR = os.environ["ROOT_DIR"]
|
||||
REPO_PATH = os.environ["REPO_PATH"]
|
||||
CODE_PATH = os.environ["CODE_PATH"]
|
||||
@@ -36,24 +35,24 @@ if __name__ == '__main__':
|
||||
print(job)
|
||||
|
||||
|
||||
if job == 'dataset':
|
||||
if job == 'dataset4j':
|
||||
from javaDS import createDS
|
||||
createDS(subject)
|
||||
elif job == 'linuxDS':
|
||||
from linuxDataset import collectBugFixPatches
|
||||
collectBugFixPatches()
|
||||
elif job =='otherDS':
|
||||
# elif job == 'linuxDS':
|
||||
# from linuxDataset import collectBugFixPatches
|
||||
# collectBugFixPatches()
|
||||
elif job =='dataset4c':
|
||||
from otherDatasets import core
|
||||
core()
|
||||
elif job =='richEditScript':
|
||||
cmd = "JAVA_HOME='" + jdk8 + "' java -jar " + join(DATA_PATH, 'FixPatternMiner-1.0.1.jar') + " " + join(DATA_PATH, 'app.properties') + " RICHEDITSCRIPT " + 'L1'
|
||||
cmd = "JAVA_HOME='" + jdk8 + "' java -jar " + join(Path(ROOT_DIR).parent, 'target','FixPatternMiner-1.0.0-jar-with-dependencies.jar') + " " + args.prop + " RICHEDITSCRIPT "
|
||||
output = shellCallTemplate(cmd)
|
||||
logging.info(output)
|
||||
|
||||
elif job =='loadRES':
|
||||
cmd = "JAVA_HOME='" + jdk8 + "' java -jar " + join(DATA_PATH, 'FixPatternMiner-1.0.1.jar') + " " + join(DATA_PATH, 'app.properties') + " LOAD " + rootType
|
||||
output = shellCallTemplate(cmd)
|
||||
logging.info(output)
|
||||
# elif job =='loadRES':
|
||||
# cmd = "JAVA_HOME='" + jdk8 + "' java -jar " + join(DATA_PATH, 'FixPatternMiner-1.0.1.jar') + " " + join(DATA_PATH, 'app.properties') + " LOAD " + rootType
|
||||
# output = shellCallTemplate(cmd)
|
||||
# logging.info(output)
|
||||
|
||||
elif job =='shapeSI':
|
||||
from pairs import shapePairs
|
||||
@@ -66,14 +65,14 @@ if __name__ == '__main__':
|
||||
importShape()
|
||||
|
||||
elif job =='compareShapes':
|
||||
# cmd = "mvn exec:java -f '/data/fixminer_source/' -Dexec.mainClass='edu.lu.uni.serval.fixminer.akka.compare.CompareTrees' -Dexec.args='"+ " shape " + join(DATA_PATH,"redis") +" ALLdumps-gumInput.rdb " + "clusterl0-gumInputALL.rdb /data/fixminer-core/python/data/richEditScript'"
|
||||
cmd = "JAVA_HOME='" + jdk8 + "' java -jar " + join(DATA_PATH, 'FixPatternMiner-1.0.1.jar') + " " + join(DATA_PATH, 'app.properties') + " COMPARE " + 'L1'
|
||||
# cmd = "mvn exec:java -f '/data/fixminer_source/' -Dexec.mainClass='edu.lu.uni.serval.richedit.akka.compare.CompareTrees' -Dexec.args='"+ " shape " + join(DATA_PATH,"redis") +" ALLdumps-gumInput.rdb " + "clusterl0-gumInputALL.rdb /data/richedit-core/python/data/richEditScript'"
|
||||
cmd = "JAVA_HOME='" + jdk8 + "' java -jar " + join(Path(ROOT_DIR).parent, 'target','FixPatternMiner-1.0.0-jar-with-dependencies.jar') + " " + args.prop + " COMPARE "
|
||||
output = shellCallTemplate(cmd)
|
||||
logging.info(output)
|
||||
|
||||
elif job == 'clusterAdditional':
|
||||
from addNewData import cluster
|
||||
cluster()
|
||||
# elif job == 'clusterAdditional':
|
||||
# from addNewData import cluster
|
||||
# cluster()
|
||||
|
||||
elif job == 'cluster':
|
||||
from abstractPatch import cluster
|
||||
@@ -125,11 +124,11 @@ if __name__ == '__main__':
|
||||
cluster(join(DATA_PATH, 'tokens'), join(DATA_PATH, 'pairsToken'),'tokens')
|
||||
stopDB(dbDir, "6380", "clusterl2-gumInputALL.rdb")
|
||||
|
||||
elif job == 'additional':
|
||||
from addNewData import core
|
||||
core()
|
||||
# from addNewData import checkWrongMembers
|
||||
# checkWrongMembers()
|
||||
# elif job == 'additional':
|
||||
# from addNewData import core
|
||||
# core()
|
||||
# # from addNewData import checkWrongMembers
|
||||
# # checkWrongMembers()
|
||||
|
||||
elif job == 'codeflaws':
|
||||
from otherDatasets import codeflaws
|
||||
|
||||
@@ -1,20 +1,20 @@
|
||||
java:
|
||||
8home: /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home
|
||||
spinfer:
|
||||
home: /Users/anil.koyuncu/projects/fixminer/spinfer/spinfer.native
|
||||
home: /Users/anil.koyuncu/projects/richedit/spinfer/spinfer.native
|
||||
#home: /Users/anil.koyuncu/projects/spinfer/spinfer.native
|
||||
dataset:
|
||||
# home: /Users/anil.koyuncu/projects/test/fixminer-core/python/data/gumInputLinux/
|
||||
home: /Users/anil.koyuncu/projects/fixminer/fixminer-data/patches/
|
||||
# home: /data/fixminer-core/python/data/gumInputLinux/
|
||||
# home: /Users/anil.koyuncu/projects/test/richedit-core/python/data/gumInputLinux/
|
||||
home: /Users/anil.koyuncu/projects/richedit/richedit-data/patches/
|
||||
# home: /data/richedit-core/python/data/gumInputLinux/
|
||||
repo: /Users/anil.koyuncu/projects/datasets
|
||||
|
||||
coccinelle:
|
||||
home: /Users/anil.koyuncu/projects/fixminer/spinfer/statics
|
||||
home: /Users/anil.koyuncu/projects/richedit/spinfer/statics
|
||||
#home: /Users/anil.koyuncu/projects/spinfer/statics
|
||||
|
||||
|
||||
fixminer:
|
||||
# datapath: /Users/anil.koyuncu/projects/test/fixminer-core/python/data/
|
||||
datapath: /Users/anil.koyuncu/projects/fixminer/fixminer-data
|
||||
# datapath: /data/fixminer-core/python/data/
|
||||
# datapath: /Users/anil.koyuncu/projects/test/richedit-core/python/data/
|
||||
datapath: /Users/anil.koyuncu/projects/richedit/richedit-data
|
||||
# datapath: /data/richedit-core/python/data/
|
||||
|
||||
+14
-5
@@ -5,6 +5,7 @@ COMMIT_DFS = os.environ["COMMIT_DFS"]
|
||||
DATASET_PATH = os.environ["REPO_PATH"]
|
||||
DATASET = os.environ["dataset"]
|
||||
ROOT = os.environ["ROOT_DIR"]
|
||||
PROJECT_LIST = os.environ["PROJECT_LIST"]
|
||||
|
||||
def filetype_fileter(filename):
|
||||
# return filename.endswith(u'.java') and not bool(re.search('test.*\/', filename))
|
||||
@@ -266,12 +267,20 @@ def getAllCommits(datasetName):
|
||||
def core():
|
||||
datasets = pd.read_csv(join(ROOT,'data', 'datasets.csv'))
|
||||
# repoList = ['FFmpeg','curl','nginx','openssl','redis','tmux','vlc']
|
||||
pjs = listdir(DATASET_PATH)
|
||||
# newRepo = ['php-src','libtiff','cpython']
|
||||
pjs = [i for i in pjs if not (i.startswith('.') or i.startswith('codeflaws'))]
|
||||
# pjs = [i for i in pjs if (i in newRepo )]
|
||||
repoList = pjs
|
||||
|
||||
pjList = PROJECT_LIST.split(',')
|
||||
if not os.path.exists(DATASET_PATH):
|
||||
os.mkdir(DATASET_PATH)
|
||||
|
||||
for repo,src in datasets.values.tolist():
|
||||
if(pjList != ['ALL']):
|
||||
if repo in pjList:
|
||||
print(repo)
|
||||
cmd = 'git -C ' + DATASET_PATH + ' clone ' + src
|
||||
shellCallTemplate(cmd)
|
||||
logging.info(repo)
|
||||
collectBugFixPatches(repo)
|
||||
else:
|
||||
cmd = 'git -C ' + DATASET_PATH + ' clone ' + src
|
||||
shellCallTemplate(cmd)
|
||||
logging.info(repo)
|
||||
|
||||
@@ -1,21 +1,21 @@
|
||||
java:
|
||||
8home: /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home
|
||||
spinfer:
|
||||
home: /Users/anilkoyuncu/projects/fixminer/spinfer/spinfer.native
|
||||
home: /Users/anilkoyuncu/projects/richedit/spinfer/spinfer.native
|
||||
#home: /Users/anil.koyuncu/projects/spinfer/spinfer.native
|
||||
dataset:
|
||||
# home: /Users/anil.koyuncu/projects/test/fixminer-core/python/data/gumInputLinux/
|
||||
home: /Users/anilkoyuncu/projects/fixminer/fixminer-core/python/data/gumInputLinux
|
||||
# home: /data/fixminer-core/python/data/gumInputLinux/
|
||||
repo: /Users/anil.koyuncu/projects/datasets
|
||||
# home: /Users/anil.koyuncu/projects/test/richedit-core/python/data/gumInputLinux/
|
||||
home: /Users/anilkoyuncu/projects/richedit/richedit-core/python/data/gumInputLinux
|
||||
# home: /data/richedit-core/python/data/gumInputLinux/
|
||||
repo: /Users/anilkoyuncu/projects/datasets
|
||||
|
||||
coccinelle:
|
||||
home: /Users/anilkoyuncu/projects/fixminer/spinfer/statics
|
||||
home: /Users/anilkoyuncu/projects/richedit/spinfer/statics
|
||||
#home: /Users/anil.koyuncu/projects/spinfer/statics
|
||||
|
||||
|
||||
fixminer:
|
||||
# datapath: /Users/anil.koyuncu/projects/test/fixminer-core/python/data/
|
||||
datapath: /Users/anilkoyuncu/projects/fixminer/fixminer-data/
|
||||
# datapath: /Users/anil.koyuncu/projects/test/richedit-core/python/data/
|
||||
datapath: /Users/anilkoyuncu/projects/richedit/richedit-data/
|
||||
projectType : c
|
||||
# datapath: /data/fixminer-core/python/data/
|
||||
# datapath: /data/richedit-core/python/data/
|
||||
|
||||
+34
-8
@@ -5,7 +5,13 @@
|
||||
<groupId>edu.lu.uni.serval</groupId>
|
||||
<artifactId>RichEdit</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<packaging>jar</packaging>
|
||||
|
||||
<!-- <parent>-->
|
||||
<!-- <groupId>edu.lu.uni.serval</groupId>-->
|
||||
<!-- <artifactId>FixPatternMiner</artifactId>-->
|
||||
<!-- <version>1.0.0</version>-->
|
||||
<!-- </parent>-->
|
||||
<!-- <packaging>pom</packaging>-->
|
||||
|
||||
<name>RichEdit</name>
|
||||
<url>http://maven.apache.org</url>
|
||||
@@ -46,21 +52,21 @@
|
||||
<dependency>
|
||||
<groupId>com.github.gumtreediff</groupId>
|
||||
<artifactId>core</artifactId>
|
||||
<scope>compile</scope>
|
||||
<!-- <scope>compile</scope>-->
|
||||
<version>${gumtree.version}</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>com.github.gumtreediff</groupId>
|
||||
<artifactId>gen.jdt</artifactId>
|
||||
<scope>compile</scope>
|
||||
<!-- <scope>compile</scope>-->
|
||||
<version>${gumtree.version}</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>com.github.gumtreediff</groupId>
|
||||
<artifactId>gen.srcml</artifactId>
|
||||
<scope>compile</scope>
|
||||
<!-- <scope>compile</scope>-->
|
||||
<version>${gumtree.version}</version>
|
||||
</dependency>
|
||||
|
||||
@@ -110,6 +116,11 @@
|
||||
<version>${commons-io.version}</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.yaml</groupId>
|
||||
<artifactId>snakeyaml</artifactId>
|
||||
<version>1.21</version>
|
||||
</dependency>
|
||||
|
||||
|
||||
|
||||
@@ -127,6 +138,19 @@
|
||||
<target>1.8</target>
|
||||
</configuration>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-source-plugin</artifactId>
|
||||
<version>2.2.1</version>
|
||||
<executions>
|
||||
<execution>
|
||||
<id>attach-sources</id>
|
||||
<goals>
|
||||
<goal>jar-no-fork</goal>
|
||||
</goals>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
@@ -141,8 +165,8 @@
|
||||
<archive>
|
||||
<manifest>
|
||||
<mainClass>
|
||||
<!--edu.lu.uni.serval.fixminer.jobs.CompareTrees-->
|
||||
edu.lu.uni.serval.fixminer.Launcher
|
||||
<!--edu.lu.uni.serval.richedit.jobs.CompareTrees-->
|
||||
edu.lu.uni.serval.Launcher
|
||||
</mainClass>
|
||||
</manifest>
|
||||
</archive>
|
||||
@@ -155,9 +179,11 @@
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
|
||||
|
||||
</project>
|
||||
|
||||
<!--edu.lu.uni.serval.fixminer.Launcher
|
||||
edu.lu.uni.serval.fixminer.jobs.CompareTrees
|
||||
<!--edu.lu.uni.serval.richedit.Launcher
|
||||
edu.lu.uni.serval.richedit.jobs.CompareTrees
|
||||
edu.lu.uni.serval.utils.ClusterToPattern
|
||||
-->
|
||||
|
||||
+33
-18
@@ -1,16 +1,16 @@
|
||||
package edu.lu.uni.serval.fixminer;
|
||||
package edu.lu.uni.serval;
|
||||
|
||||
|
||||
import edu.lu.uni.serval.fixminer.jobs.CompareTrees;
|
||||
import edu.lu.uni.serval.fixminer.jobs.EnhancedASTDiff;
|
||||
import edu.lu.uni.serval.richedit.jobs.CompareTrees;
|
||||
import edu.lu.uni.serval.richedit.jobs.EnhancedASTDiff;
|
||||
import edu.lu.uni.serval.utils.ClusterToPattern;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.yaml.snakeyaml.Yaml;
|
||||
|
||||
import java.io.FileInputStream;
|
||||
import java.io.IOException;
|
||||
import java.net.InetAddress;
|
||||
import java.net.UnknownHostException;
|
||||
import java.util.Map;
|
||||
import java.util.Properties;
|
||||
|
||||
/**
|
||||
@@ -43,26 +43,40 @@ public class Launcher {
|
||||
// else{
|
||||
// appConfigPath = "src/main/resource/"+hostname.split("\\.")[0]+".app.properties";
|
||||
// }
|
||||
if(args.length != 2)
|
||||
{
|
||||
System.out.println("Proper Usage is: \n\tfirst argument full path to .properties file (e.g. an example is located under resources) \n\tsecond argument jobType (e.g RICHEDITSCRIPT, COMPARE)");
|
||||
System.exit(0);
|
||||
}
|
||||
|
||||
String appConfigPath = args[0];
|
||||
|
||||
Yaml yaml = new Yaml();
|
||||
// InputStream inputStream = this.getClass()
|
||||
// .getClassLoader()
|
||||
// .getResourceAsStream("customer.yaml");
|
||||
Map<String, Object> obj = yaml.load(new FileInputStream(appConfigPath));
|
||||
|
||||
appProps.load(new FileInputStream(appConfigPath));
|
||||
Map<String, Object> fixminer = (Map<String, Object>) obj.get("fixminer");
|
||||
// String numOfWorkers = appProps.getProperty("numOfWorkers", "10");
|
||||
String numOfWorkers = String.valueOf(fixminer.get("numOfWorkers"));
|
||||
String portDumps = String.valueOf(fixminer.get("portDumps"));
|
||||
String projectType = (String) fixminer.get("projectType");
|
||||
|
||||
String numOfWorkers = appProps.getProperty("numOfWorkers", "10");
|
||||
String portDumps = appProps.getProperty("portDumps","6399");
|
||||
String projectType = appProps.getProperty("projectType","java");
|
||||
|
||||
String hunkLimit = appProps.getProperty("hunkLimit","10");
|
||||
String patchSize = appProps.getProperty("patchSize","50");
|
||||
String projectL = appProps.getProperty("projectList","");
|
||||
String hunkLimit = String.valueOf(fixminer.get("hunkLimit"));
|
||||
String patchSize = String.valueOf(fixminer.get("patchSize"));
|
||||
String projectL = (String) fixminer.get("projectList");
|
||||
String[] projectList = projectL.split(",");
|
||||
String input = appProps.getProperty("inputPath","FORKJOIN");
|
||||
String redisPath = appProps.getProperty("redisPath","FORKJOIN");
|
||||
String srcMLPath = appProps.getProperty("srcMLPath","FORKJOIN");
|
||||
String input = (String) fixminer.get("inputPath");
|
||||
String redisPath = (String) fixminer.get("redisPath");
|
||||
String srcMLPath = (String) fixminer.get("srcMLPath");
|
||||
|
||||
// String parameter = args[2];
|
||||
String parameter = "L1";
|
||||
// String jobType = args[1];
|
||||
String jobType = args[1];
|
||||
// String jobType = "RICHEDITSCRIPT";
|
||||
String jobType = "COMPARE";
|
||||
// String jobType = "COMPARE";
|
||||
|
||||
|
||||
mainLaunch( numOfWorkers, jobType, portDumps,projectType,input,redisPath,parameter, srcMLPath,hunkLimit,projectList,patchSize);
|
||||
@@ -121,7 +135,8 @@ public class Launcher {
|
||||
|
||||
}
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
log.error(e.getMessage());
|
||||
// e.printStackTrace();
|
||||
|
||||
}
|
||||
|
||||
+5
-5
@@ -1,7 +1,7 @@
|
||||
package edu.lu.uni.serval.fixminer;
|
||||
package edu.lu.uni.serval.richedit;
|
||||
|
||||
import edu.lu.uni.serval.fixminer.jobs.CompareTrees;
|
||||
import edu.lu.uni.serval.fixminer.jobs.EnhancedASTDiff;
|
||||
import edu.lu.uni.serval.richedit.jobs.CompareTrees;
|
||||
import edu.lu.uni.serval.richedit.jobs.EnhancedASTDiff;
|
||||
import edu.lu.uni.serval.utils.ClusterToPattern;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
@@ -15,9 +15,9 @@ import java.util.Properties;
|
||||
/**
|
||||
* Created by anilkoyuncu on 14/04/2018.
|
||||
*/
|
||||
public class Launcher {
|
||||
public class Launcher2 {
|
||||
|
||||
private static Logger log = LoggerFactory.getLogger(Launcher.class);
|
||||
private static Logger log = LoggerFactory.getLogger(Launcher2.class);
|
||||
|
||||
public static void main(String[] args) throws IOException {
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
package edu.lu.uni.serval.fixminer.ediff;
|
||||
package edu.lu.uni.serval.richedit.ediff;
|
||||
|
||||
/**
|
||||
* Created by anilkoyuncu on 18/09/2018.
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
package edu.lu.uni.serval.fixminer.ediff;
|
||||
package edu.lu.uni.serval.richedit.ediff;
|
||||
|
||||
import akka.actor.ActorRef;
|
||||
import akka.actor.Props;
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
package edu.lu.uni.serval.fixminer.ediff;
|
||||
package edu.lu.uni.serval.richedit.ediff;
|
||||
|
||||
import com.github.gumtreediff.tree.ITree;
|
||||
import edu.lu.uni.serval.utils.EDiffHelper;
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
package edu.lu.uni.serval.fixminer.ediff;
|
||||
package edu.lu.uni.serval.richedit.ediff;
|
||||
|
||||
import redis.clients.jedis.JedisPool;
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
package edu.lu.uni.serval.fixminer.ediff;
|
||||
package edu.lu.uni.serval.richedit.ediff;
|
||||
|
||||
import com.github.gumtreediff.actions.model.Action;
|
||||
import com.github.gumtreediff.gen.srcml.GumTreeCComparer;
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
package edu.lu.uni.serval.fixminer.ediff;
|
||||
package edu.lu.uni.serval.richedit.ediff;
|
||||
|
||||
import akka.actor.Props;
|
||||
import akka.actor.UntypedActor;
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
package edu.lu.uni.serval.fixminer.ediff;
|
||||
package edu.lu.uni.serval.richedit.ediff;
|
||||
|
||||
import com.github.gumtreediff.actions.model.Action;
|
||||
import com.github.gumtreediff.tree.ITree;
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
package edu.lu.uni.serval.fixminer.ediff;
|
||||
package edu.lu.uni.serval.richedit.ediff;
|
||||
|
||||
import com.github.gumtreediff.actions.model.*;
|
||||
import com.github.gumtreediff.tree.ITree;
|
||||
+1
-2
@@ -1,7 +1,6 @@
|
||||
package edu.lu.uni.serval.fixminer.ediff;
|
||||
package edu.lu.uni.serval.richedit.ediff;
|
||||
|
||||
|
||||
import com.github.gumtreediff.actions.model.Action;
|
||||
import com.github.gumtreediff.actions.model.*;
|
||||
import com.github.gumtreediff.gen.srcml.NodeMap_new;
|
||||
import com.github.gumtreediff.tree.ITree;
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
package edu.lu.uni.serval.fixminer.ediff;
|
||||
package edu.lu.uni.serval.richedit.ediff;
|
||||
|
||||
import java.io.File;
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
package edu.lu.uni.serval.fixminer.ediff;
|
||||
package edu.lu.uni.serval.richedit.ediff;
|
||||
|
||||
import redis.clients.jedis.JedisPool;
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
package edu.lu.uni.serval.fixminer.ediff;
|
||||
package edu.lu.uni.serval.richedit.ediff;
|
||||
|
||||
public interface ParserInterface {
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
package edu.lu.uni.serval.fixminer.ediff;
|
||||
package edu.lu.uni.serval.richedit.ediff;
|
||||
|
||||
import redis.clients.jedis.JedisPool;
|
||||
|
||||
+2
-3
@@ -1,6 +1,5 @@
|
||||
package edu.lu.uni.serval.fixminer.jobs;
|
||||
package edu.lu.uni.serval.richedit.jobs;
|
||||
|
||||
import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
|
||||
import edu.lu.uni.serval.utils.CallShell;
|
||||
import edu.lu.uni.serval.utils.EDiffHelper;
|
||||
import edu.lu.uni.serval.utils.PoolBuilder;
|
||||
@@ -31,7 +30,7 @@ public class CompareTrees {
|
||||
|
||||
public static void main(String redisPath, String portDumps, String dumpsName, String job,String numOfWorkers) throws Exception {
|
||||
|
||||
// shape /Users/anil.koyuncu/projects/test/fixminer-core/python/data/redis ALLdumps-gumInput.rdb clusterl0-gumInputALL.rdb /Users/anil.koyuncu/projects/test/fixminer-core/python/data/richEditScript
|
||||
// shape /Users/anil.koyuncu/projects/test/richedit-core/python/data/redis ALLdumps-gumInput.rdb clusterl0-gumInputALL.rdb /Users/anil.koyuncu/projects/test/richedit-core/python/data/richEditScript
|
||||
|
||||
// String portInner = "6380";
|
||||
String port = portDumps; //"6399";
|
||||
+6
-3
@@ -1,8 +1,8 @@
|
||||
package edu.lu.uni.serval.fixminer.jobs;
|
||||
package edu.lu.uni.serval.richedit.jobs;
|
||||
|
||||
import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
|
||||
import edu.lu.uni.serval.richedit.ediff.EDiffHunkParser;
|
||||
import edu.lu.uni.serval.utils.FileHelper;
|
||||
import edu.lu.uni.serval.fixminer.ediff.MessageFile;
|
||||
import edu.lu.uni.serval.richedit.ediff.MessageFile;
|
||||
import edu.lu.uni.serval.utils.CallShell;
|
||||
import edu.lu.uni.serval.utils.PoolBuilder;
|
||||
import me.tongfei.progressbar.ProgressBar;
|
||||
@@ -47,6 +47,9 @@ public class EnhancedASTDiff {
|
||||
}
|
||||
File folder = new File(inputPath);
|
||||
File[] listOfFiles = folder.listFiles();
|
||||
if(listOfFiles == null){
|
||||
throw new Exception("No projects found, please verify the projects in the input path");
|
||||
}
|
||||
Stream<File> stream = Arrays.stream(listOfFiles);
|
||||
List<File> folders;
|
||||
if (projectList.length == 1 && projectList[0].equals("")){
|
||||
@@ -5,7 +5,7 @@ import com.github.gumtreediff.gen.srcml.NodeMap_new;
|
||||
import com.github.gumtreediff.tree.ITree;
|
||||
import com.github.gumtreediff.tree.TreeContext;
|
||||
import com.github.gumtreediff.tree.TreeUtils;
|
||||
import edu.lu.uni.serval.fixminer.ediff.HierarchicalActionSet;
|
||||
import edu.lu.uni.serval.richedit.ediff.HierarchicalActionSet;
|
||||
import org.apache.commons.lang3.SerializationUtils;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
@@ -8,10 +8,10 @@ projectList =
|
||||
#projectList = libtiff,php-src,cpython,wireshark,gzip,gmp,lighttpd1.4,lighttpd2
|
||||
|
||||
#inputPath = /Users/anilkoyuncu/projects/gumInputLinux
|
||||
#inputPath = /Users/anil.koyuncu/projects/fixminer/fixminer-data/gumInputLinux
|
||||
#inputPath = /Users/anil.koyuncu/projects/richedit/richedit-data/gumInputLinux
|
||||
inputPath = /Users/anil.koyuncu/projects/test/fixminer-data/patches
|
||||
#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
|
||||
#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
|
||||
#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
|
||||
#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
|
||||
redisPath = /Users/anil.koyuncu/projects/test/fixminer-core/python/data/redis
|
||||
#srcMLPath= /Users/anil.koyuncu/Downloads/srcML.0.9.5/bin/srcml
|
||||
srcMLPath= /Users/anil.koyuncu/projects/test/srcML/bin/srcml
|
||||
|
||||
@@ -9,10 +9,10 @@ patchSize = 50
|
||||
projectList = libtiff,php-src,cpython,wireshark,gzip,gmp,lighttpd1.4,lighttpd2
|
||||
|
||||
#inputPath = /Users/anilkoyuncu/projects/gumInputLinux
|
||||
#inputPath = /Users/anil.koyuncu/projects/fixminer/fixminer-data/gumInputLinux
|
||||
#inputPath = /Users/anil.koyuncu/projects/richedit/richedit-data/gumInputLinux
|
||||
inputPath = /Users/anil.koyuncu/projects/test/fixminer-data/patches
|
||||
#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
|
||||
#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
|
||||
#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
|
||||
#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
|
||||
redisPath = /Users/anil.koyuncu/projects/test/fixminer-core/python/data/redis
|
||||
#srcMLPath= /Users/anil.koyuncu/Downloads/srcML.0.9.5/bin/srcml
|
||||
srcMLPath= /usr/local/bin/srcml
|
||||
|
||||
@@ -9,7 +9,7 @@ patchSize = 50
|
||||
projectList = codeflaws
|
||||
#inputPath = /Users/anilkoyuncu/projects/gumInputLinux
|
||||
inputPath = /Users/anilkoyuncu/projects/fixminer/fixminer-core/python/data/gumInputLinux
|
||||
#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
|
||||
#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
|
||||
redisPath = /Users/anilkoyuncu/projects/fixminer/fixminer-core/python/data/redis
|
||||
#srcMLPath= /Users/anil.koyuncu/Downloads/srcML/src2srcml
|
||||
srcMLPath= /usr/local/bin/srcml
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package edu.lu.uni.serval.FixPatternMiner;
|
||||
|
||||
import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
|
||||
import edu.lu.uni.serval.fixminer.ediff.HierarchicalActionSet;
|
||||
import edu.lu.uni.serval.richedit.ediff.EDiffHunkParser;
|
||||
import edu.lu.uni.serval.richedit.ediff.HierarchicalActionSet;
|
||||
|
||||
import java.io.File;
|
||||
import java.io.FileInputStream;
|
||||
@@ -33,7 +33,7 @@ public class BaseTest {
|
||||
List<HierarchicalActionSet> hierarchicalActionSets = parser.parseChangedSourceCodeWithGumTree2(prevFile, revFile, srcMLPath,false);
|
||||
return hierarchicalActionSets;
|
||||
}catch (NullPointerException n){
|
||||
// String cmd = "cp /Users/anil.koyuncu/projects/test/fixminer-data/patches/codeflaws/"+n.getMessage().split(root)[1] + " /Users/anil.koyuncu/projects/test/fixminerC/"+n.getMessage();
|
||||
// String cmd = "cp /Users/anil.koyuncu/projects/test/richedit-data/patches/codeflaws/"+n.getMessage().split(root)[1] + " /Users/anil.koyuncu/projects/test/fixminerC/"+n.getMessage();
|
||||
// CallShell cs = new CallShell();
|
||||
// try {
|
||||
// cs.runShell(cmd);
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
package edu.lu.uni.serval.FixPatternMiner;
|
||||
|
||||
import com.github.gumtreediff.tree.ITree;
|
||||
import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
|
||||
import edu.lu.uni.serval.fixminer.ediff.HierarchicalActionSet;
|
||||
import edu.lu.uni.serval.richedit.ediff.EDiffHunkParser;
|
||||
import edu.lu.uni.serval.richedit.ediff.HierarchicalActionSet;
|
||||
import edu.lu.uni.serval.utils.ClusterToPattern;
|
||||
import edu.lu.uni.serval.utils.EDiffHelper;
|
||||
import edu.lu.uni.serval.utils.PoolBuilder;
|
||||
@@ -16,16 +16,17 @@ import java.io.File;
|
||||
import java.io.IOException;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.util.*;
|
||||
import java.util.ArrayList;
|
||||
|
||||
@Ignore
|
||||
public class HunkParserTest extends BaseTest{
|
||||
|
||||
@Test
|
||||
public void testSimple() throws IOException {
|
||||
// String input = "/Users/anil.koyuncu/projects/test/fixminer-core/python/data/gumInputLinux/revFiles/7f52f3_3845d29_drivers#pci#host#pcie-altera.c";
|
||||
// String input = "/Users/anil.koyuncu/projects/test/richedit-core/python/data/gumInputLinux/revFiles/7f52f3_3845d29_drivers#pci#host#pcie-altera.c";
|
||||
|
||||
// String root = "//Users/anilkoyuncu/projects/gumInputLinux/";
|
||||
String root = "/Users/anilkoyuncu/projects/fixminer/fixminer-core/python/data/gumInputLinux/";
|
||||
String root = "/Users/anilkoyuncu/projects/richedit/richedit-core/python/data/gumInputLinux/";
|
||||
String filename ="";
|
||||
// filename ="freebsd_ceca9b8_b864ac4_sys#kern#sched_ule.c"; //too long
|
||||
// filename ="openbsd_e592ec_39c81a_sys#arch#i386#pci#pci_machdep.c"; //not parseable 56 "parameter_list" "" () ( (57 "parameter" "" () ( (22 "decl" "" () ())))
|
||||
@@ -91,7 +92,7 @@ public class HunkParserTest extends BaseTest{
|
||||
public void dumpFnction() throws Exception {
|
||||
String pattern = "function/20/gstreamer_0af74c_e8bae0_libs#gst#net#gstptpclock.c.txt_0";
|
||||
// String pattern = "function/20/FFmpeg_4c9d1c_3da860_libavutil#file_open.c.txt_0";
|
||||
ClusterToPattern.main("6399","/Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis","ALLdumps-gumInput.rdb ",pattern);
|
||||
ClusterToPattern.main("6399","/Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis","ALLdumps-gumInput.rdb ",pattern);
|
||||
}
|
||||
|
||||
@Test
|
||||
@@ -112,7 +113,7 @@ public class HunkParserTest extends BaseTest{
|
||||
public void testSimpleJava() throws IOException {
|
||||
|
||||
|
||||
String root = "/Users/anil.koyuncu/projects/test/fixminer-core/python/data/gumInput/spring-amqp/";
|
||||
String root = "/Users/anil.koyuncu/projects/test/richedit-core/python/data/gumInput/spring-amqp/";
|
||||
|
||||
String filename = "5d6e02_e597c5_spring-rabbit#src#main#java#org#springframework#amqp#rabbit#connection#ConnectionFactoryUtils.java";
|
||||
File revFile = new File(root +"/revFiles/"+filename);
|
||||
@@ -134,7 +135,7 @@ public class HunkParserTest extends BaseTest{
|
||||
final JedisPool outerPool = new JedisPool(PoolBuilder.getPoolConfig(), "localhost",Integer.valueOf("6399"),20000000);
|
||||
|
||||
EDiffHunkParser parser = new EDiffHunkParser();
|
||||
String root = "/Users/anil.koyuncu/projects/fixminer/gumInputLinux/linux/";
|
||||
String root = "/Users/anil.koyuncu/projects/richedit/gumInputLinux/linux/";
|
||||
String filename = "43f8987_f596c8_drivers#acpi#nfit#core.c";
|
||||
File revFile = new File(root + "revFiles/"+ filename);
|
||||
File prevFile =new File(root + "prevFiles/prev_"+filename);
|
||||
@@ -142,7 +143,7 @@ public class HunkParserTest extends BaseTest{
|
||||
String srcMLPath = "/Users/anil.koyuncu/Downloads/srcML/src2srcml";
|
||||
parser.parseFixPatterns(prevFile,revFile,diffFile,"gumInputLinux",outerPool,srcMLPath,"if",false);
|
||||
String key = "if/3/linux_bb67dd_0922c7_sound#soc#sof#intel#hda.c.txt_0";
|
||||
File file2load = new File("/Users/anil.koyuncu/projects/fixminer/dumps/"+ key);
|
||||
File file2load = new File("/Users/anil.koyuncu/projects/richedit/dumps/"+ key);
|
||||
byte[] dump = FileUtils.readFileToByteArray(file2load);
|
||||
|
||||
// String line = FileHelper.readFile(file2load);
|
||||
|
||||
@@ -1,9 +1,8 @@
|
||||
package edu.lu.uni.serval.FixPatternMiner;
|
||||
|
||||
import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
|
||||
import edu.lu.uni.serval.fixminer.ediff.HierarchicalActionSet;
|
||||
import edu.lu.uni.serval.richedit.ediff.EDiffHunkParser;
|
||||
import edu.lu.uni.serval.richedit.ediff.HierarchicalActionSet;
|
||||
import edu.lu.uni.serval.utils.CallShell;
|
||||
import edu.lu.uni.serval.utils.EDiffHelper;
|
||||
import org.junit.Assert;
|
||||
import org.junit.Ignore;
|
||||
import org.junit.Test;
|
||||
@@ -1236,7 +1235,7 @@ public class TestInputCases {
|
||||
List<HierarchicalActionSet> hierarchicalActionSets = parser.parseChangedSourceCodeWithGumTree2(prevFile, revFile, srcMLPath,false);
|
||||
return hierarchicalActionSets;
|
||||
}catch (NullPointerException n){
|
||||
String cmd = "cp /Users/anil.koyuncu/projects/test/fixminer-data/patches/"+project+"/"+n.getMessage().split(root)[1] + " /Users/anil.koyuncu/projects/test/fixminerC/"+n.getMessage();
|
||||
String cmd = "cp /Users/anil.koyuncu/projects/test/richedit-data/patches/"+project+"/"+n.getMessage().split(root)[1] + " /Users/anil.koyuncu/projects/test/fixminerC/"+n.getMessage();
|
||||
CallShell cs = new CallShell();
|
||||
try {
|
||||
cs.runShell(cmd);
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package edu.lu.uni.serval.FixPatternMiner;
|
||||
|
||||
import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
|
||||
import edu.lu.uni.serval.fixminer.ediff.HierarchicalActionSet;
|
||||
import edu.lu.uni.serval.richedit.ediff.EDiffHunkParser;
|
||||
import edu.lu.uni.serval.richedit.ediff.HierarchicalActionSet;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.junit.Assert;
|
||||
import org.junit.Ignore;
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
package edu.lu.uni.serval.FixPatternMiner;
|
||||
|
||||
|
||||
import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
|
||||
import edu.lu.uni.serval.fixminer.ediff.HierarchicalActionSet;
|
||||
import edu.lu.uni.serval.richedit.ediff.EDiffHunkParser;
|
||||
import edu.lu.uni.serval.richedit.ediff.HierarchicalActionSet;
|
||||
import org.junit.Assert;
|
||||
import org.junit.Ignore;
|
||||
import org.junit.Test;
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
package edu.lu.uni.serval.FixPatternMiner;
|
||||
|
||||
import com.github.gumtreediff.tree.ITree;
|
||||
import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
|
||||
import edu.lu.uni.serval.fixminer.ediff.HierarchicalActionSet;
|
||||
import edu.lu.uni.serval.richedit.ediff.HierarchicalActionSet;
|
||||
import edu.lu.uni.serval.utils.CallShell;
|
||||
import edu.lu.uni.serval.utils.EDiffHelper;
|
||||
import org.junit.Assert;
|
||||
|
||||
@@ -8,10 +8,10 @@ projectList =
|
||||
#projectList = libtiff,php-src,cpython,wireshark,gzip,gmp,lighttpd1.4,lighttpd2
|
||||
|
||||
#inputPath = /Users/anilkoyuncu/projects/gumInputLinux
|
||||
#inputPath = /Users/anil.koyuncu/projects/fixminer/fixminer-data/gumInputLinux
|
||||
#inputPath = /Users/anil.koyuncu/projects/richedit/richedit-data/gumInputLinux
|
||||
inputPath = /Users/anil.koyuncu/projects/test/fixminer-data/patches
|
||||
#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
|
||||
#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
|
||||
#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
|
||||
#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
|
||||
redisPath = /Users/anil.koyuncu/projects/test/fixminer-core/python/data/redis
|
||||
#srcMLPath= /Users/anil.koyuncu/Downloads/srcML.0.9.5/bin/srcml
|
||||
srcMLPath= /Users/anil.koyuncu/projects/test/srcML/bin/srcml
|
||||
|
||||
@@ -9,10 +9,10 @@ patchSize = 50
|
||||
projectList = libtiff,php-src,cpython,wireshark,gzip,gmp,lighttpd1.4,lighttpd2
|
||||
|
||||
#inputPath = /Users/anilkoyuncu/projects/gumInputLinux
|
||||
#inputPath = /Users/anil.koyuncu/projects/fixminer/fixminer-data/gumInputLinux
|
||||
#inputPath = /Users/anil.koyuncu/projects/richedit/richedit-data/gumInputLinux
|
||||
inputPath = /Users/anil.koyuncu/projects/test/fixminer-data/patches
|
||||
#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
|
||||
#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
|
||||
#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
|
||||
#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
|
||||
redisPath = /Users/anil.koyuncu/projects/test/fixminer-core/python/data/redis
|
||||
#srcMLPath= /Users/anil.koyuncu/Downloads/srcML.0.9.5/bin/srcml
|
||||
srcMLPath= /usr/local/bin/srcml
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
java:
|
||||
8home: /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home
|
||||
spinfer:
|
||||
home: /Users/anilkoyuncu/projects/fixminer/spinfer/spinfer.native
|
||||
coccinelle:
|
||||
home: /Users/anilkoyuncu/projects/fixminer/spinfer/statics
|
||||
|
||||
dataset:
|
||||
# home: /Users/anilkoyuncu/projects/fixminer/fixminer-core/python/data/gumInputLinux
|
||||
inputPath : /Users/anilkoyuncu/projects/test/fixminer-data/patches
|
||||
repo: /Users/anilkoyuncu/projects/test/fixminer-data/datasets
|
||||
|
||||
fixminer:
|
||||
projectType : c
|
||||
datapath: /Users/anilkoyuncu/projects/test/fixminer-data/
|
||||
|
||||
|
||||
pjName : patches
|
||||
portDumps : 6399
|
||||
numOfWorkers : 14
|
||||
hostname : localhost
|
||||
hunkLimit : 10
|
||||
patchSize : 50
|
||||
|
||||
projectList : libtiff,gzip,gmp,lighttpd1.4,lighttpd2
|
||||
inputPath : /Users/anilkoyuncu/projects/test/fixminer-data/patches
|
||||
redisPath : /Users/anilkoyuncu/projects/test/fixminer-core/python/data/redis
|
||||
srcMLPath : /usr/local/bin/srcml
|
||||
@@ -9,7 +9,7 @@ patchSize = 50
|
||||
projectList = codeflaws
|
||||
#inputPath = /Users/anilkoyuncu/projects/gumInputLinux
|
||||
inputPath = /Users/anilkoyuncu/projects/fixminer/fixminer-core/python/data/gumInputLinux
|
||||
#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
|
||||
#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
|
||||
redisPath = /Users/anilkoyuncu/projects/fixminer/fixminer-core/python/data/redis
|
||||
#srcMLPath= /Users/anil.koyuncu/Downloads/srcML/src2srcml
|
||||
srcMLPath= /usr/local/bin/srcml
|
||||
|
||||
Reference in New Issue
Block a user