changes for jar

2020-04-07 18:06:53 +02:00
parent c5463f91f8
commit 40a8b0c290
57 changed files with 1180 additions and 301 deletions
@@ -21,7 +21,7 @@
      <module name="fixminer_source" target="1.8" />
      <module name="gen.jdt" target="1.8" />
      <module name="gen.srcml" target="1.8" />
-      <module name="gumtree (1) (com.github.gumtreediff)" target="1.8" />
+      <module name="gumtree" target="1.8" />
      <module name="RichEdit" target="1.8" />
    </bytecodeTargetLevel>
  </component>
@@ -24,7 +24,7 @@ Fixminer is a systematic and automated approach to mine relevant and actionable
 ## II. Environment setup

 * OS: macOS Mojave (10.14.3)
-* JDK8: Oracle jdk1.8 (**important!**)
+* JDK8: (**important!**)
 * Download and configure Anaconda
 * Create an python environment using the [environment file](environment.yml)
  ```powershell
@@ -32,7 +32,12 @@ Fixminer is a systematic and automated approach to mine relevant and actionable
  ```
 * After creating the environment, activate it. It is containing necessary dependencies for redis, and python.
  ```powershell
-  source activate redisEnv
+  source activate fixminerEnv
+  ```
+
+* Update the config.yml file with the corresponding paths in your computer. An example config.yml file could be found under
+  ```powershell
+  fixminer_source/src/main/resources/config.yml
  ```
 <!---
 [fixminer.sh](python/fixminer.sh)
@@ -45,95 +50,52 @@ In order to launch FixMiner, execute [fixminer.sh](python/fixminer.sh)

    bash fixminer.sh /Users/..../enhancedASTDiff/python/ stats
 --->
-    
-## III. Replication Data
-Replication Data:
-    
-   [singleBR.pickle](python/data/singleBR.pickle)
-    
-    This pickle contains the list bug reports (i.e. bid) with the their corresponding fixes (i.e. commit) for each project in the dataset (i.e. project). 
-    
-   [bugReports.7z.00X](python/data/bugReports.7z.001)
-   
-    This is the dump of the bug reports archive extracted from each commit. These bug reports are not necessarily considered as BUG,CLOSED; this archive is the contins initial bug reports before identifying the fixes. 
-    
-   [gumInput.7z.001](python/data/gumInput.7z.001)
-   
-    This archive contains all the patches in our dataset, formatted in a way that can be processed by GumTree (i.e DiffEntries, prevFiles, revFiles)
-    
-   [ALLbugReportsComplete.pickle](python/data/ALLbugReportsComplete.pickle)
-   
-    The pickle object that represents the bug reports under the following columns 'bugReport', 'summary', 'description', 'created', 'updated', 'resolved', 'reporterDN', 'reporterEmail','hasAttachment', 'attachmentTime', 'hasPR', 'commentsCount'
-
-#### Data Viewer
-
-The data provided with replication package is listed in directory [python/data](python/data)
-The data is stored in different formats. (e.g. pickle, db, csv, etc..)
-
-The see content of the .pickle file the following script could be used.
-
-  ```python
-   import pickle as p
-   import gzip
-   def load_zipped_pickle(filename):
-      with gzip.open(filename, 'rb') as f:
-          loaded_object = p.load(f)
-          return loaded_object
-  ```
-Usage
-
-  ```python
-  result = load_zipped_pickle('code/LANGbugReportsComplete.pickle')
-  # Result is pandas object which can be exported to several formats
-  # Details on how to export is listed in offical library documentation
-  # https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html
-
-  ```
-

 ## IV. Step-by-Step execution

 #### Before running

-* Update [config file](python/config.yml) with corresponding user paths.
+* Update [config file](src/main/resources/config.yml) with corresponding user paths.

 * Active the conda environment from shell
  ```powershell
-  source activate redisEnv
+  source activate fixminerEnv
  ```

 In order to launch FixMiner, execute [fixminer.sh](python/fixminer.sh)

-    bash fixminer.sh [PATH_TO_PYTHON_FOLDER] [OPTIONS]
-     e.g. bash fixminer.sh Users/fixminer-core/python/ stats
-    
-#### Running Options 
+    bash fixminer.sh [JOB] [CONFIG_FILE]
+     e.g. bash fixminer.sh  dataset4c /Users/projects/release/fixminer_source/src/main/resources/config.yml

-*FixMiner* needs to specify an option to run.
+    
+#### Job Types  

-    1. 'dataset': Create a mining dataset from the project speficied in  [subjects.csv](python/data/subjects.csv)
-    Eventually dataset option is the execution of the following steps, which are merged under 'dataset' option 
-    for this demo. Eventually single options can be activated by commenting out the corresponding option in [main.py](python/main.py)
+*FixMiner* needs to specify a job to run.

-        `clone` : Clone target project repository.
-    
-        `collect` : Collect all commit from repository.
-    
-        `fix` : Collect commits linked to a bug report.
-    
-        `bugPoints` : Identify the snapshot of the repository before the bug fixing commit introducted.
-    
-        `brDownload` : Download bug reports recovered from commit log
-    
-        `brParser` : Parse bug reports to select the bug report where type labelled as BUG and status as RESOLVED or CLOSED
+   1. __dataset4j__ / __dataset4c__: Create a java/c mining dataset from the projects listed in [subjects.csv](python/data/subjects.csv) or [datasets.csv](python/data/datasets.csv) for c
+   
        
-    2. 'richEditScript': Rich edit script computation step.    
+   2. __richEditScript__: Calls the jar file produced as the results as maven package to compute Rich edit scripts.
+   This step can be invoke natively from java or using the [Launcher](src/main/java/edu/lu/uni/serval/richedit/Launcher.java) with appropriate arguments.
+   
+ ```powershell
+ java -jar FixPatternMiner-1.0.0-jar-with-dependencies.jar  /Users/projects/release/fixminer_source/src/main/resources/config.yml RICHEDITSCRIPT
+
+ ```   
+
+     
    
-    3. 'shapeSI': Search index creation for shapes. The output of this step is written to [pairs](python/data/pairs)
+   3. __shapeSI__: Search index creation for shapes. The output of this step is written to __pairs__ which will be generated under __datapath__ in [config file](src/main/resources/config.yml)
    
-    4. 'compareShapes' : ShapeTree comparison
+   4. __compareShapes__ : Calls the jar file produced as the results as maven package to compare the trees.
+                             This step can be invoke natively from java or using the [Launcher](src/main/java/edu/lu/uni/serval/richedit/Launcher.java) with appropriate arguments.
+                             
+                           ```powershell
+                           java -jar FixPatternMiner-1.0.0-jar-with-dependencies.jar  /Users/projects/release/fixminer_source/src/main/resources/config.yml COMPARETREES
+                          
+                           ```   
    
-    5. 'cluster': Forms clusters of identical shapetree. The output of this step is written to [shapes](python/data/shapes)
+   5. 'cluster': Forms clusters of identical shapetree. The output of this step is written to [shapes](python/data/shapes)
    
    6. 'actionSI': Search index creation for actions. The output of this step is written to [pairs](python/data/pairsAction)
    
@@ -233,6 +195,51 @@ It is necessary to run the FixMiner, following the order.
 --> 
    

+    
+## III. Replication Data
+Replication Data:
+    
+   [singleBR.pickle](python/data/singleBR.pickle)
+    
+    This pickle contains the list bug reports (i.e. bid) with the their corresponding fixes (i.e. commit) for each project in the dataset (i.e. project). 
+    
+   [bugReports.7z.00X](python/data/bugReports.7z.001)
+   
+    This is the dump of the bug reports archive extracted from each commit. These bug reports are not necessarily considered as BUG,CLOSED; this archive is the contins initial bug reports before identifying the fixes. 
+    
+   [gumInput.7z.001](python/data/gumInput.7z.001)
+   
+    This archive contains all the patches in our dataset, formatted in a way that can be processed by GumTree (i.e DiffEntries, prevFiles, revFiles)
+    
+   [ALLbugReportsComplete.pickle](python/data/ALLbugReportsComplete.pickle)
+   
+    The pickle object that represents the bug reports under the following columns 'bugReport', 'summary', 'description', 'created', 'updated', 'resolved', 'reporterDN', 'reporterEmail','hasAttachment', 'attachmentTime', 'hasPR', 'commentsCount'
+
+#### Data Viewer
+
+The data provided with replication package is listed in directory [python/data](python/data)
+The data is stored in different formats. (e.g. pickle, db, csv, etc..)
+
+The see content of the .pickle file the following script could be used.
+
+  ```python
+   import pickle as p
+   import gzip
+   def load_zipped_pickle(filename):
+      with gzip.open(filename, 'rb') as f:
+          loaded_object = p.load(f)
+          return loaded_object
+  ```
+Usage
+
+  ```python
+  result = load_zipped_pickle('code/LANGbugReportsComplete.pickle')
+  # Result is pandas object which can be exported to several formats
+  # Details on how to export is listed in offical library documentation
+  # https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html
+
+  ```
+



@@ -12,7 +12,12 @@
            <groupId>edu.lu.uni.serval</groupId>
            <artifactId>RichEdit</artifactId>
            <version>1.0.0</version>
-            <scope>compile</scope>
+<!--            <scope>compile</scope>-->
+        </dependency>
+        <dependency>
+            <groupId>org.yaml</groupId>
+            <artifactId>snakeyaml</artifactId>
+            <version>1.21</version>
        </dependency>
    </dependencies>
    <packaging>pom</packaging>
@@ -24,6 +29,7 @@


    <build>
+        <sourceDirectory>richedit/src/main/java/</sourceDirectory>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
@@ -47,10 +53,8 @@
                        <configuration>
                            <archive>
                                <manifest>
-                                    <mainClass>
-                                        <!--edu.lu.uni.serval.fixminer.jobs.CompareTrees-->
-                                        edu.lu.uni.serval.fixminer.Launcher
-                                    </mainClass>
+                                    <addClasspath>true</addClasspath>
+                                    <mainClass>edu.lu.uni.serval.Launcher</mainClass>
                                </manifest>
                            </archive>
                            <descriptorRefs>
@@ -3,14 +3,14 @@ java:
 spinfer:
    home:   /Users/anil.koyuncu/projects/spinfer/spinfer.native
 dataset:
-    home:   /Users/anil.koyuncu/projects/test/fixminer-data/patches/
-    repo:   /Users/anil.koyuncu/projects/test/fixminer-data/datasets
-#    home:   /data/fixminer-core/python/data/gumInputLinux/
+    home:   /Users/anil.koyuncu/projects/test/richedit-data/patches/
+    repo:   /Users/anil.koyuncu/projects/test/richedit-data/datasets
+#    home:   /data/richedit-core/python/data/gumInputLinux/

 coccinelle:
    home:   /Users/anil.koyuncu/projects/spinfer/statics

 fixminer:
-    datapath: /Users/anil.koyuncu/projects/test/fixminer-data/
+    datapath: /Users/anil.koyuncu/projects/test/richedit-data/
    projectType : c
-#    datapath: /data/fixminer-core/python/data/
+#    datapath: /data/richedit-core/python/data/
@@ -179,7 +179,7 @@ def clusterCore(clusterPath, level, match, pairsPath, root, s,action ,token=''):
    parallelRun(dumpFilesCore,workList)
    # for wl in workList:
    #     dumpFilesCore(wl)
-    # dumpFilesCore(('hive_d65d5c_96c1dc_ql#src#gen#protobuf#gen-java#org#apache#hadoop#hive#ql#io#orc#OrcProto.txt_31', 'ReturnStatement', 'tokens', '/Users/anil.koyuncu/projects/fixminer-all/enhancedASTDiff/python/data/tokens', '3', '1', '0', 10))
+    # dumpFilesCore(('hive_d65d5c_96c1dc_ql#src#gen#protobuf#gen-java#org#apache#hadoop#hive#ql#io#orc#OrcProto.txt_31', 'ReturnStatement', 'tokens', '/Users/anil.koyuncu/projects/richedit-all/enhancedASTDiff/python/data/tokens', '3', '1', '0', 10))


 def dumpFilesCore(t):
@@ -0,0 +1,13 @@
+2020-04-07 17:31:35,222 - 31200 - INFO - commons.py:setEnv - ROOT_DIR : /Users/anilkoyuncu/projects/release/test/fixminer_source/python
+2020-04-07 17:31:35,224 - 31200 - INFO - commons.py:setEnv - REPO_PATH : /Users/anilkoyuncu/projects/test/richedit-data/datasets
+2020-04-07 17:31:35,224 - 31200 - INFO - commons.py:setEnv - CODE_PATH : /Users/anilkoyuncu/projects/release/test/fixminer_source/python/code/
+2020-04-07 17:31:35,224 - 31200 - INFO - commons.py:setEnv - COMMIT_DFS : /Users/anilkoyuncu/projects/test/richedit-data/commitsDF/
+2020-04-07 17:31:35,224 - 31200 - INFO - commons.py:setEnv - BUG_POINT : /Users/anilkoyuncu/projects/test/richedit-data/bugPoints/
+2020-04-07 17:31:35,224 - 31200 - INFO - commons.py:setEnv - COMMIT_FOLDER : /Users/anilkoyuncu/projects/test/richedit-data/commits/
+2020-04-07 17:31:35,225 - 31200 - INFO - commons.py:setEnv - FEATURE_DIR : /Users/anilkoyuncu/projects/test/richedit-data/features/
+2020-04-07 17:31:35,225 - 31200 - INFO - commons.py:setEnv - CLASSIFIER_DIR : /Users/anilkoyuncu/projects/test/richedit-data/classifiers/
+2020-04-07 17:31:35,225 - 31200 - INFO - commons.py:setEnv - PREDICTION_DIR : /Users/anilkoyuncu/projects/test/richedit-data/predictions/
+2020-04-07 17:31:35,225 - 31200 - INFO - commons.py:setEnv - DATASET_DIR : /Users/anilkoyuncu/projects/test/richedit-data/datasets/
+2020-04-07 17:31:41,346 - 31200 - INFO - commons.py:shellCallTemplate - JAVA_HOME='/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home' java  -jar /Users/anilkoyuncu/projects/release/test/fixminer_source/target/FixPatternMiner-1.0.0-jar-with-dependencies.jar /Users/anilkoyuncu/projects/release/test/fixminer_source/src/main/resources/config.yml RICHEDITSCRIPT 
+2020-04-07 17:31:41,551 - 31200 - ERROR - commons.py:shellCallTemplate - Error: Could not find or load main class edu.lu.uni.serval.richedit.Launcher
+
@@ -26,6 +26,8 @@ from collections import Counter

 import datetime
 import subprocess
+from pathlib import Path
+


 sourceCodeColumns = ['packageName', 'className', 'methodNames', 'formalParameter',
@@ -72,13 +74,15 @@ def setEnv(args):


    import yaml
-    if os.uname().nodename != '':
-        with open(join(os.environ["ROOT_DIR"], os.uname().nodename + ".config.yml"), 'r') as ymlfile:
-            cfg = yaml.load(ymlfile)
-    else:
-        with open(join(os.environ["ROOT_DIR"], "config.yml"), 'r') as ymlfile:
-            cfg = yaml.load(ymlfile)
+    # if os.uname().nodename != '':
+    #     with open(join(os.environ["ROOT_DIR"], os.uname().nodename + ".config.yml"), 'r') as ymlfile:
+    #         cfg = yaml.load(ymlfile)
+    # else:
+    #     with open(join(os.environ["ROOT_DIR"], "config.yml"), 'r') as ymlfile:
+    #         cfg = yaml.load(ymlfile)

+    with open(args.prop, 'r') as ymlfile:
+        cfg = yaml.load(ymlfile)
    # for section in cfg:
    #     print(section)
    # print(cfg['mysql'])
@@ -88,10 +92,11 @@ def setEnv(args):
    os.environ["JDK8"] = cfg['java']['8home']
    os.environ["spinfer"] = cfg['spinfer']['home']
    os.environ["coccinelle"] = cfg['coccinelle']['home']
-    os.environ["dataset"] = cfg['dataset']['home']
+    os.environ["dataset"] = cfg['dataset']['inputPath']
    os.environ["REPO_PATH"] = cfg['dataset']['repo']
    os.environ["DATA_PATH"] = cfg['fixminer']['datapath']
    os.environ["PROJECT_TYPE"] = cfg['fixminer']['projectType']
+    os.environ["PROJECT_LIST"] = cfg['fixminer']['projectList']

    # import yaml
    #
@@ -160,11 +165,12 @@ def getRun():
    # parser.add_argument('-subject', dest='subject', help='Environment')
    parser.add_argument('-root', dest='root', help='root folder')
    parser.add_argument('-job',dest='job',help='job name')
+    parser.add_argument('-prop',dest='prop',help='property file')


    args = parser.parse_args()

-    if args.root is None or args.job is None:
+    if args.root is None or args.job is None or args.prop is None:
        parser.print_help()
        raise AttributeError
    return args
@@ -1,17 +1,17 @@
 java:
    8home:  /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home
 spinfer:
-    home:   /Users/anil.koyuncu/projects/fixminer/spinfer/spinfer.native
+    home:   /Users/anil.koyuncu/projects/richedit/spinfer/spinfer.native
    #home:   /Users/anil.koyuncu/projects/spinfer/spinfer.native
 dataset:
-#    home:   /Users/anil.koyuncu/projects/test/fixminer-core/python/data/gumInputLinux/
-    home:   /data/fixminer-core/python/data/gumInputLinux/
+#    home:   /Users/anil.koyuncu/projects/test/richedit-core/python/data/gumInputLinux/
+    home:   /data/richedit-core/python/data/gumInputLinux/

 coccinelle:
-    home:   /Users/anil.koyuncu/projects/fixminer/spinfer/statics
+    home:   /Users/anil.koyuncu/projects/richedit/spinfer/statics
    #home:   /Users/anil.koyuncu/projects/spinfer/statics


 fixminer:
-#    datapath: /Users/anil.koyuncu/projects/test/fixminer-core/python/data/
-    datapath: /data/fixminer-core/python/data/
+#    datapath: /Users/anil.koyuncu/projects/test/richedit-core/python/data/
+    datapath: /data/richedit-core/python/data/
@@ -0,0 +1,46 @@
+Group,Subject,Repo,GitRepo,Branch
+Apache,CAMEL,camel,https://github.com/apache/camel.git,master
+Apache,HBASE,hbase,https://github.com/apache/hbase.git,master
+Apache,HIVE,hive,https://github.com/apache/hive.git,master
+Commons,CODEC,commons-codec,https://github.com/apache/commons-codec.git,trunk
+Commons,COLLECTIONS,commons-collections,https://github.com/apache/commons-collections.git,master
+Commons,COMPRESS,commons-compress,https://github.com/apache/commons-compress.git,master
+Commons,CONFIGURATION,commons-configuration,https://github.com/apache/commons-configuration.git,trunk
+Commons,CRYPTO,commons-crypto,https://github.com/apache/commons-crypto.git,master
+Commons,CSV,commons-csv,https://github.com/apache/commons-csv.git,master
+Commons,IO,commons-io,https://github.com/apache/commons-io.git,master
+Commons,WEAVER,commons-weaver,https://github.com/apache/commons-weaver.git,master
+JBoss,ENTESB,fuse,https://github.com/jboss-fuse/fuse.git,6.3.0.redhat
+JBoss,JBMETA,metadata,https://github.com/jboss/metadata.git,master
+Wildfly,ELY,wildfly-elytron,https://github.com/wildfly-security/wildfly-elytron.git,master
+Wildfly,SWARM,wildfly-swarm,https://github.com/wildfly-swarm/wildfly-swarm.git,master
+Wildfly,WFARQ,wildfly-arquillian,https://github.com/wildfly/wildfly-arquillian.git,master
+Wildfly,WFCORE,wildfly-core,https://github.com/wildfly/wildfly-core.git,master
+Wildfly,WFLY,wildfly,https://github.com/wildfly/wildfly.git,master
+Wildfly,WFMP,wildfly-maven-plugin,https://github.com/wildfly/wildfly-maven-plugin.git,master
+Spring,AMQP,spring-amqp,https://github.com/spring-projects/spring-amqp,master
+Spring,ANDROID,spring-android,https://github.com/spring-projects/spring-android,master
+Spring,BATCH,spring-batch,https://github.com/spring-projects/spring-batch,master
+Spring,BATCHADM,spring-batch-admin,https://github.com/spring-projects/spring-batch-admin,master
+Spring,DATACMNS,spring-data-commons,https://github.com/spring-projects/spring-data-commons,master
+Spring,DATAGRAPH,spring-data-neo4j,https://github.com/spring-projects/spring-data-neo4j,master
+Spring,DATAJPA,spring-data-jpa,https://github.com/spring-projects/spring-data-jpa,master
+Spring,DATAMONGO,spring-data-mongodb,https://github.com/spring-projects/spring-data-mongodb,master
+Spring,DATAREDIS,spring-data-redis,https://github.com/spring-projects/spring-data-redis,master
+Spring,DATAREST,spring-data-rest,https://github.com/spring-projects/spring-data-rest,master
+Spring,LDAP,spring-ldap,https://github.com/spring-projects/spring-ldap,master
+Spring,MOBILE,spring-mobile,https://github.com/spring-projects/spring-mobile,master
+Spring,ROO,spring-roo,https://github.com/spring-projects/spring-roo,master
+Spring,SEC,spring-security,https://github.com/spring-projects/spring-security,master
+Spring,SECOAUTH,spring-security-oauth,https://github.com/spring-projects/spring-security-oauth,master
+Spring,SGF,spring-data-gemfire,https://github.com/spring-projects/spring-data-gemfire,master
+Spring,SHDP,spring-hadoop,https://github.com/spring-projects/spring-hadoop,master
+Spring,SHL,spring-shell,https://github.com/spring-projects/spring-shell,master
+Spring,SOCIAL,spring-social,https://github.com/spring-projects/spring-social,master
+Spring,SOCIALFB,spring-social-facebook,https://github.com/spring-projects/spring-social-facebook,master
+Spring,SOCIALLI,spring-social-linkedin,https://github.com/spring-projects/spring-social-linkedin,master
+Spring,SOCIALTW,spring-social-twitter,https://github.com/spring-projects/spring-social-twitter,master
+Spring,SPR,spring-framework,https://github.com/spring-projects/spring-framework,master
+Spring,SWF,spring-webflow,https://github.com/spring-projects/spring-webflow,master
+Spring,SWS,spring-ws,https://github.com/spring-projects/spring-ws,master
+
@@ -1,6 +1,6 @@
 #!/bin/bash


-source activate redisEnv
+source activate fixminerEnv

-PYTHONPATH=$(pwd) python -u main.py -root $(pwd) -job $1
+PYTHONPATH=$(pwd) python -u main.py -root $(pwd) -job $1 -prop $2
@@ -15,7 +15,6 @@ if __name__ == '__main__':
        setEnv(args)

        job = args.job
-        job = 'indexClusters'
        ROOT_DIR = os.environ["ROOT_DIR"]
        REPO_PATH = os.environ["REPO_PATH"]
        CODE_PATH = os.environ["CODE_PATH"]
@@ -36,24 +35,24 @@ if __name__ == '__main__':
        print(job)


-        if job == 'dataset':
+        if job == 'dataset4j':
            from javaDS import createDS
            createDS(subject)
-        elif job == 'linuxDS':
-            from linuxDataset import collectBugFixPatches
-            collectBugFixPatches()
-        elif job =='otherDS':
+        # elif job == 'linuxDS':
+        #     from linuxDataset import collectBugFixPatches
+        #     collectBugFixPatches()
+        elif job =='dataset4c':
            from otherDatasets import core
            core()
        elif job =='richEditScript':
-            cmd = "JAVA_HOME='" + jdk8 + "' java -jar " + join(DATA_PATH, 'FixPatternMiner-1.0.1.jar') + " " + join(DATA_PATH, 'app.properties') + " RICHEDITSCRIPT " + 'L1'
+            cmd = "JAVA_HOME='" + jdk8 + "' java  -jar " + join(Path(ROOT_DIR).parent, 'target','FixPatternMiner-1.0.0-jar-with-dependencies.jar') + " " + args.prop + " RICHEDITSCRIPT "
            output = shellCallTemplate(cmd)
            logging.info(output)

-        elif job =='loadRES':
-            cmd = "JAVA_HOME='" + jdk8 + "' java -jar " + join(DATA_PATH, 'FixPatternMiner-1.0.1.jar') + " " + join(DATA_PATH, 'app.properties') + " LOAD " + rootType
-            output = shellCallTemplate(cmd)
-            logging.info(output)
+        # elif job =='loadRES':
+        #     cmd = "JAVA_HOME='" + jdk8 + "' java -jar " + join(DATA_PATH, 'FixPatternMiner-1.0.1.jar') + " " + join(DATA_PATH, 'app.properties') + " LOAD " + rootType
+        #     output = shellCallTemplate(cmd)
+        #     logging.info(output)

        elif job =='shapeSI':
            from pairs import shapePairs
@@ -66,14 +65,14 @@ if __name__ == '__main__':
            importShape()

        elif job =='compareShapes':
-             # cmd = "mvn exec:java -f '/data/fixminer_source/' -Dexec.mainClass='edu.lu.uni.serval.fixminer.akka.compare.CompareTrees' -Dexec.args='"+ " shape " + join(DATA_PATH,"redis") +" ALLdumps-gumInput.rdb " + "clusterl0-gumInputALL.rdb /data/fixminer-core/python/data/richEditScript'"
-            cmd = "JAVA_HOME='" + jdk8 + "' java  -jar " + join(DATA_PATH, 'FixPatternMiner-1.0.1.jar') + " " + join(DATA_PATH, 'app.properties') + " COMPARE " + 'L1'
+             # cmd = "mvn exec:java -f '/data/fixminer_source/' -Dexec.mainClass='edu.lu.uni.serval.richedit.akka.compare.CompareTrees' -Dexec.args='"+ " shape " + join(DATA_PATH,"redis") +" ALLdumps-gumInput.rdb " + "clusterl0-gumInputALL.rdb /data/richedit-core/python/data/richEditScript'"
+            cmd = "JAVA_HOME='" + jdk8 + "' java  -jar " + join(Path(ROOT_DIR).parent, 'target','FixPatternMiner-1.0.0-jar-with-dependencies.jar') + " " + args.prop + " COMPARE "
            output = shellCallTemplate(cmd)
            logging.info(output)

-        elif job == 'clusterAdditional':
-            from addNewData import cluster
-            cluster()
+        # elif job == 'clusterAdditional':
+        #     from addNewData import cluster
+        #     cluster()

        elif job == 'cluster':
            from abstractPatch import cluster
@@ -125,11 +124,11 @@ if __name__ == '__main__':
            cluster(join(DATA_PATH, 'tokens'), join(DATA_PATH, 'pairsToken'),'tokens')
            stopDB(dbDir, "6380", "clusterl2-gumInputALL.rdb")

-        elif job == 'additional':
-            from addNewData import core
-            core()
-            # from addNewData import checkWrongMembers
-            # checkWrongMembers()
+        # elif job == 'additional':
+        #     from addNewData import core
+        #     core()
+        #     # from addNewData import checkWrongMembers
+        #     # checkWrongMembers()

        elif job == 'codeflaws':
            from otherDatasets import codeflaws
@@ -1,20 +1,20 @@
 java:
    8home:  /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home
 spinfer:
-    home:   /Users/anil.koyuncu/projects/fixminer/spinfer/spinfer.native
+    home:   /Users/anil.koyuncu/projects/richedit/spinfer/spinfer.native
    #home:   /Users/anil.koyuncu/projects/spinfer/spinfer.native
 dataset:
-#    home:   /Users/anil.koyuncu/projects/test/fixminer-core/python/data/gumInputLinux/
-    home:   /Users/anil.koyuncu/projects/fixminer/fixminer-data/patches/
-#    home:   /data/fixminer-core/python/data/gumInputLinux/
+#    home:   /Users/anil.koyuncu/projects/test/richedit-core/python/data/gumInputLinux/
+    home:   /Users/anil.koyuncu/projects/richedit/richedit-data/patches/
+#    home:   /data/richedit-core/python/data/gumInputLinux/
    repo:   /Users/anil.koyuncu/projects/datasets

 coccinelle:
-    home:   /Users/anil.koyuncu/projects/fixminer/spinfer/statics
+    home:   /Users/anil.koyuncu/projects/richedit/spinfer/statics
    #home:   /Users/anil.koyuncu/projects/spinfer/statics


 fixminer:
-#    datapath: /Users/anil.koyuncu/projects/test/fixminer-core/python/data/
-    datapath: /Users/anil.koyuncu/projects/fixminer/fixminer-data
-#    datapath: /data/fixminer-core/python/data/
+#    datapath: /Users/anil.koyuncu/projects/test/richedit-core/python/data/
+    datapath: /Users/anil.koyuncu/projects/richedit/richedit-data
+#    datapath: /data/richedit-core/python/data/
@@ -5,6 +5,7 @@ COMMIT_DFS = os.environ["COMMIT_DFS"]
 DATASET_PATH = os.environ["REPO_PATH"]
 DATASET = os.environ["dataset"]
 ROOT = os.environ["ROOT_DIR"]
+PROJECT_LIST = os.environ["PROJECT_LIST"]

 def filetype_fileter(filename):
    # return filename.endswith(u'.java') and not bool(re.search('test.*\/', filename))
@@ -266,16 +267,24 @@ def getAllCommits(datasetName):
 def core():
    datasets = pd.read_csv(join(ROOT,'data', 'datasets.csv'))
    # repoList = ['FFmpeg','curl','nginx','openssl','redis','tmux','vlc']
-    pjs = listdir(DATASET_PATH)
-    # newRepo = ['php-src','libtiff','cpython']
-    pjs = [i for i in pjs if not (i.startswith('.') or i.startswith('codeflaws'))]
-    # pjs = [i for i in pjs if (i in newRepo )]
-    repoList = pjs
+
+    pjList = PROJECT_LIST.split(',')
+    if not os.path.exists(DATASET_PATH):
+        os.mkdir(DATASET_PATH)
+
    for repo,src in datasets.values.tolist():
-        cmd = 'git -C ' + DATASET_PATH + ' clone ' + src
-        shellCallTemplate(cmd)
-        logging.info(repo)
-        collectBugFixPatches(repo)
+        if(pjList != ['ALL']):
+            if repo in pjList:
+                 print(repo)
+                 cmd = 'git -C ' + DATASET_PATH + ' clone ' + src
+                 shellCallTemplate(cmd)
+                 logging.info(repo)
+                 collectBugFixPatches(repo)
+        else:
+            cmd = 'git -C ' + DATASET_PATH + ' clone ' + src
+            shellCallTemplate(cmd)
+            logging.info(repo)
+            collectBugFixPatches(repo)

 def codeflaws():
    cf = listdir(join(DATASET_PATH,'codeflaws'))
@@ -1,21 +1,21 @@
 java:
    8home:  /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home
 spinfer:
-    home:   /Users/anilkoyuncu/projects/fixminer/spinfer/spinfer.native
+    home:   /Users/anilkoyuncu/projects/richedit/spinfer/spinfer.native
    #home:   /Users/anil.koyuncu/projects/spinfer/spinfer.native
 dataset:
-#    home:   /Users/anil.koyuncu/projects/test/fixminer-core/python/data/gumInputLinux/
-    home:   /Users/anilkoyuncu/projects/fixminer/fixminer-core/python/data/gumInputLinux
-#    home:   /data/fixminer-core/python/data/gumInputLinux/
-    repo:   /Users/anil.koyuncu/projects/datasets
+#    home:   /Users/anil.koyuncu/projects/test/richedit-core/python/data/gumInputLinux/
+    home:   /Users/anilkoyuncu/projects/richedit/richedit-core/python/data/gumInputLinux
+#    home:   /data/richedit-core/python/data/gumInputLinux/
+    repo:   /Users/anilkoyuncu/projects/datasets

 coccinelle:
-    home:   /Users/anilkoyuncu/projects/fixminer/spinfer/statics
+    home:   /Users/anilkoyuncu/projects/richedit/spinfer/statics
    #home:   /Users/anil.koyuncu/projects/spinfer/statics


 fixminer:
-#    datapath: /Users/anil.koyuncu/projects/test/fixminer-core/python/data/
-    datapath: /Users/anilkoyuncu/projects/fixminer/fixminer-data/
+#    datapath: /Users/anil.koyuncu/projects/test/richedit-core/python/data/
+    datapath: /Users/anilkoyuncu/projects/richedit/richedit-data/
    projectType : c
-#    datapath: /data/fixminer-core/python/data/
+#    datapath: /data/richedit-core/python/data/
@@ -5,7 +5,13 @@
 	<groupId>edu.lu.uni.serval</groupId>
 	<artifactId>RichEdit</artifactId>
 	<version>1.0.0</version>
-	<packaging>jar</packaging>
+
+<!--	<parent>-->
+<!--		<groupId>edu.lu.uni.serval</groupId>-->
+<!--		<artifactId>FixPatternMiner</artifactId>-->
+<!--		<version>1.0.0</version>-->
+<!--	</parent>-->
+<!--	<packaging>pom</packaging>-->

 	<name>RichEdit</name>
 	<url>http://maven.apache.org</url>
@@ -35,7 +41,7 @@
 			<artifactId>progressbar</artifactId>
 			<version>0.8.1</version>
 		</dependency>
-				
+
 		<!-- akka -->
 		<dependency>
 			<groupId>com.typesafe.akka</groupId>
@@ -46,21 +52,21 @@
 		<dependency>
 			<groupId>com.github.gumtreediff</groupId>
 			<artifactId>core</artifactId>
-			<scope>compile</scope>
+<!--			<scope>compile</scope>-->
 			<version>${gumtree.version}</version>
 		</dependency>

 		<dependency>
 			<groupId>com.github.gumtreediff</groupId>
 			<artifactId>gen.jdt</artifactId>
-			<scope>compile</scope>
+<!--			<scope>compile</scope>-->
 			<version>${gumtree.version}</version>
 		</dependency>

 		<dependency>
 			<groupId>com.github.gumtreediff</groupId>
 			<artifactId>gen.srcml</artifactId>
-			<scope>compile</scope>
+<!--			<scope>compile</scope>-->
 			<version>${gumtree.version}</version>
 		</dependency>

@@ -82,7 +88,7 @@
 			<artifactId>logback-core</artifactId>
 			<version>${logback.version}</version>
 		</dependency>
-		
+
 		<dependency>
 			<groupId>junit</groupId>
 			<artifactId>junit</artifactId>
@@ -110,6 +116,11 @@
 			<version>${commons-io.version}</version>
 		</dependency>

+		<dependency>
+			<groupId>org.yaml</groupId>
+			<artifactId>snakeyaml</artifactId>
+			<version>1.21</version>
+		</dependency>



@@ -127,6 +138,19 @@
 					<target>1.8</target>
 				</configuration>
 			</plugin>
+			<plugin>
+				<groupId>org.apache.maven.plugins</groupId>
+				<artifactId>maven-source-plugin</artifactId>
+				<version>2.2.1</version>
+				<executions>
+					<execution>
+						<id>attach-sources</id>
+						<goals>
+							<goal>jar-no-fork</goal>
+						</goals>
+					</execution>
+				</executions>
+			</plugin>

 			<plugin>
 				<groupId>org.apache.maven.plugins</groupId>
@@ -141,8 +165,8 @@
 							<archive>
 								<manifest>
 									<mainClass>
-										<!--edu.lu.uni.serval.fixminer.jobs.CompareTrees-->
-										edu.lu.uni.serval.fixminer.Launcher
+										<!--edu.lu.uni.serval.richedit.jobs.CompareTrees-->
+										edu.lu.uni.serval.Launcher
 									</mainClass>
 								</manifest>
 							</archive>
@@ -155,9 +179,11 @@
 			</plugin>
 		</plugins>
 	</build>
+
+
 </project>

-		<!--edu.lu.uni.serval.fixminer.Launcher
-		edu.lu.uni.serval.fixminer.jobs.CompareTrees
+		<!--edu.lu.uni.serval.richedit.Launcher
+		edu.lu.uni.serval.richedit.jobs.CompareTrees
 edu.lu.uni.serval.utils.ClusterToPattern
 		-->
@@ -1,16 +1,16 @@
-package edu.lu.uni.serval.fixminer;
+package edu.lu.uni.serval;


-import edu.lu.uni.serval.fixminer.jobs.CompareTrees;
-import edu.lu.uni.serval.fixminer.jobs.EnhancedASTDiff;
+import edu.lu.uni.serval.richedit.jobs.CompareTrees;
+import edu.lu.uni.serval.richedit.jobs.EnhancedASTDiff;
 import edu.lu.uni.serval.utils.ClusterToPattern;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
+import org.yaml.snakeyaml.Yaml;

 import java.io.FileInputStream;
 import java.io.IOException;
-import java.net.InetAddress;
-import java.net.UnknownHostException;
+import java.util.Map;
 import java.util.Properties;

 /**
@@ -43,26 +43,40 @@ public class Launcher {
 //        else{
 //             appConfigPath = "src/main/resource/"+hostname.split("\\.")[0]+".app.properties";
 //        }
+        if(args.length != 2)
+        {
+            System.out.println("Proper Usage is: \n\tfirst argument full path to .properties file (e.g. an example is located under resources) \n\tsecond argument jobType (e.g RICHEDITSCRIPT, COMPARE)");
+            System.exit(0);
+        }
+
        String appConfigPath = args[0];
+
+        Yaml yaml = new Yaml();
+//        InputStream inputStream = this.getClass()
+//                .getClassLoader()
+//                .getResourceAsStream("customer.yaml");
+        Map<String, Object> obj = yaml.load(new FileInputStream(appConfigPath));
+
        appProps.load(new FileInputStream(appConfigPath));
+        Map<String, Object> fixminer = (Map<String, Object>) obj.get("fixminer");
+//        String numOfWorkers = appProps.getProperty("numOfWorkers", "10");
+        String numOfWorkers = String.valueOf(fixminer.get("numOfWorkers"));
+        String portDumps = String.valueOf(fixminer.get("portDumps"));
+        String projectType = (String) fixminer.get("projectType");

-        String numOfWorkers = appProps.getProperty("numOfWorkers", "10");
-        String portDumps = appProps.getProperty("portDumps","6399");
-        String projectType = appProps.getProperty("projectType","java");
-
-        String hunkLimit = appProps.getProperty("hunkLimit","10");
-        String patchSize = appProps.getProperty("patchSize","50");
-        String projectL = appProps.getProperty("projectList","");
+        String hunkLimit = String.valueOf(fixminer.get("hunkLimit"));
+        String patchSize = String.valueOf(fixminer.get("patchSize"));
+        String projectL = (String) fixminer.get("projectList");
        String[] projectList = projectL.split(",");
-        String input = appProps.getProperty("inputPath","FORKJOIN");
-        String redisPath = appProps.getProperty("redisPath","FORKJOIN");
-        String srcMLPath = appProps.getProperty("srcMLPath","FORKJOIN");
+        String input = (String) fixminer.get("inputPath");
+        String redisPath = (String) fixminer.get("redisPath");
+        String srcMLPath = (String) fixminer.get("srcMLPath");

 //        String parameter = args[2];
        String parameter = "L1";
-//        String jobType = args[1];
+        String jobType = args[1];
 //        String jobType = "RICHEDITSCRIPT";
-        String jobType = "COMPARE";
+//        String jobType = "COMPARE";


        mainLaunch( numOfWorkers, jobType, portDumps,projectType,input,redisPath,parameter, srcMLPath,hunkLimit,projectList,patchSize);
@@ -121,7 +135,8 @@ public class Launcher {

            }
        } catch (Exception e) {
-            e.printStackTrace();
+            log.error(e.getMessage());
+//            e.printStackTrace();

        }

@@ -1,7 +1,7 @@
-package edu.lu.uni.serval.fixminer;
+package edu.lu.uni.serval.richedit;

-import edu.lu.uni.serval.fixminer.jobs.CompareTrees;
-import edu.lu.uni.serval.fixminer.jobs.EnhancedASTDiff;
+import edu.lu.uni.serval.richedit.jobs.CompareTrees;
+import edu.lu.uni.serval.richedit.jobs.EnhancedASTDiff;
 import edu.lu.uni.serval.utils.ClusterToPattern;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -15,9 +15,9 @@ import java.util.Properties;
 /**
 * Created by anilkoyuncu on 14/04/2018.
 */
-public class Launcher {
+public class Launcher2 {

-    private static Logger log = LoggerFactory.getLogger(Launcher.class);
+    private static Logger log = LoggerFactory.getLogger(Launcher2.class);

    public static void main(String[] args) throws IOException {

@@ -1,4 +1,4 @@
-package edu.lu.uni.serval.fixminer.ediff;
+package edu.lu.uni.serval.richedit.ediff;

 /**
 * Created by anilkoyuncu on 18/09/2018.
@@ -1,4 +1,4 @@
-package edu.lu.uni.serval.fixminer.ediff;
+package edu.lu.uni.serval.richedit.ediff;

 import akka.actor.ActorRef;
 import akka.actor.Props;
@@ -1,4 +1,4 @@
-package edu.lu.uni.serval.fixminer.ediff;
+package edu.lu.uni.serval.richedit.ediff;

 import com.github.gumtreediff.tree.ITree;
 import edu.lu.uni.serval.utils.EDiffHelper;
@@ -1,4 +1,4 @@
-package edu.lu.uni.serval.fixminer.ediff;
+package edu.lu.uni.serval.richedit.ediff;

 import redis.clients.jedis.JedisPool;

@@ -1,4 +1,4 @@
-package edu.lu.uni.serval.fixminer.ediff;
+package edu.lu.uni.serval.richedit.ediff;

 import com.github.gumtreediff.actions.model.Action;
 import com.github.gumtreediff.gen.srcml.GumTreeCComparer;
@@ -1,4 +1,4 @@
-package edu.lu.uni.serval.fixminer.ediff;
+package edu.lu.uni.serval.richedit.ediff;

 import akka.actor.Props;
 import akka.actor.UntypedActor;
@@ -1,4 +1,4 @@
-package edu.lu.uni.serval.fixminer.ediff;
+package edu.lu.uni.serval.richedit.ediff;

 import com.github.gumtreediff.actions.model.Action;
 import com.github.gumtreediff.tree.ITree;
@@ -1,4 +1,4 @@
-package edu.lu.uni.serval.fixminer.ediff;
+package edu.lu.uni.serval.richedit.ediff;

 import com.github.gumtreediff.actions.model.*;
 import com.github.gumtreediff.tree.ITree;
@@ -1,7 +1,6 @@
-package edu.lu.uni.serval.fixminer.ediff;
+package edu.lu.uni.serval.richedit.ediff;


-import com.github.gumtreediff.actions.model.Action;
 import com.github.gumtreediff.actions.model.*;
 import com.github.gumtreediff.gen.srcml.NodeMap_new;
 import com.github.gumtreediff.tree.ITree;
@@ -1,4 +1,4 @@
-package edu.lu.uni.serval.fixminer.ediff;
+package edu.lu.uni.serval.richedit.ediff;

 import java.io.File;

@@ -1,4 +1,4 @@
-package edu.lu.uni.serval.fixminer.ediff;
+package edu.lu.uni.serval.richedit.ediff;

 import redis.clients.jedis.JedisPool;

@@ -1,4 +1,4 @@
-package edu.lu.uni.serval.fixminer.ediff;
+package edu.lu.uni.serval.richedit.ediff;

 public interface ParserInterface {
 	
@@ -1,4 +1,4 @@
-package edu.lu.uni.serval.fixminer.ediff;
+package edu.lu.uni.serval.richedit.ediff;

 import redis.clients.jedis.JedisPool;

@@ -1,6 +1,5 @@
-package edu.lu.uni.serval.fixminer.jobs;
+package edu.lu.uni.serval.richedit.jobs;

-import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
 import edu.lu.uni.serval.utils.CallShell;
 import edu.lu.uni.serval.utils.EDiffHelper;
 import edu.lu.uni.serval.utils.PoolBuilder;
@@ -31,7 +30,7 @@ public class CompareTrees {

    public static void main(String redisPath, String portDumps, String dumpsName, String job,String numOfWorkers) throws Exception {

-        // shape /Users/anil.koyuncu/projects/test/fixminer-core/python/data/redis ALLdumps-gumInput.rdb clusterl0-gumInputALL.rdb /Users/anil.koyuncu/projects/test/fixminer-core/python/data/richEditScript
+        // shape /Users/anil.koyuncu/projects/test/richedit-core/python/data/redis ALLdumps-gumInput.rdb clusterl0-gumInputALL.rdb /Users/anil.koyuncu/projects/test/richedit-core/python/data/richEditScript

 //        String portInner = "6380";
        String port = portDumps; //"6399";
@@ -1,8 +1,8 @@
-package edu.lu.uni.serval.fixminer.jobs;
+package edu.lu.uni.serval.richedit.jobs;

-import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
+import edu.lu.uni.serval.richedit.ediff.EDiffHunkParser;
 import edu.lu.uni.serval.utils.FileHelper;
-import edu.lu.uni.serval.fixminer.ediff.MessageFile;
+import edu.lu.uni.serval.richedit.ediff.MessageFile;
 import edu.lu.uni.serval.utils.CallShell;
 import edu.lu.uni.serval.utils.PoolBuilder;
 import me.tongfei.progressbar.ProgressBar;
@@ -47,6 +47,9 @@ public class EnhancedASTDiff {
 		}
 		File folder = new File(inputPath);
 		File[] listOfFiles = folder.listFiles();
+		if(listOfFiles == null){
+			throw new Exception("No projects found, please verify the projects in the input path");
+		}
 		Stream<File> stream = Arrays.stream(listOfFiles);
 		List<File> folders;
 		if (projectList.length == 1 && projectList[0].equals("")){
@@ -5,7 +5,7 @@ import com.github.gumtreediff.gen.srcml.NodeMap_new;
 import com.github.gumtreediff.tree.ITree;
 import com.github.gumtreediff.tree.TreeContext;
 import com.github.gumtreediff.tree.TreeUtils;
-import edu.lu.uni.serval.fixminer.ediff.HierarchicalActionSet;
+import edu.lu.uni.serval.richedit.ediff.HierarchicalActionSet;
 import org.apache.commons.lang3.SerializationUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -8,10 +8,10 @@ projectList =
 #projectList = libtiff,php-src,cpython,wireshark,gzip,gmp,lighttpd1.4,lighttpd2

 #inputPath = /Users/anilkoyuncu/projects/gumInputLinux
-#inputPath = /Users/anil.koyuncu/projects/fixminer/fixminer-data/gumInputLinux
+#inputPath = /Users/anil.koyuncu/projects/richedit/richedit-data/gumInputLinux
 inputPath = /Users/anil.koyuncu/projects/test/fixminer-data/patches
-#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
-#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
+#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
+#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
 redisPath = /Users/anil.koyuncu/projects/test/fixminer-core/python/data/redis
 #srcMLPath=  /Users/anil.koyuncu/Downloads/srcML.0.9.5/bin/srcml
 srcMLPath=  /Users/anil.koyuncu/projects/test/srcML/bin/srcml
@@ -9,10 +9,10 @@ patchSize = 50
 projectList = libtiff,php-src,cpython,wireshark,gzip,gmp,lighttpd1.4,lighttpd2

 #inputPath = /Users/anilkoyuncu/projects/gumInputLinux
-#inputPath = /Users/anil.koyuncu/projects/fixminer/fixminer-data/gumInputLinux
+#inputPath = /Users/anil.koyuncu/projects/richedit/richedit-data/gumInputLinux
 inputPath = /Users/anil.koyuncu/projects/test/fixminer-data/patches
-#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
-#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
+#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
+#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
 redisPath = /Users/anil.koyuncu/projects/test/fixminer-core/python/data/redis
 #srcMLPath=  /Users/anil.koyuncu/Downloads/srcML.0.9.5/bin/srcml
 srcMLPath=  /usr/local/bin/srcml
@@ -9,7 +9,7 @@ patchSize = 50
 projectList = codeflaws
 #inputPath = /Users/anilkoyuncu/projects/gumInputLinux
 inputPath = /Users/anilkoyuncu/projects/fixminer/fixminer-core/python/data/gumInputLinux
-#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
+#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
 redisPath = /Users/anilkoyuncu/projects/fixminer/fixminer-core/python/data/redis
 #srcMLPath=  /Users/anil.koyuncu/Downloads/srcML/src2srcml
 srcMLPath=  /usr/local/bin/srcml
@@ -1,7 +1,7 @@
 package edu.lu.uni.serval.FixPatternMiner;

-import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
-import edu.lu.uni.serval.fixminer.ediff.HierarchicalActionSet;
+import edu.lu.uni.serval.richedit.ediff.EDiffHunkParser;
+import edu.lu.uni.serval.richedit.ediff.HierarchicalActionSet;

 import java.io.File;
 import java.io.FileInputStream;
@@ -33,7 +33,7 @@ public class BaseTest {
            List<HierarchicalActionSet> hierarchicalActionSets = parser.parseChangedSourceCodeWithGumTree2(prevFile, revFile, srcMLPath,false);
            return hierarchicalActionSets;
        }catch (NullPointerException n){
-//            String cmd = "cp /Users/anil.koyuncu/projects/test/fixminer-data/patches/codeflaws/"+n.getMessage().split(root)[1] + " /Users/anil.koyuncu/projects/test/fixminerC/"+n.getMessage();
+//            String cmd = "cp /Users/anil.koyuncu/projects/test/richedit-data/patches/codeflaws/"+n.getMessage().split(root)[1] + " /Users/anil.koyuncu/projects/test/fixminerC/"+n.getMessage();
 //            CallShell cs = new CallShell();
 //            try {
 //                cs.runShell(cmd);
@@ -1,8 +1,8 @@
 package edu.lu.uni.serval.FixPatternMiner;

 import com.github.gumtreediff.tree.ITree;
-import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
-import edu.lu.uni.serval.fixminer.ediff.HierarchicalActionSet;
+import edu.lu.uni.serval.richedit.ediff.EDiffHunkParser;
+import edu.lu.uni.serval.richedit.ediff.HierarchicalActionSet;
 import edu.lu.uni.serval.utils.ClusterToPattern;
 import edu.lu.uni.serval.utils.EDiffHelper;
 import edu.lu.uni.serval.utils.PoolBuilder;
@@ -16,16 +16,17 @@ import java.io.File;
 import java.io.IOException;
 import java.time.Duration;
 import java.time.Instant;
-import java.util.*;
+import java.util.ArrayList;
+
@Ignore
 public class HunkParserTest extends BaseTest{

    @Test
    public void testSimple() throws IOException {
-//        String input = "/Users/anil.koyuncu/projects/test/fixminer-core/python/data/gumInputLinux/revFiles/7f52f3_3845d29_drivers#pci#host#pcie-altera.c";
+//        String input = "/Users/anil.koyuncu/projects/test/richedit-core/python/data/gumInputLinux/revFiles/7f52f3_3845d29_drivers#pci#host#pcie-altera.c";

 //        String root = "//Users/anilkoyuncu/projects/gumInputLinux/";
-        String root = "/Users/anilkoyuncu/projects/fixminer/fixminer-core/python/data/gumInputLinux/";
+        String root = "/Users/anilkoyuncu/projects/richedit/richedit-core/python/data/gumInputLinux/";
        String filename ="";
 //        filename ="freebsd_ceca9b8_b864ac4_sys#kern#sched_ule.c"; //too long
 //        filename ="openbsd_e592ec_39c81a_sys#arch#i386#pci#pci_machdep.c"; //not parseable 56 "parameter_list" "" () ( (57 "parameter" "" () ( (22 "decl" "" () ())))
@@ -91,7 +92,7 @@ public class HunkParserTest extends BaseTest{
    public void dumpFnction() throws Exception {
        String pattern = "function/20/gstreamer_0af74c_e8bae0_libs#gst#net#gstptpclock.c.txt_0";
 //        String pattern = "function/20/FFmpeg_4c9d1c_3da860_libavutil#file_open.c.txt_0";
-        ClusterToPattern.main("6399","/Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis","ALLdumps-gumInput.rdb ",pattern);
+        ClusterToPattern.main("6399","/Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis","ALLdumps-gumInput.rdb ",pattern);
    }

    @Test
@@ -112,7 +113,7 @@ public class HunkParserTest extends BaseTest{
    public void testSimpleJava() throws IOException {


-        String root = "/Users/anil.koyuncu/projects/test/fixminer-core/python/data/gumInput/spring-amqp/";
+        String root = "/Users/anil.koyuncu/projects/test/richedit-core/python/data/gumInput/spring-amqp/";

        String filename = "5d6e02_e597c5_spring-rabbit#src#main#java#org#springframework#amqp#rabbit#connection#ConnectionFactoryUtils.java";
        File revFile = new File(root +"/revFiles/"+filename);
@@ -134,7 +135,7 @@ public class HunkParserTest extends BaseTest{
        final JedisPool outerPool = new JedisPool(PoolBuilder.getPoolConfig(), "localhost",Integer.valueOf("6399"),20000000);

        EDiffHunkParser parser = new EDiffHunkParser();
-        String root = "/Users/anil.koyuncu/projects/fixminer/gumInputLinux/linux/";
+        String root = "/Users/anil.koyuncu/projects/richedit/gumInputLinux/linux/";
        String filename = "43f8987_f596c8_drivers#acpi#nfit#core.c";
        File revFile = new File(root + "revFiles/"+ filename);
        File prevFile =new File(root + "prevFiles/prev_"+filename);
@@ -142,7 +143,7 @@ public class HunkParserTest extends BaseTest{
        String srcMLPath = "/Users/anil.koyuncu/Downloads/srcML/src2srcml";
        parser.parseFixPatterns(prevFile,revFile,diffFile,"gumInputLinux",outerPool,srcMLPath,"if",false);
        String key = "if/3/linux_bb67dd_0922c7_sound#soc#sof#intel#hda.c.txt_0";
-        File file2load = new File("/Users/anil.koyuncu/projects/fixminer/dumps/"+ key);
+        File file2load = new File("/Users/anil.koyuncu/projects/richedit/dumps/"+ key);
        byte[] dump = FileUtils.readFileToByteArray(file2load);

 //        String line = FileHelper.readFile(file2load);
@@ -1,9 +1,8 @@
 package edu.lu.uni.serval.FixPatternMiner;

-import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
-import edu.lu.uni.serval.fixminer.ediff.HierarchicalActionSet;
+import edu.lu.uni.serval.richedit.ediff.EDiffHunkParser;
+import edu.lu.uni.serval.richedit.ediff.HierarchicalActionSet;
 import edu.lu.uni.serval.utils.CallShell;
-import edu.lu.uni.serval.utils.EDiffHelper;
 import org.junit.Assert;
 import org.junit.Ignore;
 import org.junit.Test;
@@ -1236,7 +1235,7 @@ public class TestInputCases {
        List<HierarchicalActionSet> hierarchicalActionSets = parser.parseChangedSourceCodeWithGumTree2(prevFile, revFile, srcMLPath,false);
        return hierarchicalActionSets;
    }catch (NullPointerException n){
-        String cmd = "cp /Users/anil.koyuncu/projects/test/fixminer-data/patches/"+project+"/"+n.getMessage().split(root)[1] + " /Users/anil.koyuncu/projects/test/fixminerC/"+n.getMessage();
+        String cmd = "cp /Users/anil.koyuncu/projects/test/richedit-data/patches/"+project+"/"+n.getMessage().split(root)[1] + " /Users/anil.koyuncu/projects/test/fixminerC/"+n.getMessage();
        CallShell cs = new CallShell();
        try {
            cs.runShell(cmd);
@@ -1,7 +1,7 @@
 package edu.lu.uni.serval.FixPatternMiner;

-import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
-import edu.lu.uni.serval.fixminer.ediff.HierarchicalActionSet;
+import edu.lu.uni.serval.richedit.ediff.EDiffHunkParser;
+import edu.lu.uni.serval.richedit.ediff.HierarchicalActionSet;
 import org.apache.commons.lang3.StringUtils;
 import org.junit.Assert;
 import org.junit.Ignore;
@@ -1,8 +1,8 @@
 package edu.lu.uni.serval.FixPatternMiner;


-import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
-import edu.lu.uni.serval.fixminer.ediff.HierarchicalActionSet;
+import edu.lu.uni.serval.richedit.ediff.EDiffHunkParser;
+import edu.lu.uni.serval.richedit.ediff.HierarchicalActionSet;
 import org.junit.Assert;
 import org.junit.Ignore;
 import org.junit.Test;
@@ -1,8 +1,7 @@
 package edu.lu.uni.serval.FixPatternMiner;

 import com.github.gumtreediff.tree.ITree;
-import edu.lu.uni.serval.fixminer.ediff.EDiffHunkParser;
-import edu.lu.uni.serval.fixminer.ediff.HierarchicalActionSet;
+import edu.lu.uni.serval.richedit.ediff.HierarchicalActionSet;
 import edu.lu.uni.serval.utils.CallShell;
 import edu.lu.uni.serval.utils.EDiffHelper;
 import org.junit.Assert;
@@ -8,10 +8,10 @@ projectList =
 #projectList = libtiff,php-src,cpython,wireshark,gzip,gmp,lighttpd1.4,lighttpd2

 #inputPath = /Users/anilkoyuncu/projects/gumInputLinux
-#inputPath = /Users/anil.koyuncu/projects/fixminer/fixminer-data/gumInputLinux
+#inputPath = /Users/anil.koyuncu/projects/richedit/richedit-data/gumInputLinux
 inputPath = /Users/anil.koyuncu/projects/test/fixminer-data/patches
-#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
-#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
+#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
+#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
 redisPath = /Users/anil.koyuncu/projects/test/fixminer-core/python/data/redis
 #srcMLPath=  /Users/anil.koyuncu/Downloads/srcML.0.9.5/bin/srcml
 srcMLPath=  /Users/anil.koyuncu/projects/test/srcML/bin/srcml
@@ -9,10 +9,10 @@ patchSize = 50
 projectList = libtiff,php-src,cpython,wireshark,gzip,gmp,lighttpd1.4,lighttpd2

 #inputPath = /Users/anilkoyuncu/projects/gumInputLinux
-#inputPath = /Users/anil.koyuncu/projects/fixminer/fixminer-data/gumInputLinux
+#inputPath = /Users/anil.koyuncu/projects/richedit/richedit-data/gumInputLinux
 inputPath = /Users/anil.koyuncu/projects/test/fixminer-data/patches
-#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
-#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
+#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
+#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
 redisPath = /Users/anil.koyuncu/projects/test/fixminer-core/python/data/redis
 #srcMLPath=  /Users/anil.koyuncu/Downloads/srcML.0.9.5/bin/srcml
 srcMLPath=  /usr/local/bin/srcml
@@ -0,0 +1,28 @@
+java:
+    8home:  /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home
+spinfer:
+    home:   /Users/anilkoyuncu/projects/fixminer/spinfer/spinfer.native
+coccinelle:
+    home:   /Users/anilkoyuncu/projects/fixminer/spinfer/statics
+
+dataset:
+#    home:   /Users/anilkoyuncu/projects/fixminer/fixminer-core/python/data/gumInputLinux
+    inputPath : /Users/anilkoyuncu/projects/test/fixminer-data/patches
+    repo:   /Users/anilkoyuncu/projects/test/fixminer-data/datasets
+
+fixminer:
+    projectType : c
+    datapath: /Users/anilkoyuncu/projects/test/fixminer-data/
+
+
+    pjName : patches
+    portDumps : 6399
+    numOfWorkers : 14
+    hostname : localhost
+    hunkLimit : 10
+    patchSize : 50
+
+    projectList : libtiff,gzip,gmp,lighttpd1.4,lighttpd2
+    inputPath : /Users/anilkoyuncu/projects/test/fixminer-data/patches
+    redisPath : /Users/anilkoyuncu/projects/test/fixminer-core/python/data/redis
+    srcMLPath :  /usr/local/bin/srcml
@@ -9,7 +9,7 @@ patchSize = 50
 projectList = codeflaws
 #inputPath = /Users/anilkoyuncu/projects/gumInputLinux
 inputPath = /Users/anilkoyuncu/projects/fixminer/fixminer-core/python/data/gumInputLinux
-#redisPath = /Users/anil.koyuncu/projects/fixminer/fixminer-core/python/data/redis
+#redisPath = /Users/anil.koyuncu/projects/richedit/richedit-core/python/data/redis
 redisPath = /Users/anilkoyuncu/projects/fixminer/fixminer-core/python/data/redis
 #srcMLPath=  /Users/anil.koyuncu/Downloads/srcML/src2srcml
 srcMLPath=  /usr/local/bin/srcml