Merge pull request #83 from sahilseth/devel

Devel

CONTRIBUTING.md

@@ -0,0 +1,40 @@
+# How to contribute
+
+Thanks for the support! If you could, please submit all pull requests in `devel` branch. We test things in `devel` and push then to `master` 
+after a few tests.
+
+
+## Adding a new platform
+
+<div class="alert alert-warning" role="alert">
+Possible issue: Need to add a new platform
+</div>
+
+Adding a new platform involves [a few steps](https://github.com/sahilseth/flowr/issues/7), briefly we need to consider the following steps where changes would be necessary.
+
+1. **job submission**: One needs to add a new template for the new platform. Several [examples](https://github.com/sahilseth/flowr/blob/master/inst/conf) are available as described in the previous section.
+2. **parsing job ids**: flowr keeps a log of all submitted jobs, and also to pass them along as a dependency to subsequent jobs. This is taken care by the [parse_jobids()](https://github.com/sahilseth/flowr/blob/master/R/parse-jobids.R) function. Each job scheduler shows the jobs id, when you submit a job, but each shows it in a slightly different pattern. To accommodate this one can use regular expressions as described in the relevant section of the [flowr config](https://github.com/sahilseth/flowr/blob/master/inst/conf/flowr.conf).
+
+3. **render dependency**: After collecting job ids from previous jobs, flowr renders them as a dependency for subsequent
+jobs. This is handled by [render_dependency.PLATFORM](https://github.com/sahilseth/flowr/blob/master/R/render-dependency.R) functions.
+4. **recognize new platform**: Flowr needs to be made aware of the new platform, for this we need to add a new class using the platform name. This is essentially a wrapper around the [job class](https://github.com/sahilseth/flowr/blob/master/R/class-def.R)
+
+Essentially this requires us to add a new line like: `setClass("torque", contains = "job")`.
+
+
+5. **killing jobs**: Just like submission flowr needs to know what command to use to kill jobs. This is defined in detect_kill_cmd function.
+
+
+
+There are several [job scheduling](http://en.wikipedia.org/wiki/Job_scheduler) systems
+available and we try to support the major players. Adding support is
+quite easy if we have access to them. Your favourite not in the list?
+re-open this issue, with details on the platform:
+[adding platforms](https://github.com/sahilseth/flowr/issues/7)
+
+
+
+<div class="alert alert-warning" role="alert">
+Possible issue: For other issues upload the error shown in the out files to 
+[github issues tracker](https://github.com/sahilseth/flowr/issues).
+</div>

README.md

@@ -14,7 +14,7 @@
 
 [![Build Status](https://travis-ci.org/sahilseth/flowr.svg?branch=master)](https://travis-ci.org/sahilseth/flowr)
 [![cran](http://www.r-pkg.org/badges/version/flowr)](https://cran.r-project.org/package=flowr)
-[![codecov.io](http://codecov.io/github/sahilseth/flowr/coverage.svg?branch=devel)](http://codecov.io/github/sahilseth/flowr?branch=devel)
+<!--[![codecov.io](http://codecov.io/github/sahilseth/flowr/coverage.svg?branch=devel)](http://codecov.io/github/sahilseth/flowr?branch=devel)-->
 ![downloads](http://cranlogs.r-pkg.org/badges/grand-total/flowr)
 
 <!--
@@ -79,8 +79,20 @@ install.packages("flowr", repos = c(CRAN="http://cran.rstudio.com", DRAT="http:/
 library(flowr) ## load the library
 setup() ## copy flowr bash script; and create a folder flowr under home.
 
-## Run an example pipeline
+# Run an example pipeline
+
+# style 1: sleep_pipe() function creates system cmds
 flowr run x=sleep_pipe platform=local execute=TRUE
+
+# style 2: we start with a tsv of system cmds
+# get example files
+wget --no-check-certificate http://raw.githubusercontent.com/sahilseth/flowr/master/inst/pipelines/sleep_pipe.tsv
+wget --no-check-certificate http://raw.githubusercontent.com/sahilseth/flowr/master/inst/pipelines/sleep_pipe.def
+
+# submit to local machine
+flowr to_flow x=sleep_pipe.tsv def=sleep_pipe.def platform=local execute=TRUE
+# submit to local LSF cluster
+flowr to_flow x=sleep_pipe.tsv def=sleep_pipe.def platform=lsf execute=TRUE
 ```
 
 **Example pipelines** [inst/pipelines](https://github.com/sahilseth/flowr/blob/devel/inst/pipelines)