documentation.md

Documentation for working on the DSAIDE/DSAIRM packages

The DSAIDE and DSAIRM packages are very similar in structure, so the text below applies to both. I mostly write DSAIDE, sometimes DSAIRM. Just substitute with whichever package you work on.

Package structure/use

• The main use case is the user calling the dsaidemenu() function. This function calls a Shiny app (in the /inst/DSAIDE folder), which is the main user interface. The Shiny app in turn uses the various generate_ functions to build the UI. Users can select models/apps, which are each encoded by one or more simulate_ function.
• An alternative use case is for users to write code that directly calls the simulate_ functions.
• The simulate_ functions can also be downloaded and modified by the user.
• The package documentation contains more information.

Package organization

Folder structure

• /auxiliary contains folders with related resources that are not used/needed for the package use/build but are useful (to the package authors) for maintenance/development/advertising/tracking of the package
• /data contains data used as part of the R package. The data files are documented in data.R in the /R folder.
• /doc contains the vignettes, this folder is auto-generated and should not be edited, see below.
• /docs contains the package website created by the pkgdown package. Rebuild with pkgdown::build_site(). Don't edit manually.
• /inst contains several important subfolders, further described below.
• /man contains the documentation for all public functions, automatically generated by roxygen. Do not edit.
• /Meta contains vignette information and is auto-created by devtools when package is built. Ignore.
• /pkgdown contains extra files for styling of pkgdown created website. Edit to change layout of package website.
• /R contains the main R functions, i.e. the simulators and various helper functions. See more details below.
• /tests contains code/unit tests, done with the testthat package. Add tests as new functionality is added.
• /vignettes contains the vignette - this is copied to /inst/doc during package building. edits should be done to the file in the /vignettes folder, not the /inst/doc folder.

Functions/files in the R folder

• All simulators, which can be called directly by users, start with simulate_.
• The functions/files generate_text, generate_ggplot and generate_plotly can take results returned from the simulator functions and generate plots. Users might want to use those combined with running the simulate_ functions.
• All other functions/files are only needed for the package/UI to work. They are public/exported, but are generally not meant to be directly accessed by users.
• While the helper functions are generally called by the main Shiny app, none directly handle reactive objects, all inputs and outputs are non-reactive objects/variables.

Materials inside the /inst folder

The /inst folder contains several subfolders

• /appinformation contains files with documentation and settings for each simulator/app. Each App has an Rmd+HTML file containing the documentation. Those files contain the documentation displayed at the bottom of each app, and app-specific settings that are needed for proper display and running through the UI. Each app also has a _tasktable.tsv file. Those are automatically generated when an Rmd file is turned into an HTML file. They countain all the instructions and results for each What to do section in tabular form, for easy processing when generating solutions. There is also a bibliography/references file, and the very important apptable.tsv (tab separated table) file. The latter contains details for each app which are being parsed by various functions in the package, see details below.
• DSAIDE contains the main app.R Shiny app. It also includes a css file for styling and a google analytics file for tracking users when deployed to a server. A subfolder created by rsconnect during package upload to a shiny server (e.g.) shinyapps.io might also be present. See top of app.R file for documentation on how to deploy the package to a server.
• helperfunctions contain various non-exported functions that are used when building the package, mainly when generating the html version of the documentation from the Rmd files.
• /media sub-folder contains figures (poth as SVG and png) for each model, which are used as part of the documentation (i.e. inside the app-specific Rmd/html file).
• /simulatorfunctions subfolder contains the R code/files for all simulator functions. These are copies of the simulatorfunctions from the /R folder. (That's needed because during package building, R functions are processed such that their roxygen documentation can't be easily processed.) These R files are used by the package to create the shiny UI (see the generate_shinyinput function). They are also contained in a zip folder which can be downloaded by users for easy access and editing. It is important to keep the files in sync with those in the main /R folder. The processing_script inside the /docsfordevelopers folder ensures that, thus should be run whenever there are changes.

A folder called /mbmodels might be present and can contain Rds files of underlying simulation models that have been created with the modelbuilder package. If those are present and the apptable.tsv file specifies they should be used, then the UI for a model is generated by using the information in the mbmodel, instead of parsing the documentation header of a simulation file. Currently, this is turned off and the folder might not be present.

apptable.tsv

This file, located inside /inst/appinformation, contains important meta-data for each app. Any new app needs to be added to this file. The columns are as follows:

• appid - name by which app is known internally. Needs to match entry in main app.R file.
• apptitle - Title for app to be shown in main menu.
• docname - name of html file that contains app documentation.
• modelfigname - name of file containing model figure.
• simfunction - name of underlying R code and function that is used by a given app.
• simfunction2 - a few apps use more than one underlying function, provide the second one here.
• underlying_function - a few apps call another underyling simulator function from their main function. Note this here.
• mbmodel_possible - an indicator if a given model could be generated through the modelbuilder package. Only possible for simulation models, i.e. not models that themselves call simualtion models, such as fitting, etc.
• use_mbmodel - indicator to state if the underlying mbmodel Rds file (in the mbmodels folder) should be used to generate the inputs for the UI or not (see generate_shinyinput file).
• use_doc - indicator if the top portion of the function (the roxygen documentation part) should be used to generate inputs for the UI. One should use either mbmodel or doc.
• mbmodelname - name of mbmodel, needed if we use mbmodel for UI generation.
• nplots - number of plots that should be returned for a given app. Almost always 1.
• modeltype - type of underlying model(s). This is used inside the run_model function and different actions are taken based on the model type.
• otherinputs - shiny command that is needed to generate additional UI elements that are not automatically generated through parsing.

Adding new apps/features

If you plan to build/contribute new apps or new features to the package, I would be delighted to include them! The best idea is to first contact me (through Github or email) and tell me that you would like to contribute. We can then discuss a bit before you embark on the effort.

To build a new app, you need to create the following:

• The main simulator_NNN function.
• The NNN_documentation.Rmd file.

If you contribute/send those files, I will - after testing - include them into the package. You'll of course receive full credit as package contributor.

Using the DSAIDE solutions

• Each _documentation.Rmd file has the tasks and outcomes to record stored as a table. When knitting/processing this file, both the html file of the documentation is generated and a .tsv file containing a table with the tasks and quantities to be reported, as well as other columns.
• The .tsv files should be copied into the (private) DSAIDEsolutions repository into the tasktable_files folder.
• Solutions and answers to each question are added by scripts in the DSAIDEsolutions repo. Those can be further processed with the grading app (not part of DSAIDE).

Information for package development

To work on package (through RStudio):

• Fork and clone package.
• Load DSAIDE.Rproj in RStudio. Edit files as needed.
• Use devtools/roxygen to build and document the package. Either use RStudio's functionality (or command line) to test/build packages using devtools. The package follows Hadley's recommendations and workflow described here: http://r-pkgs.had.co.nz/
• Optionally, use RStudio tie-in with github to sync project to github (the 'git' tab). Alternativley, use your favorite git client.
• Rtools needs to be installed (on Windows).

Dependency packages

All libraries/packages needed to allow the package to run should be loaded via the DESCRIPTION file and not in separate R files. See that file for dependencies.

Additional packages are needed for development (but not use) of the package. Those are listed in the suggests section of the DESCRIPTION file.

Package logo

According to pkgdown, logo should be includes as described here: https://pkgdown.r-lib.org/reference/build_home.html#package-logo

To update simulation functions

Some simulation functions are coded 'by hand', some are generated through modelbuilder. To see which is which, look at the use_mbmodel column in /inst/appinformation/apptable.tsv

To work with data

See here for how data is handled/documented: https://r-pkgs.org/data.html documentation for data is in /R/data.R

To update R documentation and vignette

• Edit documentation inside R functions.
• Edit vignette inside the /vignettes folder.
• To re-build html documentation files, copy and zip simulator functions, re-build documentation, build vignettes and rebuild site, run the processing-script in the 'helperfunctions' folder.

To build the package

• in RStudio, use the functions in the 'build' tab to test and build the package.
• Run clean and rebuild, then build and reload using menu, or devtools::load_all()
• Run the check, fix any errors
• Run the processing-script.R file

To deploy package to shinyappsio

to deploy, follow these steps (also listed in the main app.R file):

1. go into the folder where this file (app.R) resides
2. install the package through CRAN or github if we want to use the github version devtools::install_github('ahgroup/DSAIDE')
3. uncomment this line of code library('DSAIDE')
4. with the above 'library' statement active, deploy with: run rsconnect::deployApp(account = 'epibiouga') as suitable, change the account to another one, e.g. handelgroup tokens need to be set up for the connection to work
5. comment out the library command again

To-do for CRAN release

• "by hand" edit the DESCRIPTION file to make sure it's up to date
• update news.md and cran-comments.md
• run processing-script to Re-build documentation, vignettes and copy latest versions of simulator functions - see above
• Run check and make sure no problems occur
• Sync everything to github
• Run devtools::check_rhub(), devtools::check_win_release() and devtools::check_win_devel(), devtools::check_mac_release()
• Do a test run of devtools::release() - fix any remaining issues, then re-run and release

Notes on 'notes'

• If CRAN check complains about "Undefined global functions or variables" in ggplot2 code. Add those to a globals.R file or inside the menu.R file.

To manage github actions for CI

• Run

Trouble-shooting

• If you get the 'can't find qpdf error' during build, try this (on windows): https://stackoverflow.com/questions/41570633/how-to-build-qpdf-on-windows