You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: qlty-plugins/plugins/plugin_guide.md
+136-18
Original file line number
Diff line number
Diff line change
@@ -1,39 +1,157 @@
1
-
# Qlty Plugins Guide
1
+
# Qlty Plugin Development Guide
2
2
3
-
This is a loose guide on how to add new plugins.
3
+
The plugins directory contains definitions and tests for plugins supported by the Qlty CLI.
4
4
5
-
This plugins repo basically contains definitions and tests for various plugins supported by the main Qlty application.
5
+
Creating a plugin in Qlty typically the following steps:
6
6
7
-
## Toml file
7
+
1. Creating the folder structure and a valid plugin.toml file
8
+
2. (Sometimes) Creating a parser in Rust which understands the tool's output format
9
+
3. Creating at least 1 test target file in the fixtures directory with its expected output as a snapshot
8
10
9
-
The code to add support for a given plugin (`MY_PLUGIN`) is placed under `linters/MY_PLUGIN`.
10
-
The definitions for a given plugin can be found in `plugin.toml` file, which contains instructions on how to install the plugin, as well as how to execute it.
11
+
## Plugin Structure / Definition
12
+
13
+
(N.B.: Copying and editing an existing plugin is a good way to get started, but this section helps put these files into context.)
14
+
15
+
A plugin consists of:
16
+
17
+
1. A top-level folder for `MY_PLUGIN` at `linters/${MY_PLUGIN}`
18
+
2. The plugin definition file `linters/${MY_PLUGIN}/plugin.toml` located at the top level of this folder, along with a simple test runner
19
+
3. A "fixtures" directory under plugins containing the test target and snapshot for the target.
20
+
21
+
## The Plugin Definition File ("plugin.toml")
22
+
23
+
The plugin definition file (`plugin.toml`) is the heart of the plugin; it contains instructions on how to install the plugin; its capabilities; and how to run it.
24
+
25
+
You'll typically see at least a section for the definition of the plugin (`[plugins.definitions.${MY_PLUGIN}]`) as well as one or more sections for each "driver" the plugin supports. For a plugin that supports both formatting and linting, you'd see two a section for each one of these drivers: `[plugins.definitions.${MY_PLUGIN}.drivers.lint]` and `[plugins.definitions.${MY_PLUGIN}.drivers.format]`
11
26
12
27
### Plugin Installation
13
28
14
-
For plugins which have a usable executable release on Github, those can be installed via Github releases using `[plugins.releases.MY_PLUGIN]`.
15
-
Example: `linters/gitleaks/plugin.toml`.
29
+
Every plugin.toml needs to define how to find/install the plugin. There are a variety of options, listed below:
30
+
31
+
### Plugin Installation: GitHub Release
32
+
33
+
If the plugin lives on GitHub, as many do, and stores its releases on GitHub, the plugin can define (and later reference) a "releases" section.
34
+
35
+
```
36
+
[plugins.releases.${MY_PLUGIN}]
37
+
github = "plugin-owner/plugin-repo"
38
+
download_type = "executable"
39
+
```
40
+
41
+
`download_type` can also be defined as a `zip` or `targz`
42
+
43
+
The plugin definition references this release as follows:
44
+
45
+
```
46
+
[plugins.definitions.${MY_PLUGIN}]
47
+
releases = ["${MY_PLUGIN}"]
48
+
```
49
+
50
+
NB: The GitHub Release installation type uses a heuristic to determine the appropriate download URL for the platform/architecture in question. If the release doesn't follow this heuristic, you may find you may need to define each architecture/platform individually
51
+
52
+
### Plugin Installation: Downloads
53
+
54
+
If the plugin binaries can be downloaded, you can also define download sections for each known platform / architecture. This is also appropriate for some GitHub Releases which do not follow a URL standard we recognize for architecture/platform.
55
+
56
+
Define a download section for each architecture/platform the plugin supports. E.g. and substitute a variable ${version} if the version is in the URL.
In the main plugin definition section, reference these downloads as follows:
73
+
74
+
```
75
+
[plugins.definitions.${MY_PLUGIN}]
76
+
downloads = ["${MY_PLUGIN}"]
77
+
```
78
+
79
+
### Plugin Installation: With a Runtime
80
+
81
+
For plugins which require a runtime, such as python, define the runtime and package in the main plugin defintion. For example:
82
+
83
+
```
84
+
[plugins.definitions.${MY_PLUGIN}]
85
+
runtime = "python"
86
+
package = "${MY_PLUGIN}"
87
+
```
88
+
89
+
### Plugin Driver(s)
90
+
91
+
Drivers are defined under `[plugins.definitions.${MY_PLUGIN}.drivers.${driver}]` and contain the script, success codes, output, output format, batch and a few other options.
16
92
17
-
For plugins which require a runtime, such as python, they require the package and runtime option in their definitions `[plugins.definitions.MY_PLUGIN]`.
18
-
Example: `linters/black/plugin.toml`.
93
+
-`success_codes`: An array of exit codes that denote that the plugin run finished successfully. Ordinarily you might expect this to just be 0 to denote a binary exit successfully, but because many plugins use a 0 vs 1 (or 2) to differentiate between successful runs that did not find any issues vs successful runs that did find issues, Qlty accepts an array of valid exit codes.
19
94
20
-
### Plugin Run
95
+
##Creating a Parser
21
96
22
-
The script and the various options to run the plugin correctly can also be found in the `plugin.toml` file. `[plugins.definitions.MY_PLUGIN.drivers.lint]` contains the script, success codes, output, output format, batch and a few other options.
97
+
If the plugin supports a standard format, like SARIF, you are set, and do not need to create a parser. Many tools output structured (like JSON), but not standardized, output, which requires a Rust parser to translate that structure into Qlty issues.
23
98
24
-
## Tests
99
+
Parsers can be found in `qlty_check/source/parser/*` and are the best source for learning how to write a parser. Writing a parser involves:
25
100
26
-
The snapshot tests are under the `linters/MY_PLUGIN/fixtures` directory. The my `linters/MY_PLUGIN/MY_PLUGIN.test.ts` file contains the call to the test function `linterCheckTest` with the name of the plugin as argument.
101
+
1. Writing the Rust code to translate the output format
102
+
2. Writing an inline test within the parser file for this parser
103
+
3. Adding new references to the parser within the codebase
27
104
28
-
The `.qlty/qlty.toml` file in the mian `qlty` repo needs to be updated when adding new plugins.
105
+
### Parser tests
106
+
107
+
Each parser contains an inline test which typically can be run within VSCode by clicking "Run tests". VSCode simply runs (`cargo test --package qlty-check --lib -- parser::reek::test::parse --exact --show-output` e.g.) behind the scenes.
108
+
109
+
Typically, you'll update the input of the test by hand to match the tool's output.
110
+
111
+
And you can use `insta` to write the test output for you automatically, which helps prevents test failures from small character deltas in manually written output. Install `insta` with:
29
112
30
113
```
31
-
[sources.default]
32
-
directory = "/PATH_TO_PLUGINS/plugins"
114
+
cargo install cargo-insta
33
115
```
34
116
117
+
Run `cargo insta review` to display and accept/reject test output.
118
+
119
+
## Plugin Tests
120
+
121
+
### Test setup
122
+
123
+
The first time you run any tests you'll need to run `npm install` at the top level of the `qlty-plugin` directory.
124
+
125
+
The snapshot tests are under the `linters/MY_PLUGIN/fixtures` directory. The my `linters/${MY_PLUGIN}/${MY_PLUGIN}.test.ts` file contains the call to the test function `linterCheckTest` with the name of the plugin as argument -- copying and adjusting a runner from an existing plugin is easiest.
126
+
127
+
Create the "fixtures" directory structure if one does not exist as well as:
128
+
129
+
130
+
- an example "target" typically named something like basic.in.py (extension dependent on plugin). This needs to be a file with issues the plugin identifies
131
+
- a \_\_snapshots\_\_ directory containing snaphotted output from a plugin run against the fixture file created above.
132
+
- in the snaphost directory, a snapshot file for each version / file named e.g. `basic_v0.2.2.shot` where the version matches the plugin version you're testing against.
133
+
134
+
### Running tests
135
+
35
136
Once that is setup you can run `npm test` to run the test suite or `npm test MY_PLUGIN` to run tests for a specific plugin.
36
137
138
+
```
139
+
npm test ${PLUGIN_NAME}
140
+
```
141
+
142
+
e.g. `npm test reek`
143
+
### Using Local Plugin Definitions
144
+
145
+
Qlty's default source for plugin definitions is the github.com/qltysh/qlty remote repository, which is fine for most cases, but is not always appropriate for plugin development where developers expect that their local plugin definitions to be used.
146
+
147
+
To ensure a run of Qlty is using your local plugin definitions, adjust the relevant `.qlty/qlty.toml` definition file to instead reference a local source:
You can use `DEBUG=qlty:MY_PLUGIN` env variable to get debug logs which include a path to the sandbox for a given plugin or `DEBUG=qlty:*` for all of them.
@@ -50,7 +168,7 @@ Also to test and add snapshots for the updated versions of the plugins, you can
50
168
51
169
NOTE: It will add snapshots in .shot files, but will not validate with older snapshot files.
52
170
53
-
### Some gachas and solutions
171
+
### Some gotchas and solutions
54
172
55
173
- You can use the logs in `.qlty` folder to ensure plugin is installing correctly. Make sure the runtime version and plugin version are correctly defined in `.qlty/qlty.toml` in case of installation issues.
0 commit comments