Skip to content

Commit

Permalink
Merge branch 'master' into master
Browse files Browse the repository at this point in the history
  • Loading branch information
@JamesWoolfenden
JamesWoolfenden authored Nov 7, 2024
2 parents 190a04b + d533d1d commit 5687304
Show file tree
Hide file tree
Showing 18 changed files with 791 additions and 164 deletions.
8 changes: 4 additions & 4 deletions .pre-commit-config.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ default_language_version:
python: python3.11
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
rev: v5.0.0
hooks:
- id: check-json
- id: check-merge-conflict
Expand All @@ -29,7 +29,7 @@ repos:
hooks:
- id: shell-lint
- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.41.0
rev: v0.42.0
hooks:
- id: markdownlint
exclude: src/testdata|testdata
Expand All @@ -39,7 +39,7 @@ repos:
- id: terraform-fmt
language_version: python3.11
- repo: https://github.com/gruntwork-io/pre-commit
rev: v0.1.23
rev: v0.1.24
hooks:
- id: gofmt
- id: goimports
Expand All @@ -52,7 +52,7 @@ repos:
- id: go-mod-tidy
- id: go-generate
- repo: https://github.com/jameswoolfenden/ghat
rev: v0.1.6
rev: v0.1.8
hooks:
- id: ghat-go
name: ghat
Expand Down
41 changes: 23 additions & 18 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,23 @@

Pike is a tool to determine the minimum permissions required to run a TF/IAC run:

Pike currently supports Terraform and supports multiple providers (AWS, GCP, AZURE),
Pike currently supports Terraform and supports multiple providers (AWS, GCP and AZURE);
Azure is the newest with AWS having the most supported resources
<https://github.com/JamesWoolfenden/pike/tree/master/src/mapping>.
Feel free to submit PR or Issue if you find an issue or even better add new resources, and then I'll take a look at
merging it ASAP.

**CAVEAT** The outputs of this tool are your first step, if you have AWS, you can now generate resources partially, there are no conditions and even partial resources are wildcarded (for now).
(for AWS)
**best practice** would go further (and I am working on it as well), you will need to modify these permissions to the minimum required in your environment by adding these
restrictions, you can also deploy using short-lived credentials (using this tool or Vault) (in AWS so far), generating short-lived credentials for your build
**CAVEAT** The outputs of this tool are your first step, if you have AWS, you can now generate resources partially,
there are no conditions and even partial resources are wild-carded (for now).
(for AWS)minimum
**best practice** would go further (and I am working on it as well), you will need to modify these permissions to the
required in your environment by adding these
restrictions, you can also deploy using short-lived credentials (using this tool or Vault) (in AWS so far), generating
short-lived credentials for your build
and then remotely (REMOTE) supply and invoke your builds (INVOKE).

Ideally I would like to do this for you, but these policies are currently determined statically (QUICKER), and unrecorded intentions can be impossible to infer.
Ideally I would like to do this for you, but these policies are currently determined statically (QUICKER), and
unrecorded intentions can be impossible to infer.

## Table of Contents

Expand Down Expand Up @@ -360,7 +364,7 @@ on:
- master
```
To authenticate the GitHub API you will need to set you GitHub Personal Access Token, as the environment variable
To authenticate with the GitHub API, you will need to set your GitHub Personal Access Token, as the environment variable
*GITHUB_TOKEN*
To Invoke a workflow, it is then:
Expand All @@ -371,7 +375,7 @@ pike invoke -workflow master.yml -branch master -repository JamesWoolfenden/terr

I created Invoke to be used in tandem with the new remote command which supplies temporary credentials to a workflow.

**Note The GitHub API is rate limited, usually 5000 calls per hour.
**Note The GitHub API is rate-limited, usually 5000 calls per hour.

```shell
pike make -d ./module/aws/terraform-aws-s3/example/examplea
Expand Down Expand Up @@ -408,7 +412,7 @@ This looks in the readme for the deliminators:
<!-- END OF PRE-COMMIT-PIKE DOCS HOOK -->
```

and replaces is either with json or Terraform like so:
and replaces is either with JSON or Terraform like so:

```markdown
This is the policy required to build this project:
Expand Down Expand Up @@ -548,7 +552,7 @@ IAM Policy arn:aws:iam::680235478471:policy/basic versus Local ../modules/aws/te
## Pull

Pull adds the ability to work with Git repositories (thanks to **go-git**),
to output the required permissions in json or Terraform:
to output the required permissions in JSON or Terraform:

```bash
./pike pull
Expand Down Expand Up @@ -759,7 +763,7 @@ Make build

## Inspect

This new feature is in *beta*, and is not yet fully supported and currently only for AWS.
This new feature is in *beta* and is not yet fully supported and currently only for AWS.
When Pike is run with inspect, it will scan your code and output a policy that is required to deploy the code, as normal,
but it will also detect the running IAM credentials.
It will then report on the overlap between the running credentials and the minimum policy.
Expand Down Expand Up @@ -850,22 +854,23 @@ e.g. *aws_security_group.json*

Datasources are the easiest to start with, I have a script (resource.ps1 - add pwsh with **brew install --cask powershell**)
that creates a blank mapping file and tf
resource, but you've seen the example json file - make one without any entries.
You also need to create a minimal resource/datasource, that you are trying to figure out the permissions for, and place it in the correct dir
resource, but you've seen the example JSON file - make one without any entries.
You also need to create a minimal resource/datasource, that you are trying to figure out the permissions for, and place
it in the correct dir
e.g../terraform/aws, I have a script for making a profile for the profile in the role directory.
You can then tf using the empty role against the resource/datasource with no permissions.
The debug output from the tf run will help you figure out the permissions you need to add to your basic role.
You then update your "basic" role.

Issues?
The providers don't always tell you want you need to add,
The providers, don't always tell you what you need to add,
you will need to check the IAM docs and the online IAM policymakers.
Not all resource are as easy as others, anything that make/scripts CF internally.
Not all resources are as easy as others, anything that make/scripts CF internally.
Some roles require *Passrole* and *CreateLinkedRole* but won't say so. Trail and error

#### What about "attributes" ?
#### What about "attributes"?

Some cloud providers require extra permissions depending on the attributes you add, this is how this is handled.
Some cloud providers require extra permissions depending on the attributes you add; this is how this is handled.
Build out your tf resources to cover all reasonable scenarios.

#### Eventual consistency
Expand All @@ -885,7 +890,7 @@ var securityGroup []byte

### Add to provider Scan

Once you have added the json import above you just need to update the lookup table,
Once you have added the JSON import, as above, you then need to update the lookup table,
so we can read it and get the permissions:

```txt
Expand Down
54 changes: 26 additions & 28 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,52 +5,52 @@ go 1.22.2
require (
github.com/JamesWoolfenden/arn v0.2.5
github.com/aws/aws-sdk-go v1.55.5
github.com/aws/aws-sdk-go-v2 v1.30.5
github.com/aws/aws-sdk-go-v2/config v1.27.35
github.com/aws/aws-sdk-go-v2/service/iam v1.35.3
github.com/aws/aws-sdk-go-v2 v1.32.3
github.com/aws/aws-sdk-go-v2/config v1.28.1
github.com/aws/aws-sdk-go-v2/service/iam v1.37.3
github.com/go-git/go-git/v5 v5.12.0
github.com/google/go-github/v47 v47.1.0
github.com/hashicorp/go-version v1.7.0
github.com/hashicorp/hc-install v0.9.0
github.com/hashicorp/hcl/v2 v2.22.0
github.com/hashicorp/terraform-exec v0.21.0
github.com/jameswoolfenden/identity v0.0.3
github.com/jameswoolfenden/identity v0.0.5
github.com/rs/zerolog v1.33.0
github.com/urfave/cli/v2 v2.27.4
golang.org/x/crypto v0.27.0
golang.org/x/exp v0.0.0-20240909161429-701f63a606c0
golang.org/x/net v0.29.0
github.com/urfave/cli/v2 v2.27.5
golang.org/x/crypto v0.28.0
golang.org/x/exp v0.0.0-20241009180824-f66d83c29e7c
golang.org/x/net v0.30.0
golang.org/x/oauth2 v0.23.0
)

require (
dario.cat/mergo v1.0.1 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/ProtonMail/go-crypto v1.1.0-alpha.5-proton // indirect
github.com/ProtonMail/go-crypto v1.1.0-beta.0-proton // indirect
github.com/agext/levenshtein v1.2.3 // indirect
github.com/apparentlymart/go-textseg/v15 v15.0.0 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.17.33 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.13 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.17 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.17 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.17.42 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.18 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.22 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.22 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.19 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.22.8 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.26.8 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.30.8 // indirect
github.com/aws/smithy-go v1.21.0 // indirect
github.com/cloudflare/circl v1.4.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.3 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.24.3 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.3 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.32.3 // indirect
github.com/aws/smithy-go v1.22.0 // indirect
github.com/cloudflare/circl v1.5.0 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.5 // indirect
github.com/cyphar/filepath-securejoin v0.3.2 // indirect
github.com/cyphar/filepath-securejoin v0.3.4 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-git/go-billy/v5 v5.5.0 // indirect
github.com/go-git/go-billy/v5 v5.6.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-retryablehttp v0.7.7 // indirect
github.com/hashicorp/terraform-json v0.22.1 // indirect
github.com/hashicorp/terraform-json v0.23.0 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
Expand All @@ -65,9 +65,9 @@ require (
github.com/zclconf/go-cty v1.15.0 // indirect
golang.org/x/mod v0.21.0 // indirect
golang.org/x/sync v0.8.0 // indirect
golang.org/x/sys v0.25.0 // indirect
golang.org/x/text v0.18.0 // indirect
golang.org/x/tools v0.25.0 // indirect
golang.org/x/sys v0.26.0 // indirect
golang.org/x/text v0.19.0 // indirect
golang.org/x/tools v0.26.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
)

Expand All @@ -78,5 +78,3 @@ require (
github.com/yudai/golcs v0.0.0-20170316035057-ecda9a501e82 // indirect
github.com/yudai/pp v2.0.1+incompatible // indirect
)

replace github.com/hashicorp/go-cty => github.com/zclconf/go-cty v1.13.2
Loading

0 comments on commit 5687304

Please sign in to comment.