Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update v0.1.3 #12

Merged
merged 49 commits into from
Aug 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
c4f02b8
convert to chainguard image for consistency
BarryBoot Feb 24, 2024
0ddc27f
default config for local docker-compose
BarryBoot Feb 24, 2024
9465cc3
use bitnami mongo image for parity with live environment
BarryBoot Feb 24, 2024
99666da
initial workflow
BarryBoot Feb 24, 2024
a28f01d
Merge branch 'main' into develop
deowk Apr 23, 2024
35f9d7e
update workflow
BarryBoot Apr 25, 2024
28f1e2c
update workflow
BarryBoot Apr 25, 2024
76b32f8
update workflow
BarryBoot Apr 25, 2024
bb60ab7
update workflow
BarryBoot Apr 25, 2024
3839fa1
accept DS001 - chainguard uses :latest
BarryBoot Apr 25, 2024
543465a
Merge pull request #3 from AIBlockOfficial/update-workflow
BarryBoot Apr 25, 2024
fdd218e
remove nodeType
BarryBoot Apr 25, 2024
31ba9a6
Merge pull request #4 from AIBlockOfficial/update-workflow
BarryBoot Apr 25, 2024
90c2728
Bugfix for missing cache entry defaulting to DB
BHouwens Jun 4, 2024
c00d25e
Fixing unit tests and cargo fmt
BHouwens Jun 4, 2024
6dcf869
Update to valence_core v0.1.4
BHouwens Jun 4, 2024
44b5347
Merge pull request #6 from AIBlockOfficial/bugfix_default_handling
RobinP122 Jun 7, 2024
5b7e210
Merge branch 'main' into develop
BHouwens Jun 12, 2024
5ebb359
Save and load cuckoo filter from disk
BHouwens Jun 14, 2024
20adfd4
Cargo fmt
BHouwens Jun 17, 2024
2ddfe45
Merge pull request #7 from AIBlockOfficial/save_cuckoo_to_disk
BHouwens Jun 20, 2024
75d65a2
Working deserialization for return data
BHouwens Jun 20, 2024
1b7fdb4
Cargo fmt
BHouwens Jun 20, 2024
61b7645
Merge pull request #8 from AIBlockOfficial/bugfix_json_value
BHouwens Jun 20, 2024
ff92708
Updating valence_core to v0.1.5
RobinP122 Jun 26, 2024
c4025b0
Changing 'delete' to 'del'
BHouwens Jun 26, 2024
2723c4f
Merge pull request #9 from AIBlockOfficial/bugfix_json_value
BHouwens Jun 26, 2024
712b184
Base implementation of del_data route
RobinP122 Jun 26, 2024
af489f7
Moved db crate from valence_core to Valence app. This simplifies the …
RobinP122 Jun 27, 2024
e5c1d68
Fix warning
RobinP122 Jun 27, 2024
6aec404
Bump up version to 0.1.2
RobinP122 Jun 27, 2024
b9c822e
Set valence_core crate to latest version (0.1.7)
RobinP122 Jun 27, 2024
0c63186
Merge pull request #10 from AIBlockOfficial/feature_db
BHouwens Jun 27, 2024
a7f14d0
Removed Valence Market plugin. Will be reintroduced at a later stage.
RobinP122 Jun 27, 2024
c724091
Moved db crate from valence_core to Valence app. This simplifies the …
RobinP122 Jun 27, 2024
1c17f39
Bump up version to 0.1.2
RobinP122 Jun 27, 2024
f84b503
Set valence_core crate to latest version (0.1.7)
RobinP122 Jun 27, 2024
faf0549
Removed Valence Market plugin. Will be reintroduced at a later stage.
RobinP122 Jun 27, 2024
550e84e
Added del_data handler to route
RobinP122 Jun 27, 2024
e839a7a
Fixed bug in db get_data & del_data implementation cleanup
RobinP122 Jul 2, 2024
524f3fa
Updated generic errors
RobinP122 Jul 2, 2024
e012358
Cleanup and update valence_core version containing new APIErrorType e…
RobinP122 Jul 4, 2024
8883660
Update README.md
RobinP122 Jul 4, 2024
91a6ac5
Merge branch 'develop' into feature_delete
RobinP122 Jul 4, 2024
c1acc96
Merge pull request #11 from AIBlockOfficial/feature_delete
RobinP122 Jul 4, 2024
b49ac2b
Bump up version to v0.1.3
RobinP122 Jul 4, 2024
26634d6
Fix tests and warning.
RobinP122 Jul 12, 2024
a090697
FIx for tests
BHouwens Aug 20, 2024
93204d2
Revert to 0.1.2
BHouwens Aug 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/code-scan.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
name: code-scan

on:
push:
branches:
- main
pull_request:

jobs:
scan-code:
uses: aiblockofficial/workflows/.github/workflows/scan-code.yml@main
28 changes: 28 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: build-scan

on:
push:
branches:
- main
- develop

permissions:
contents: read
packages: write
security-events: write

env:
CARGO_TERM_COLOR: always

jobs:
build:
uses: aiblockofficial/workflows/.github/workflows/build.yml@main
with:
REGISTRY: ${{ vars.REGISTRY }}
REPOSITORY: ${{ vars.REPOSITORY }}
scan-image:
uses: aiblockofficial/workflows/.github/workflows/scan-image.yml@main
secrets: inherit
needs: build
with:
IMAGE: ${{ vars.REGISTRY }}/${{ vars.REPOSITORY }}:${{ github.sha }}
16 changes: 16 additions & 0 deletions .github/workflows/manual-build-and-push.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: manual-build-and-push

permissions:
contents: read
packages: write
security-events: write

env:
CARGO_TERM_COLOR: always

jobs:
build:
uses: aiblockofficial/workflows/.github/workflows/build.yml@main
with:
REGISTRY: ${{ vars.REGISTRY }}
REPOSITORY: ${{ vars.REPOSITORY }}
17 changes: 17 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: build-and-push

on:
release:
types: [published]

permissions:
contents: read
packages: write
security-events: write

jobs:
build:
uses: aiblockofficial/workflows/.github/workflows/build.yml@main
with:
REGISTRY: ${{ vars.REGISTRY }}
REPOSITORY: ${{ vars.REPOSITORY }}
6 changes: 6 additions & 0 deletions .trivyignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Accept the risk


# Ignore misconfigurations
# https://avd.aquasec.com/misconfig/dockerfile/general/avd-ds-0001/
AVD-DS-0001
5 changes: 2 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "valence"
version = "0.1.1"
version = "0.1.2"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
Expand All @@ -23,5 +23,4 @@ tracing = "0.1.37"
tracing-subscriber = "0.3.17"
tracing-futures = "0.2.3"
warp = "0.3.5"
valence_core = "0.1.3"
valence_market = "0.1.0"
valence_core = "0.1.8"
9 changes: 2 additions & 7 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -31,19 +31,14 @@ RUN cargo build --target x86_64-unknown-linux-musl --release
####################################################################################################
## Final image
####################################################################################################
FROM alpine
FROM cgr.dev/chainguard/glibc-dynamic:latest

# Import from builder.
COPY --from=builder /etc/passwd /etc/passwd
COPY --from=builder /etc/group /etc/group
USER nonroot

WORKDIR /valence

# Copy our build
COPY --from=builder /valence/target/x86_64-unknown-linux-musl/release/valence ./
COPY --from=builder /valence/config.toml ./

# Use an unprivileged user.
USER valence:valence

CMD ["/valence/valence"]
34 changes: 29 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,9 @@
<a href="#getting-started">Getting Started</a>
<ul>
<li><a href="#prerequisites">Prerequisites</a></li>
<li><a href="#running-the-server">Running The Server</a></li>
</ul>
<li><a href="#installation">Installation</a></li>
<li><a href="#running-the-server">Running the server</a></li>
</ul>
</li>
<li>
<a href="#how-it-works">How it Works</a>
Expand All @@ -44,6 +45,7 @@
<ul>
<li><a href="#set_data">set_data</a></li>
<li><a href="#get_data">get_data</a></li>
<li><a href="#del_data">del_data</a></li>
</ul>
</li>
<li><a href="#further-work">Further Work</a></li>
Expand Down Expand Up @@ -89,7 +91,7 @@ docker build -t valence .

..

### 🏎️ Running
### 🏎️ Running the server

To use the server as is, you can simply run the following in the root folder of the repo:

Expand Down Expand Up @@ -133,14 +135,17 @@ Sets data in the Redis instance and marks it for pending retrieval in the server
}
```

The body of the `set_data` call would contain the data being exchanged:
The body of the `set_data` call would contain the `value_id` for that entry and the `data` being exchanged :

```json
{
"data_id": "EntryId"
"data": "hello Bob"
}
```

`data_id` is required and allows for mutiple entries under one address. If the `data_id` value is the same as an existing entry for that address, it is updated. If the `data_id` is unique it will be added to the hashmap for that address

The headers that Alice sends in her call will be validated by the Valence, after which they'll be stored at Bob's address for his later retrieval using the `get_data` call.

..
Expand All @@ -152,12 +157,31 @@ Gets pending data from the server for a given address. To retrieve data for Bob,
[
{
"address": "76e...dd6", // Bob's public key address
"public_key": "a4c...e45" // Bob's public key corresponding to his address
"signature": "b9f...506", // Bob's signature of the public key
}
]
```

If `data_id` is provided in the request (`get_data/[value_id]`), the specific entry associated to that id is retrieved. If no `data_id` is provided, the full hashmap is retrieved.

Again, the Valence will validate the signature before returning the data to Bob.

##### **<img src="https://img.shields.io/badge/DEL-FF0000" alt="DEL"/> `del_data`**
Delete pending data from the server for a given address. To delete data for Bob, he only has to supply his credentials in the call header:

```json
[
{
"address": "76e...dd6", // Bob's public key address
"public_key": "a4c...e45" // Bob's public key corresponding to his address
"signature": "b9f...506", // Bob's signature of the public key
}
]
```

If `data_id` is provided in the request (`del_data/[value_id]`), the specific entry associated to that id is deleted. If no `data_id` is provided, the full hashmap is deleted.

Again, the Valence will validate the signature before returning the data to Bob.

**For best practice, it's recommended that Alice and Bob encrypt their data using their private keys, before exchanging it with each other.** This ensures that the data exchange is E2E encrypted, and that the Valence maintains no knowledge of the data's content.
Expand All @@ -171,7 +195,7 @@ Again, the Valence will validate the signature before returning the data to Bob.
- [x] Match public key to address for `get_data` (resolved by using address directly for retrieval)
- [ ] Add a rate limiting mechanism
- [x] Set Redis keys to expire (handle cache lifetimes)
- [ ] Handle multiple data entries per address
- [x] Handle multiple data entries per address
- [ ] Add tests

<p align="left">(<a href="#top">back to top</a>)</p>
Expand Down
10 changes: 5 additions & 5 deletions config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@ extern_port = 3030
cache_url = "redis://0.0.0.0"
cache_port = "6379"
cache_password = ""
db_protocol = "mongodb://"
db_url = "0.0.0.0"
db_protocol = "mongodb://127.0.0.1"
db_url = ""
db_port = "27017"
db_user = "root"
db_password = "example"
db_user = ""
db_password = ""
body_limit = 4096
cache_ttl = 600 # cache lifetime in seconds

# Plug-in options
market = true
market = false
57 changes: 32 additions & 25 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
version: '3'

networks:
app-tier:
driver: bridge

services:
cache:
image: redis:6.2.6-alpine
networks:
- app-tier
restart: always
ports:
- 6379:6379
Expand All @@ -10,37 +17,37 @@ services:
command: redis-server --save 20 1 --loglevel warning
volumes:
- cache:/data/cache
container_name: cache
db:
image: mongo
image: bitnami/mongodb:7.0.5
networks:
- app-tier
restart: always
ports:
- 27017:27017
expose:
- 27017
environment:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: example
volumes:
- db:/data/db
container_name: db
valence:
build: .
image: valence
ports:
- 3030:3030
expose:
- 3030
restart: always
depends_on:
- cache
- db
links:
- cache
- db
volumes:
- ${PWD}/logs:/app/logs
container_name: valence
# environment:
# MONGODB_ROOT_USER: root
# volumes:
# - db:/data/db
# valence:
# build: .
# image: valence
# networks:
# - app-tier
# ports:
# - 3030:3030
# expose:
# - 3030
# restart: always
# depends_on:
# - cache
# - db
# # links:
# # - cache
# # - db
# volumes:
# - ${PWD}/logs:/app/logs
volumes:
cache:
driver: local
Expand Down
Loading
Loading