Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server readme updates #611

Merged
merged 2 commits into from
Jun 22, 2022
Merged

Server readme updates #611

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
55a9a2a
server readme updates
walkah Jun 17, 2022
ea36f11
doc updates
walkah Jun 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions fission-web-server/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,23 @@ To mimic the full "fission stack" for local development, you can use the include
- `docker compose exec dns-auth pdnsutil create-zone runfission.test`
- `docker compose exec dns-auth pdnsutil create-zone fissionuser.test`
- `docker compose exec dns-auth pdnsutil create-zone fissionapp.test`
4. Point your local DNS resolver to localhost.
4. Create some initial DNS records:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With postgres being in docker compose, I believe you need something like the following somewhere in here after docker compose up.

docker compose exec postgres psql -U postgres
postgres=# create database web_api
postgres-# \q

Or with createdb instead.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With postgres being in docker compose, I believe you need something like the following somewhere in here after docker compose up.

If you're using the env.example from the repo it defines POSTGRES_DB - which will create the database on first run (see https://github.com/docker-library/docs/blob/master/postgres/README.md#postgres_db )

- `docker compose exec dns-auth pdnsutil add-record runfission.test. @ A "127.0.0.1"`
- `docker compose exec dns-auth pdnsutil add-record fissionuser.test. gateway A "127.0.0.1"`
- `docker compose exec dns-auth pdnsutil add-record fissionapp.test. gateway A "127.0.0.1"`
5. Point your local DNS resolver to localhost.
- on macOS: this is under System Preferences > Network > Advanced.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that for me I added 127.0.0.1 and also 192.168.1.1 (default DNS for me if none specified manually). Without this if the container is down I can't reach anything, like git pull will hang for example.

- on Linux: Add `nameserver 127.0.0.1` to `/etc/resolv.conf`

You can now build / run the haskell server. The included `server.yaml.example` is configured to work with the local docker setup.
6. Pin the CID for the new app placeholder:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this? The "new app placeholder"? What app and where did the CID come from? Also when I run this I get the following error.

$ docker compose exec ipfs ipfs pin add -r QmRVvvMeMEPi1zerpXYH9df3ATdzuB63R1wf3Mz5NS5HQN
service "ipfs" is not running container #1

Looks like that ipfs container isn't starting up the daemon or at least not keeping it running.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this? The "new app placeholder"? What app and where did the CID come from? Also when I run this I get the following error.

When you create an app from the CLI (via fission app register) we populate that with placeholder page (with a fission logo). When we get your environment far enough along to be able to register, you'd notice that app register will timeout (in my experience) trying to pull that CID from the network - we assume it's already pinned locally.

Looks like that ipfs container isn't starting up the daemon or at least not keeping it running.

docker compose logs ipfs could help illuminate what's going on here. My top two guesses are: 1) you have an ipfs daemon running already and there are port conflicts or 2) you're not using the docker-compose.yml from this branch (what's in main currently fails to start on M1 Macs specifically).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have this issue as well. Logs:

fission-web-server-ipfs-1  | Initializing daemon...
fission-web-server-ipfs-1  | go-ipfs version: 0.11.0-67220ed
fission-web-server-ipfs-1  | Repo version: 11
fission-web-server-ipfs-1  | System version: amd64/linux
fission-web-server-ipfs-1  | Golang version: go1.16.7
fission-web-server-ipfs-1  | 2022/06/22 15:34:49 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.
fission-web-server-ipfs-1  | 2022/06/22 15:34:49 [zeroconf] no suitable IPv4 interface: udp4: failed to join any of these interfaces: [{15 1500 eth0 02:42:ac:19:00:04 up|broadcast|multicast}]
fission-web-server-ipfs-1  | 2022/06/22 15:34:49 [zeroconf] no suitable IPv6 interface: udp6: failed to join any of these interfaces: [{15 1500 eth0 02:42:ac:19:00:04 up|broadcast|multicast}]
fission-web-server-ipfs-1  | 2022-06-22T15:34:49.623Z   ERROR   cmd/ipfs        ipfs/daemon.go:430    error from node construction: no supported interface
fission-web-server-ipfs-1  | 
fission-web-server-ipfs-1  | Error: no supported interface

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have this issue as well. Logs:

yeah, this is an issue with 0.11 ... i've bumped the docker-compose in this PR to use 0.13 (which fixes the issue) or there's a config workaround here: ipfs/kubo#8645 (comment)

`docker compose exec ipfs ipfs pin add -r QmRVvvMeMEPi1zerpXYH9df3ATdzuB63R1wf3Mz5NS5HQN`
7. Copy `addon-manifest.json.example` to `addon-manifest.json` and change `CHANGE_ME_TO_NUMBER` to a random number (e.g. 42)
8. Copy `server.yaml.example` to `server.yaml`
9. Build / install the server (if using nix-shell you can run `server-install`)
10. Run the server (if using nix-shell you can run `server-debug`)

You should now have a rest server running on port `1337` (test: http://runfission.test:1337/ping).

You can use the fission CLI to register users and deploy apps: `fission setup -R runfission.test:1337` (always pass the `-R runfission.test:1337` to commands).
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remove ~/.config/fission first. Then do the following.

»  fission -R runfission.test:1337 setup 
🌱 Setting up environment
🪐 Downloading managed IPFS for macOS
🕖🎛️  Configuring managed IPFS
🔑 Setting up keys
🏠 Do you have an existing account? [Y/n] n
Username: justin
Email: justin@justinjohnson.io
🚫 There was a server error. Please try again or contact Fission support at https://fission.codes

Server log is below, and I assume this is a similar issue to the pinning error due to ipfs not running.

2022-06-20 10:15:00.183934: [debug] 🐙🚰 Running IPFS request across cluster (streaming)
@(library/Fission/Web/Server/Types.hs:476:5)
2022-06-20 10:15:00.184007: [debug] 🐙🎬 Starting request to cluster node: http://localhost:5001
@(library/Fission/Web/Server/Types.hs:481:7)
2022-06-20 10:15:00.184852: [error] IPFS Daemon error: ConnectionError (HttpExceptionRequest Request {
  host                 = "localhost"
  port                 = 5001
  secure               = False
  requestHeaders       = [("Accept","application/json;charset=utf-8")]
  path                 = "/api/v0/pin/add"
  queryString          = "?arg=Qmc5m94Gu7z62RC8waSKkZUrCCBJPyHbkpmGzEePxy2oXJ&progress=true"
  method               = "POST"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}
 (ConnectionFailure Network.Socket.connect: <socket: 20>: does not exist (Connection refused)))
@(library/Fission/Web/Server/Error.hs:59:3)

Copy link

@justindotpub justindotpub Jun 20, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I believe the issue with ipfs not starting is due to the network being missing from the docker compose.

  ipfs:
    image: ipfs/go-ipfs:v0.11.0
    ports:
      - "4001:4001"
      - "5001:5001"
      - "8080:8080"
    volumes:
      - ./data/ipfs:/data/ipfs
    networks:
      - default

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this...

ipfs/kubo#8645

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, modified ./data/ipfs/config with the workaround at ipfs/kubo#8645 (comment) and the daemon starts up now.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get the following when trying to set up a new account after I have everything working.

»  fission -R runfission.test:1337 setup
🌱 Setting up environment
🪐 Downloading managed IPFS for macOS
🕖🎛️  Configuring managed IPFS
🔑 Setting up keys
🏠 Do you have an existing account? [Y/n] n
Username: justin
Email: justin@justinjohnson.io
🚫 It looks like that account already exists. Please try again or contact Fission support at https://fission.codes

Logs have the following

2022-06-20 11:09:50.723688: [debug] (SQL) SELECT "id","public_key","exchange_keys","email","username","role","active","verified","data_root","data_root_size","heroku_add_on_id","secret_digest","inserted_at","modified_at" FROM "user" WHERE "username"=?; [PersistText "justin"]
@(<unknown>:0:0)
2022-06-20 11:09:50.726376: [debug] (SQL) SELECT "id","public_key","exchange_keys","email","username","role","active","verified","data_root","data_root_size","heroku_add_on_id","secret_digest","inserted_at","modified_at" FROM "user" WHERE "username"=?; [PersistText "justin"]
@(<unknown>:0:0)
2022-06-20 11:09:50.728027: [debug] (SQL) SELECT "id","public_key","exchange_keys","email","username","role","active","verified","data_root","data_root_size","heroku_add_on_id","secret_digest","inserted_at","modified_at" FROM "user" WHERE "public_key"=?; [PersistText "URQ3zxrfFFJRyhcWAdpcqSST3UuYVecW+DgIsq6UR/4="]
@(<unknown>:0:0)
2022-06-20 11:09:50.732241: [error] Username justin already exists
@(library/Fission/Web/Server/Error.hs:59:3)

But I don't see the user in the db

»  docker compose exec postgres psql -U postgres -d web_api
psql (11.16)
Type "help" for help.

web_api=# select * from user;
   user   
----------
 postgres
(1 row)

I have no postgres running locally, and only one in docker, so I think I'm in the right db.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no postgres running locally, and only one in docker, so I think I'm in the right db.

try SELECT * FROM "user" (quotes important) ... it looks like it's pulling from the PG user db ... unfortunate name conflict :-/

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per out Zoom call, I'm assigning this back to you. (I believe it was assigned to me just for review purposes.) Additional items we discussed...

  • peer id being wrong in server.yaml, need to grab from the ipfs node running in docker
  • send_in_blue api key for email verification, will actually send email as no stubbed out
  • ipfs not starting due to MDNS issue on M1, workaround at Zeroconf networking issue (Docker For Mac, M1 Max, 0.11) ipfs/kubo#8645 (comment), but sounded like fixed in latest ipfs in docker, and I wasn't running your branch because I thought it was just README updates

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@justincjohnson I've pushed the updates. 👍 👎 ?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:


#### Local DNS troubleshooting

Expand Down
2 changes: 1 addition & 1 deletion fission-web-server/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ services:
- "53:53/udp"
- "8082:8082" # HTTP API for the recursor
ipfs:
image: ipfs/go-ipfs:v0.11.0
image: ipfs/go-ipfs:v0.13.0
ports:
- "4001:4001"
- "5001:5001"
Expand Down