Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sysadmin guide to the documentation #1865

Merged
merged 23 commits into from
Jul 9, 2021
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/sysadmin/install.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
## Install

See [here](/docs/getting-started/install.html)

## Set up users

After system is set up, look at [here](/docs/sysadmin/tsctl.html) to add users.
130 changes: 130 additions & 0 deletions docs/sysadmin/troubleshooting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
## Troubleshooting playbook

- is it only affecting one user?
- is it only affecting one sketch / timeline?
- can the issue be reproduced on demo.timesketch.org?
- Any output on Chrome console?
- Try a different device / clear caches from browser
- Look in nginx / webserver logs
- Look in Timesketch / celery worker logs
- Look in ES logs
- Look in postgres logs

## UX issues

To troubleshoot UX issues, the first look should go to the Chrome developer console and look for any output / errors.

To raise issues related to UX, please provide the following:

- Copy out the full error message(s) from Chrome Console
- The URL (without your local IP or Domain)
- What was clicked before it happend
- Any entries in the server side logs (see below)?
- Share a screenshot if possible
- If possible details about the event / sketch
- Was it an imported plaso file or CSV or JSONL?
- Was the data imported via Web or API client?

## Docker

To get all your docker instances, run:

```shell
docker container list
```

Which will show something like

```shell
b756f334d281 us-docker.pkg.dev/osdfir-registry/timesketch/dev:latest "/docker-entrypoint.…" 8 days ago Up 2 days 127.0.0.1:5000->5000/tcp timesketch-dev
7768635b4798 us-docker.pkg.dev/osdfir-registry/timesketch/notebook:latest "jupyter notebook" 8 days ago Up 2 days 127.0.0.1:8844->8844/tcp, 8899/tcp notebook
6d2e7da9453c justwatch/elasticsearch_exporter:1.1.0 "/bin/elasticsearch_…" 8 days ago Up 2 days 9114/tcp es-metrics-exporter
78443764330b grafana/grafana:latest "/run.sh" 8 days ago Up 2 days 127.0.0.1:3000->3000/tcp grafana
51a576407ad2 docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2 "/tini -- /usr/local…" 8 days ago Up 2 days 9200/tcp, 9300/tcp elasticsearch
bf85b40ed003 redis:6.0.10-alpine "docker-entrypoint.s…" 8 days ago Up 2 days 6379/tcp redis
f78f8b1f13d1 postgres:13.1-alpine "docker-entrypoint.s…" 8 days ago Up 2 days 5432/tcp postgres
```

If one of these is not up, you might need to troubleshoot docker.

## Troubleshooting Database Schema Changes

See [docs/learn/server-admin](docs/learn/server-admin#troubleshooting-database-schema-changes)

## Issues importing plaso file

- Which plaso version is installed on the Timesketch server?
- Which plaso version was used to create the plaso file?
- Is the issue for both web upload and import_client?

Try to run the following in the Docker container after the file was uploaded (but not successfull imported):

```shell
pinfo.py $FILENAME
```

Should give the following error message:

```shell
2020-08-19 14:40:48,912 [ERROR] (MainProcess) PID:568 <pinfo_tool> Format of storage file: $FILENAME not supported
```

## logs

All of those are subject to change depending on your operating system.

### nginx / webserver

```shell
/var/log/nginx/access
```

Good starter is to run the following grep:

```shell
grep "http_code:500" /var/log/nginx/access.log #to get all Error 500
```

If you started the webserver with docker, look in the console where you started:

```shell
docker exec -it $CONTAINER_ID gunicorn --reload -b 0.0.0.0:5000 --log-file - --timeout 600 timesketch.wsgi:application
```

Or run the following where ```$CONTAINER_ID``` is the ID from your ```timesketch-dev```or ```timesketch``` docker image

```shell
docker logs $CONTAINER_ID
```

### Timesketch / celery worker

See your console output if you started the workers with:

```shell
docker exec -it $CONTAINER_ID celery -A timesketch.lib.tasks worker --loglevel=debug
```

### ElasticSearch

```shell
/var/log/elasticsearch
```

Or run the following where ```$CONTAINER_ID``` is the ID from your ```elasticsearch``` docker image

```shell
docker logs $CONTAINER_ID
```

### Postgress

```shell
/var/log/postgresql/
```

Or run the following where ```$CONTAINER_ID``` is the ID from your ```postgres``` docker image

```shell
docker logs $CONTAINER_ID
```
File renamed without changes.
File renamed without changes.