redis

Quick reference

# scan for keys without the blocking `keys` command, to a max of 10
scan 0 MATCH foo:* COUNT 10 TYPE hash

# get Set members for quicker lookup versus scan or keys commands
smembers actors

# check if in set
sismember actors "Brad"

# check connectivity
redis-cli ping
PONG

# Check if keys exists
EXISTS zone:lemon
(integer) 1 # true

# get Epoch time
TIME

# get values of a Hash ( without Keys )
HVALS foo:white

# get keys of a Hash ( without Values )
HKEYS foo:white

Server and cli

# start server
redis-server /usr/local/etc/redis.conf &

# shutdown server
echo -e "AUTH ${REDIS_USER} ${REDIS_PSWD}\nshutdown" | redis-cli

# Delete all keys
redis-cli FLUSHALL

# connect when URL include Port
redis-cli -u redis://${REDIS_URL}

# connect when ACL set
redis-cli -u redis://${REDIS_URL} --user ${REDIS_USER} --pass ${REDIS_PSWD} --no-auth-warning

# start redis-client with TLS enabled
redis-cli --tls --cacart <path to server certificate file >

# Test you can write a record
echo -e "AUTH ${REDIS_USER} ${REDIS_PSWD}\nset foo bar" | redis-cli -u redis://${REDIS_URL}

# Test if Auth is switched on
(printf "PING\r\nPING\r\nPING\r\n"; sleep 1) | nc localhost 6379

Deeper reference

# scan for keys, without using the blocking `keys` command
scan 0 MATCH *foo* COUNT 10

#SCAN
Iterates the set of keys in the currently selected Redis database.

#SSCAN
Iterates elements of Sets types.

#HSCAN
Iterates fields of Hash types and their associated values.

#ZSCAN
Iterates elements of Sorted Set types and their associated scores.

<https://www.alexedwards.net/blog/working-with-redis>

# authenticate and list all keys
echo -e "AUTH foo ${REDIS_PASSWORD}\nkeys *" | redis-cli

# authenticate and whoami
echo -e "AUTH foo ${REDIS_PASSWORD}\nacl whoami" | redis-cli

# authenticate and list ACL users
echo -e "AUTH foo ${REDIS_PASSWORD}\nacl list" | redis-cli 

# Get location of conf file, Uptime, Redis version 
info server

# Useful stuff like db0:keys=1,expires=0,avg_ttl=0
info keyspace

# Number of connected clients. max clients
info clients

# See which users are set
acl list

# debug commands are hitting Redis server
monitor

# get databases
redis-cli config get databases

# list all keys [ blocking operation ]
keys *

# list keys starting with `d`
keys d*

# Set key SC-90 and string 0080
set sc-90 0080

# Get key
get sc-90
"0080"

# Multiple set
MSET key1 "Hello" key2 "World"

# Set list
rpush sc-01 "-" "1654066738" "Foobar" 

# Get list
lrange sc-01 0 3
1) "-"
2) "1654066738"
3) "Foobar"


# set with expiry
setex foo 5 bar
OK

get foo
"bar"

get foo
(nil)

# get all Ticket user
zrange foo:ticketAAA:usage 0 -1

# get score of single user
ZMSCORE foo:ticketAAA:usage "yves"
1) "2"

# increment score of a single user
ZINCRBY foo:ticketAAA:usage 5 "bob"

# get fresh score of single user
ZMSCORE foo:ticketAAA:usage "bob"

# Time to live
ttl foo
(integer) -2

# json

JSON.SET object . '{"foo": "bar", "ans": 42}'
OK
JSON.GET object
"{"foo":"bar","ans":42}"
JSON.GET object .ans
"42"

Security

Model

Security Model outakes:

• Designed to be accessed by trusted clients inside trusted environments.
• Not exposed to the Internet.
• Use the redis.conf file in production.
• Always run Redis as a dedicated non-privileged user.
• Enforce least privilege with ACLs.

Access Control List in Redis 6.0+

# Redis 6.0 User
# type INFO to get location of Redis conf file
/usr/local/etc/redis.conf
# comment out: requirepass foobared
# DANGER: the default user 
user default on >pickle allcommands allkeys

# turn off default user
acl setuser default off

# Common dev user ( can't access ACL or Config )
# Syntax: "on" equals "can login".  '>' means passwords.
ACL SETUSER foouser on >pickle allcommands -@dangerous +acl|whoami allkeys

# view @dangerous roles
acl cat dangerous

# If user can run Config commands, can change the box to something else ( like a Crypto Miner )
config get *

# New admin user ( can admin config, acl but not get/set keys )
ACL SETUSER rick on >pickle +@admin

# Stop a new admin deleting all databases and keys:
ACL SETUSER newadminuser on >pickle +@admin -FLUSHDB -FLUSHALL

# New user who can only GET and SET with limited keys
ACL SETUSER cacheservice on >pickleme +set +get ~cache:*

# delete all data
ACL FLUSHALL

# find local ACL file and uncomment
▶ cat /usr/local/etc/redis.conf | grep aclfile  
aclfile /usr/local/etc/users.acl

Why Least Privilege with limited Roles ?

# Attacker looks for Keys
> SCAN 0 COUNT 3
1) "3"
2) 1) "foo:approved:14-June-2022"
   2) "foo:approved:13-June-2022"
   3) "foo:approved:15-June-2022"

# KEYS get all hashes with match
# A sophiscated attacker would use SCAN over KEYS
# Blocks until complete as can be long running
KEYS foo:ticket*:details
1) "foo:ticketB:details"
2) "foo:ticketA:details"
3) "foo:ticketC:details"

# MONITOR 
 # Echo back what is sent to Redis

KEYS foo:ticket*:details

# Same as above but also returns the database number
scan 0 MATCH foo:ticket*:details COUNT 1000
1) "0"
2) 1) "foo:ticketB:details"
   2) "foo:ticketA:details"
   3) "foo:ticketC:details"

# Attacker wants to know Type of the Key
> type foo:approved:14-June-2022
zset

# Attacker can then dump from Key
> zrange foo:approved:14-June-2022 0 -1
1) "dave"
2) "yves"
3) "alice"
4) "bob"

# or if it was a Hash
HGETALL foo:approved:14-June-2022
< lots of sensitive objects >

# Backdoor user
ACL SETUSER badfoo on ><pswd> +@all ~*

# Migrate
MIGRATE remote-server 8000 "" 0 5000 keys

Legacy passwords

Changing the global password on a running redis-server:

# a single global password prior to Redis 6
config set requirepass p@ssw0rd

# authenticate
auth p@ssw0rd

Or persist the change in the conf file:

# type INFO to get location of Redis conf file
/usr/local/etc/redis.conf

# uncomment the line
requirepass foobared

# Now auth via
auth foobared
OK

Data Structures

Mindset shift & speed

How do you get all Redis keys based on a value inside the Hash i.e. all People with Blue Eyes ? You could scan and then parse for a value, with code. How about an alternative ? When you add the Hash add the key to a Set based on the value you care about. For example:

# add
SADD foo:approved:today bob
SADD foo:approved:tomorrow bob
SADD foo:approved:today alice

# get
smembers foo:approved:today
1) "alice"
2) "bob"

# check if in set
sismember foo:approved:today "alice"

# number of set member
scard foo:approved:today

# remove a member
pop, srem or smove

This keeps many operations at the Redis level. Of course, you could get all data from Redis, manipulate it in code, then put Set all the data in Redis. But this stop you thinking about "what querie do I need to write to extract the data I care about?". You have already done it. You added the Hash to a Set that can be quickly retrieved with code gymanistics.

Sorted set

There was a lot of requests for an expiring Sorted Set feature. But there were strong arguments not to implement it: redis/redis#135

export TODAY_START_EPOCH=$(date -j -f "%b %d %Y %T" "Jun 06 2022 00:00:00" "+%s") && \
    export TOMORROW_START_EPOCH=$(date -j -f "%b %d %Y %T" "Jun 07 2022 00:00:00" "+%s")  && \
    export TODAY_ADD2_START_EPOCH=$(date -j -f "%b %d %Y %T" "Jun 08 2022 00:00:00" "+%s")

# print dates
env | grep EPOCH 
TODAY_START_EPOCH=1654470000
TOMORROW_START_EPOCH=1654556400
TODAY_ADD2_START_EPOCH=1654642800

# add Score of TODAY_START_EPOCH to Content
redis-cli ZADD foo:approved ${TODAY_START_EPOCH} bob 
redis-cli ZADD foo:approved ${TODAY_START_EPOCH} alice 
redis-cli ZADD foo:approved ${TOMORROW_START_EPOCH} dave
redis-cli ZADD foo:approved ${TODAY_ADD2_START_EPOCH} yves

# or set via the redis-cli pipe
# redis_zset_users.txt
MULTI
ZADD foo:approved 1654470000 bob
ZADD foo:approved 1654470000 alice
ZADD foo:approved 1654556400 dave
ZADD foo:approved 1654642800 Yves
EXEC
cat redis_zset_users.txt  | redis-cli --pipe

# Get people from June 22 and June 23 with scores
redis-cli --raw ZRANGEBYSCORE foo:approved ${TODAY_START_EPOCH} ${TOMORROW_START_EPOCH} WITHSCORES
1655766000
bob
1655766000
dave
1655852400

# Get people from June 22 and June 23 withOUT scores
redis-cli --raw ZRANGEBYSCORE foo:approved ${TODAY_START_EPOCH} ${TOMORROW_START_EPOCH}           
alice
bob
dave

# get all from June 22 2022 onwards
redis-cli --raw ZRANGEBYSCORE foo:approved ${TODAY_START_EPOCH} +inf
alice
bob
dave
yves

redis-cli --raw ZRANGEBYSCORE foo:approved ${TOMORROW_START_EPOCH} +inf
dave
yves

redis-cli --raw ZRANGEBYSCORE foo:approved -inf ${TODAY_START_EPOCH}   
alice
bob


# Limited results to top returned record
ZRANGE foo:approved (1 +inf BYSCORE LIMIT 1 1
1) "bob"

# Check if person is member of Sorted Set
redis-cli --raw ZSCORE foo:approved bob                         
1655766000

redis-cli --raw ZSCORE foo:approved yves
1655938800

redis-cli --raw ZSCORE foo:approved non_existent_usr
<nil>

# Count sorted set
redis-cli --raw ZCARD foo:approved        
4

# Count by from Today until End
redis-cli --raw ZCOUNT foo:approved ${TODAY_START_EPOCH} +inf           
4

# ZINCRBY will add the member "bob" and increment the score
# ZADD will replace "bob" with a new "bob"
redis-cli --raw ZINCRBY foo:approved 1 "bob"

ZINCRBY foo:approved 1 bob
"1"
ZINCRBY foo:approved 1 bob
"2"
ZINCRBY foo:approved 1 bob
"3"

ZADD foo:approved 100 bob
(integer) 0

ZINCRBY foo:approved 1 bob
"101"

# send multiple transactions
# redis_cmds.txt
MULTI
INCR foo:total
INCR foo:total:june:2022
HMSET foo:ticketA:details name A desc "A description" created_date 1654325648
HMSET foo:ticketB:details name B desc "B description" created_date 1654325655
HMSET foo:ticketC:details name C desc "C description" created_date 1654325766
ZADD foo:ticketA:usage 1 bob
ZADD foo:ticketA:usage 1 alice
ZADD foo:ticketA:usage 1 yves
ZINCRBY foo:ticketA:usage 9 bob
ZINCRBY foo:ticketA:usage 2 alice
ZADD foo:ticketB:usage 1 yves
ZINCRBY foo:ticketB:usage 3 alice
EXEC


▶ cat redis_cmds.txt | redis-cli --pipe
All data transferred. Waiting for the last reply...
Last reply received from server.
errors: 0, replies: 14

▶ redis-cli --raw zrange foo:ticketAAA:usage 0 -1
alice
bob


# Check if hash exists
HEXISTS foo:ticketA:details name
(integer) 1

# Get all fields inside a hash
HKEYS foo:ticketA:details
1) "name"
2) "desc"
3) "created_date"

# Get count of all rows in Sorted Set
ZCOUNT foo:ticketA:usage -inf +inf
3

# highest scorer with score
ZREVRANGE foo:ticketA:usage 0 0 withscores
1) "bob"
2) "10"

# same as previous
ZRANGE foo:ticketA:usage -1 -1 WITHSCORES