Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

typed binary snapshot compression / delta compression #2

Merged
merged 32 commits into from
Nov 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
fbb4cc3
Adds initial `binary-bitstream-buffer` plugin
Marak Nov 11, 2023
851125c
Adds initial tests for binary bitstream buffer
Marak Nov 11, 2023
4c7c2ae
Adds tests for message encode / decode
Marak Nov 11, 2023
1a22a2e
Adds additional schema properties
Marak Nov 11, 2023
5f491ac
Adds string support
Marak Nov 11, 2023
263ea6f
Unifies use of `id` and auto-id across all entities
Marak Nov 11, 2023
6eb1fe9
Adds back missing entity creation call
Marak Nov 11, 2023
38510fd
Updates edge server to use `bbb`
Marak Nov 11, 2023
d3ae6d9
Refactors data encoding pipeline into separate functions
Marak Nov 11, 2023
08d8bee
Enables deltaCompression by default
Marak Nov 12, 2023
68f7a2a
deltaCompression is now scoped per player
Marak Nov 12, 2023
b6165bd
Adds float2Int float encoding
Marak Nov 12, 2023
0202cb8
Switches Float64 to float encoded Int32
Marak Nov 12, 2023
f93ec09
Refactors bbb to accept nested schema definitions
Marak Nov 12, 2023
797e98d
Switches Float64 to float encoded Int32
Marak Nov 12, 2023
0a042bc
Refactors bbb into generic codec
Marak Nov 12, 2023
b075514
Adds tests and new bbb4
Marak Nov 13, 2023
bb33b66
Refactors bbb to use Visitor pattern
Marak Nov 14, 2023
37fc5a4
Adds api wrapper, removes legacy code
Marak Nov 14, 2023
ccb89f1
All tests passing
Marak Nov 14, 2023
9322776
Updates Client/Server to latest bbb API
Marak Nov 14, 2023
edf079a
Adds benchmark for snapshot compression
Marak Nov 14, 2023
c1695a4
Updates Client / Server to use `@yantra-core/supreme` encoder
Marak Nov 14, 2023
f3327b2
Adds `Schema` plugin
Marak Nov 15, 2023
30bf071
Adds missing `Schema` plugin
Marak Nov 15, 2023
75a2d3a
Refactors `game.entities` from object to Map
Marak Nov 15, 2023
855afe6
Enables delta compression by default
Marak Nov 15, 2023
c45c1f9
Updates CloudFlare edge server to new APIs
Marak Nov 15, 2023
baf5c4c
Renames `gametick` enum action -> `GAMETICK`
Marak Nov 16, 2023
9e2b5f0
Updates tests to new API
Marak Nov 16, 2023
451289b
Adds `@msgpack/msgpack` to dev deps
Marak Nov 16, 2023
fd2f68f
Updates Phaser to latest API
Marak Nov 16, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions mantra-client/client.js
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ let game = new Game({
isClient: true,
mouse: false,
physics: 'matter', // 'matter', 'physx'
graphics: ['phaser'], // 'babylon', 'css', 'phaser'
graphics: ['babylon'], // 'babylon', 'css', 'phaser'
collisions: true,
camera: 'follow',
options: {
Expand All @@ -105,6 +105,7 @@ game
.use(new plugins.Bullet())
.use(new plugins.Block())

game.use(new plugins.Schema());

game.use(new plugins.InputLegend());
game.use(new plugins.PingTime());
Expand Down Expand Up @@ -208,7 +209,7 @@ if (mode === 'online') {
depth: 200,
position: {
x: 0,
y: -500
y: -1000
},
});

Expand Down
2 changes: 1 addition & 1 deletion mantra-client/public/clock.html
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@
socket.onmessage = function (event) {
count++;
let data = JSON.parse(event.data);
if (data.action !== 'gametick') {
if (data.action !== 'GAMETICK') {
console.log('WebSocket message received:', event.data);
$('#output').append('<p>' + event.data + '</p>');
} else {
Expand Down
2 changes: 1 addition & 1 deletion mantra-client/public/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@
socket.onmessage = function (event) {
count++;
let data = JSON.parse(event.data);
if (data.action !== 'gametick') {
if (data.action !== 'GAMETICK') {
console.log('WebSocket message received:', event.data);
$('#output').append('<p>' + event.data + '</p>');
} else {
Expand Down
4 changes: 3 additions & 1 deletion mantra-edge/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,9 @@
"cloud-test": "vitest"
},
"dependencies": {
"@msgpack/msgpack": "^3.0.0-beta2",
"@yantra-core/mantra": "*",
"matter-js": "^0.19.0"
"matter-js": "^0.19.0",
"protobufjs": "^7.2.5"
}
}
101 changes: 82 additions & 19 deletions mantra-edge/src/index.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
import { Game, plugins } from '../../mantra-game/Game.js';
import deltaEncoding from '@yantra-core/mantra/plugins/snapshots/SnapShotManager/deltaEncoding.js';
import deltaCompression from '@yantra-core/mantra/plugins/snapshots/SnapShotManager/deltaCompression.js';
import { encode } from "@msgpack/msgpack";

let config = {};
config.msgpack = true;
config.bbb = false;
config.protobuf = false; // see: https://github.com/protobufjs/protobuf.js/pull/1941
config.deltaCompression = true; // only send differences between int values

// let lastMessageTime = 0;
const MAX_BUFFER_SIZE = 100;
Expand Down Expand Up @@ -34,12 +42,11 @@ export class Ayyo {

// Use Plugins to add systems to the game
this.gameLogic
// .use(new plugins.Schema())
.use(new plugins.Bullet())
.use(new plugins.Block())
.use(new plugins.Block({ MIN_BLOCK_SIZE: 200 }))
.use(new plugins.Collision())
.use(new plugins.Border({ autoBorder: false }))


}

async initialize() {
Expand All @@ -56,7 +63,8 @@ export class Ayyo {
width: 2000,
});

/* TODO: better data compression / client-side prediction
/* TODO: better data compression / client-side prediction */

// Your game start-up logic goes here
this.gameLogic.createEntity({
type: 'BLOCK',
Expand All @@ -68,8 +76,7 @@ export class Ayyo {
y: -500
},
});
*/


}

async reset() {
Expand All @@ -94,27 +101,34 @@ export class Ayyo {

// Generate a unique player ID for this connection
// remark: replace with global entity counter that is INT
const playerEntityId = 'player_' + Math.random().toString(36).substr(2, 9);
console.log('handleSession', playerEntityId)
// Store the connected player with its WebSocket
this.connectedPlayers[playerEntityId] = websocket;

//const playerEntityId = 'player_' + Math.random().toString(36).substr(2, 9);
let playerEntityId;
const playerName = 'player_' + Math.random().toString(36).substr(2, 9);
let ent;
// Create the player entity in the game logic
try {
// Create a new player with a unique ID
this.gameLogic.createEntity({
id: playerEntityId,
ent = this.gameLogic.createEntity({
name: playerName,
type: 'PLAYER',
});
// this.gameLogic.systems.playerCreation.createPlayer(playerEntityId);
} catch (err) {
console.log("ERROR", err)
}


playerEntityId = ent.id;
// Store the connected player with its WebSocket
this.connectedPlayers[playerEntityId] = websocket;


try {
websocket.send(JSON.stringify({
action: 'assign_id',
playerId: playerEntityId
playerName: playerName,
playerId: ent.id

}));
} catch (err) {
console.log(err)
Expand All @@ -123,6 +137,7 @@ export class Ayyo {

// Check if a new ticker needs to be elected
if (!this.tickerId) {
// console.log("ELECT NEW TICKER", playerEntityId)
this.electNewTicker(playerEntityId);
}

Expand All @@ -134,6 +149,9 @@ export class Ayyo {
delete this.connectedPlayers[playerEntityId];
this.gameLogic.removeEntity(playerEntityId); // You need to implement this method in Game.js

if (config.deltaCompression) {
deltaCompression.resetState(playerEntityId);
}

if (playerEntityId === this.tickerId) {
this.tickerId = null; // Clear the tickerId
Expand All @@ -155,7 +173,7 @@ export class Ayyo {
// Perform the requested action based on the "action" property of the message
switch (message.action) {
case 'gameTick':

// console.log('got game tick')
this.bufferGameTick(message, playerEntityId);

if (this.tickerId !== playerEntityId) {
Expand All @@ -165,6 +183,7 @@ export class Ayyo {
break;

case 'player_input':
// console.log('ahhhh', playerEntityId, message.controls)
this.gameLogic.systems.entityInput.handleInputs(playerEntityId, { controls: message.controls });
break;

Expand Down Expand Up @@ -213,6 +232,7 @@ export class Ayyo {
// Elect a new ticker when a player connects or when the current ticker disconnects
electNewTicker(newTickerId) {
this.tickerId = newTickerId;
// console.log("electNewTicker", this.connectedPlayers, newTickerId)
const tickerWs = this.connectedPlayers[newTickerId];
if (tickerWs) {
tickerWs.send(JSON.stringify({
Expand Down Expand Up @@ -261,20 +281,63 @@ export class Ayyo {
this.gameLogic.gameTick();

// Send updates to clients
this.sendPlayerSnapshots();
try {
this.sendPlayerSnapshots();
} catch (err) {
console.log('ERROR this.sendPlayerSnapshots()')
console.log(err.message);

}
}

sendPlayerSnapshots() {

Object.keys(this.connectedPlayers).forEach(playerId => {

const playerSnapshot = this.gameLogic.getPlayerSnapshot(playerId);
const lastProcessedInput = this.gameLogic.lastProcessedInput[playerId];

if (playerSnapshot) {
try {


let deltaEncoded = deltaEncoding.encode(playerId, playerSnapshot);
if (deltaEncoded) {
const ws = this.connectedPlayers[playerId];
ws.send(JSON.stringify({ action: 'gametick', snapshot: deltaEncoded, lastProcessedInput: lastProcessedInput }));
let newSnapshot;

if (config.deltaCompression) {
newSnapshot = deltaCompression.compress(playerId, deltaEncoded);
} else {
newSnapshot = playerSnapshot;
}

if (config.protobuf) {
// Create a new message
let _message = this.gameLogic.Message.fromObject({ id: newSnapshot.id, action: 'GAMETICK', state: newSnapshot.state, lastProcessedInput: lastProcessedInput }); // or use .fromObject if conversion is necessary
// Encode a message to an Uint8Array (browser) or Buffer (node)
let buffer = game.Message.encode(_message).finish();
const ws = this.connectedPlayers[playerId];
ws.send(buffer);
} else if (config.bbb) {
let BBB = new bbb();
// TODO: add data encoding layer here
let bbbEncoded = BBB.encodeMessage({ id: newSnapshot.id, action: 'GAMETICK', state: newSnapshot.state, lastProcessedInput: lastProcessedInput });

const ws = this.connectedPlayers[playerId];
ws.send(bbbEncoded.byteArray);
} else if (config.msgpack) {
const ws = this.connectedPlayers[playerId];
let msg = { id: newSnapshot.id, action: 'GAMETICK', state: newSnapshot.state, lastProcessedInput: lastProcessedInput };
let buffer = encode(msg);
ws.send(buffer);

}
else {
const ws = this.connectedPlayers[playerId];
ws.send(JSON.stringify({ id: newSnapshot.id, action: 'GAMETICK', state: newSnapshot.state, lastProcessedInput: lastProcessedInput }));
}


}
} catch (err) {
console.log(err);
Expand Down Expand Up @@ -311,7 +374,7 @@ export class Ayyo {
const playerSnapshot = this.gameLogic.getPlayerSnapshot(playerId);
if (playerSnapshot) {
const ws = this.connectedPlayers[playerId];
ws.send(JSON.stringify({ action: 'gametick', snapshot: playerSnapshot }));
ws.send(JSON.stringify({ action: 'GAMETICK', state: playerSnapshot.state }));
}
});
return new Response('Game ticked', { status: 200 });
Expand Down
8 changes: 5 additions & 3 deletions mantra-game/Component/Component.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,9 @@ class Component {
}

// After setting the value, update the corresponding entity in the game.entities
if (this.game && this.game.entities && this.game.entities[entityId]) {
this.game.entities[entityId][this.name] = this.get(entityId);
if (this.game && this.game.entities && this.game.entities.has(entityId)) {
let existing = this.game.entities.get(entityId);
existing[this.name] = this.get(entityId);
}

}
Expand Down Expand Up @@ -60,12 +61,13 @@ class Component {
delete this.data[key];
}


/* Removed 11/15/23, do we need to do this, or will entity update by reference?
// After removing the component data, update the entity in game.entities if necessary
const entityId = Array.isArray(key) ? key[0] : key;
if (this.game && this.game.entities && this.game.entities[entityId]) {
delete this.game.entities[entityId][this.name];
}
*/
}
}

Expand Down
5 changes: 5 additions & 0 deletions mantra-game/Game.js
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ class Game {

this.snapshotQueue = [];

this.tick = 0;

// Game settings
this.width = width;
this.height = height;
Expand Down Expand Up @@ -252,6 +254,9 @@ class Game {
// TODO: move to separate function
// Loads external js script files sequentially
loadScripts(scripts, finalCallback) {
if (this.isServer) {
return;
}
const loadScript = (index) => {
if (index < scripts.length) {
let script = document.createElement('script');
Expand Down
8 changes: 6 additions & 2 deletions mantra-game/System/SystemsManager.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,13 @@ class SystemsManager {
if (this.systems.has(systemName)) {
throw new Error(`System with name ${systemName} already exists!`);
}
// TODO: add this later
// Remark: Defaulting all plugins as EE has been disabled for now
// Remark: We need to piece meal this / granularize it so that the wildcard regex
// registers the new system in event emitter
eventEmitter.bindClass(system, systemName)
// the best option we can do is have the event binding be default off
// then setup defaults for the required default systems ( possibly listening event / etc )
// eventEmitter.bindClass(system, systemName)

// binds system to local instance Map
this.systems.set(systemName, system);
// binds system to game.systems scope for convenience
Expand Down
1 change: 1 addition & 0 deletions mantra-game/browser-shim.js
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ MANTRA.plugins = {
InputLegend: require('./plugins/input-legend/InputLegend.js').default,
PingTime: require('./plugins/ping-time/PingTime.js').default,
SnapshotSize: require('./plugins/snapshot-size/SnapshotSize.js').default,
Schema: require('./plugins/schema/Schema.js').default,
CurrentFPS: require('./plugins/current-fps/CurrentFPS.js').default,
Keyboard: require('./plugins/keyboard/Keyboard.js').default,
Lifetime: require('./plugins/lifetime/Lifetime.js').default,
Expand Down
2 changes: 1 addition & 1 deletion mantra-game/lib/gameTick.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ let lastTick = Date.now();
let hzMS = 16.666; // 60 FPS

function gameTick() {

this.tick++;
// Calculate deltaTime in milliseconds
let now = Date.now();
let deltaTimeMS = now - lastTick; // Delta time in milliseconds
Expand Down
1 change: 1 addition & 0 deletions mantra-game/lib/schema/make-proto.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
protoc --js_out=import_style=commonjs,binary:. messageSchema.proto
Loading