Purple Martians
Technical Code Descriptions
Netgame - State Correction
Overview
Game state variables
Global variables for state correction
How a client gets a new state from the server
How a client applies the new state
How the server sends a new state dif to a client
Packets used for state correction
Overview
The state correction method perodically sends the entire game state from the server to the client.
This fixes any drift or loss of sync that may have occured with the game move sync method.
The entire game state is over 100,000 bytes.
To pass this large chunk of data across the network, two compression methods are used.
The first method is a state and dif scheme, to detect only the data that has changed from the last state.
The second method uses the zlib compression library to compress the dif from the first method.
Using zlib only resulted in 5% to 10% compression. Using both gets 1% - 2%.
WIth 1024 byte packets, I can usually send the entire game state in 1 or 2 packets.
When a state dif is made without a previous state to base it on it is larger and needs 4-8 packets.
This happens for the initial state, or rarely if the base state is lost somehow.
Game state variables
The game state variables take 104640 bytes.
// 40,000 bytes for the 100 x 100 block level
extern int l[100][100];
// 1440 bytes for 8 players
extern struct player players[NUM_PLAYERS];
// 4000 bytes for lifts:
extern struct lift lifts[NUM_LIFTS];
// 40,000 bytes for 500 items
extern int item[500][16]; // item ints
extern al_fixed itemf[500][4]; // item fixeds
// 19,200 bytes for 100 enemies
extern int Ei[100][32]; // enemy ints
extern al_fixed Efi[100][16]; // enemy fixeds
// 104,640 bytes total
To work with this huge chuck of data as a whole, I move it to a char array like this:
char tmp[104640];
game_vars_to_state(tmp);
void game_vars_to_state(char * b)
{
int size = 0, offset = 0;
offset += size; size = sizeof(players); memcpy(b+offset, players, size);
offset += size; size = sizeof(Ei); memcpy(b+offset, Ei, size);
offset += size; size = sizeof(Efi); memcpy(b+offset, Efi, size);
offset += size; size = sizeof(item); memcpy(b+offset, item, size);
offset += size; size = sizeof(itemf); memcpy(b+offset, itemf, size);
offset += size; size = sizeof(lifts); memcpy(b+offset, lifts, size);
offset += size; size = sizeof(l); memcpy(b+offset, l, size);
}
This is how the char array is put back into the game variables:
void state_to_game_vars(char * b)
{
int size = 0, offset = 0;
size = sizeof(players); memcpy(players, b+offset, size); offset += size;
size = sizeof(Ei); memcpy(Ei, b+offset, size); offset += size;
size = sizeof(Efi); memcpy(Efi, b+offset, size); offset += size;
size = sizeof(item); memcpy(item, b+offset, size); offset += size;
size = sizeof(itemf); memcpy(itemf, b+offset, size); offset += size;
size = sizeof(lifts); memcpy(lifts, b+offset, size); offset += size;
size = sizeof(l); memcpy(l, b+offset, size); offset += size;
}
I call these large char arrays 'states'.
The client uses 3 and the server uses 16 (2 for each player).
Global variables for state correction
These are the global data stuctures used for the game state correction algorithm:
#define STATE_SIZE 104640
// server's copies of client states
extern char srv_client_state[8][2][STATE_SIZE];
extern int srv_client_state_frame_num[8][2];
// local client's states
extern char client_state_buffer[STATE_SIZE]; // buffer for building compressed dif from packet pieces
extern int client_state_buffer_pieces[16]; // to mark packet pieces as received
extern char client_state_base[STATE_SIZE]; // last ack state
extern int client_state_base_frame_num; // last ack state frame_num
extern char client_state_dif[STATE_SIZE]; // uncompressed dif
extern int client_state_dif_src; // uncompressed dif src frame_num
extern int client_state_dif_dst; // uncompressed dif dst frame_num
How a client gets a new state from the server
A client receives 'stdf' packets from the server. Each packet contains pieces of a compressed dif state.
These packets have up to 1000 bytes of data each and are put into 'client_state_buffer' at the appropriate offset.
When all the pieces have been received, 'client_state_buffer' is uncompressed into 'client_state_dif'.
The source and destination frame of 'client_state_dif' are updated to mark it as valid.
int process_stdf_packet(void)
{
int src = PacketGet4ByteInt();
int dst = PacketGet4ByteInt();
int seq = PacketGet1ByteInt();
int max_seq = PacketGet1ByteInt();
int sb = PacketGet4ByteInt();
int sz = PacketGet4ByteInt();
memcpy(client_state_buffer + sb, packetbuffer+22, sz); // put the piece of data in the buffer
client_state_buffer_pieces[seq] = dst; // mark it with destination frame_num
int complete = 1; // did we just get the last packet? (yes by default)
for (int i=0; i< max_seq; i++)
if (client_state_buffer_pieces[i] != dst) complete = 0; // no, if any piece not at latest frame_num
if (complete)
{
// decompress client_state_buffer to client_state_dif
uncompress((Bytef*)client_state_dif, sizeof(client_state_dif), (Bytef*)client_state_buffer, sizeof(client_state_buffer));
// mark dif data with new src and dst
client_state_dif_src = src;
client_state_dif_dst = dst;
}
}
How a client applies the new state
Every frame the client checks to see if it has a dif that matches both these conditions:
- destination frame matches current frame
- source frame matches the client's stored base state
If these conditions are met:
- the dif is applied to the base state
- the base state is used to overwrite the game variables
- the base state's frame number is updated to the current frame
- an acknowledge packet is sent to the server
void client_apply_diff()
{
if (frame_num == client_state_dif_dst) // current frame_num is dif destination
if (client_state_base_frame_num == client_state_dif_src) // stored base state matches dif source
{
apply_state_dif(client_state_base, client_state_dif, STATE_SIZE); // apply dif to base state
state_to_game_vars(client_state_base); // copy modified base state to game_vars
client_state_base_frame_num = frame_num; // update client base frame_num
Packet("stak"); // send acknowledge to server with last state sucessfully applied
PacketPut1ByteInt(p);
PacketPut1ByteInt(dif_corr);
PacketPut4ByteInt(frame_num);
ClientSend(packetbuffer, packetsize);
}
}
How the server sends a new state dif to a client
The server keeps 2 states for each client.
The 1st state:
srv_client_state[p][0]
srv_client_state_frame_num[p][0]
- is the base (or last acknowledged) state for that client
- the server uses it to make new difs based on that state
- the client also keeps a copy of it and uses it to apply the difs it gets from the server
The 2nd state:
srv_client_state[p][1]
srv_client_state_frame_num[p][1]
- is used as the destination state when making a dif for a client
- is kept by the server until acknowledged by the client, or a new dif is made
Here is how the server makes a dif to send to a client:
- the current server game state is copied to the 2nd state
// get current state
game_vars_to_state(srv_client_state[p][1]);
// set dif dest to current frame_num
srv_client_state_frame_num[p][1] = frame_num;
- a dif is made by subtracting the 2nd state from the 1st state
// make a new dif from base and current
get_state_dif(srv_client_state[p][0], srv_client_state[p][1], dif, STATE_SIZE);
void get_state_dif(char *a, char *b, char *c, int size)
{
for (int i=0; i < size; i++)
c[i] = a[i] - b[i];
}
- the dif is compressed
// compress dif to cmp
uLongf destLen= sizeof(cmp);
compress2((Bytef*)cmp, (uLongf*)&destLen, (Bytef*)dif, sizeof(dif), zlib_cmp);
int cp = destLen;
- the compressed dif is broken into 1000 byte pieces and sent with 'stdf' packets to the client
- each 'stdf' packet has a header with all the information needed to re-assemble, decompress and apply
// break compressed dif into smaller pieces
int start_byte = 0;
int num_packets = (cmp_size / 1000) + 1;
for (int packet_num=0; packet_num < num_packets; packet_num++)
{
int packet_data_size = 1000; // default size
if (start_byte + packet_data_size > cmp_size) packet_data_size = cmp_size - start_byte; // last piece is smaller
Packet("stdf");
PacketPut4ByteInt(srv_client_state_frame_num[p][0]); // src frame_num
PacketPut4ByteInt(srv_client_state_frame_num[p][1]); // dst frame_num
PacketPut1ByteInt(packet_num);
PacketPut1ByteInt(num_packets);
PacketPut4ByteInt(start_byte);
PacketPut4ByteInt(packet_data_size);
memcpy(packetbuffer+packetsize, cmp+start_byte, packet_data_size);
packetsize += packet_data_size;
ServerSendTo(packetbuffer, packetsize, players1[p].who, p);
start_byte+=1000;
}
After the 'stdf' packets are sent, the server expects an acknowledge packet 'stak' from the client.
When an acknowledgement is received, the 2nd state is copied to the 1st state and becomes the new base.
If no acknowledgement is received, the next dif will use the same base as the last one. (but with a new destination)
if(PacketRead("stak"))
{
int ack_pc = PacketGet4ByteInt(); // client acknowledged up to here
if (ack_pc == srv_client_state_frame_num[p][1]) // check to make sure we have a copy of acknowledged state
{
// acknowledged state is out new base state
memcpy(srv_client_state[p][0], srv_client_state[p][1], STATE_SIZE); // copy 1 to 0
srv_client_state_frame_num[p][0] = ack_pc; // new frame_num
}
else // we don't have a copy of acknowledged state !!! // set new base failed!
{
// reset base to zero
memset(srv_client_state[p][0], 0, STATE_SIZE);
// set base frame_num to 0
srv_client_state_frame_num[p][0] = 0;
}
}
Resetting base state
Resetting the base to zero means setting all 104,640 bytes to zero, and setting the frame_num to 0;
On the server, a client's base state is used to make new difs to send to that client.
The client applies difs from the server to its local copy of the same base state, and acknowledges them.
If the server does not have a copy of the state acknowledged by the client, it resets the client's base to zero.
The next dif sent will be based on state 0 and will be considerably larger than a regular dif.
If a client receives a dif that has base state zero, it resets its base to zero.
At the start of a level, the first dif the server sends will be from base zero.
After that it should be quite rare for the base to be reset.
It can happpen if difs are being sent with too short of a delay. (stdf_freq set too low)
A client gets a dif, applies it, updates its base, and sends an acknowledge.
Before server gets the acknowledge, the server sends a new dif, overwriting the dest used to make the last one.
Then when the server does get the acknowledge, it no longer has the neeeded copy of the client's new base.
Packets used for state correction
Packet: 'stdf'
description: 'state dif'
direction: server to client
- 4 bytes (source frame_num)
- 4 bytes (destination frame_num)
- 1 byte (packet sequence num)
- 1 byte (packet sequence total)
- 4 bytes (data start byte)
- 4 bytes (data size)
- up to 1000 bytes of compressed dif data
Packet: 'stak'
description: 'state dif acknowledge'
direction: client to server
- 1 byte (player)
- 1 byte (diff corr)
- 4 bytes (client frame_num)