From Spice

Contents 1 Marshalling layer for compatibility 2 Summary 3 Owner 4 Current Status 5 Description

Marshalling layer for compatibility

Summary

Currently, spice reads and writes directly from the network without any interemediary marshalling layer. This means that there is really no way to easily centralize any checking we will do from the network. It also means all the internal types used in the networking are frozen and can't be changed.

In addition, we want to be able to add a compatibility layer for those times that we need to change the protocol in an incompatible fashion, which is only possible if the marshalling is split out and in a replaceable way.

Owner

• Alex Larsson

Current Status

• Targeted Release: spice-0.6

Description

When spice reads and writes a message it just treats the internal message structures as a bunch of bytes and send it as-is on the network socket. This has several disadvantages:

• The validation of all the data in the message (are offsets inside message, are the lenghts right, etc) is not done in a centralized place, but rather everywhere in the code where the message is used. This is hard to review and easy to get wrong.

• The structures used internally to represent operations (for instance drawing operations in the tree) are direct copies from the network data, and thus are frozen forever since these are defined by the protocol. This is problematic as we can't change the internals around like we want for spice-internal reasons. For instance, we want to change the order of the fields of SpiceRect internally so that it matches the pixman rectangle type, allowing us to skip conversions in many places.

• Since the messages are frozen as per the above we can't easily add optional arguments to them. For instance when Surfaces were added to spice we added a destination surface field to every drawing operation, which is highly incompatible. However, if we had a marshalling layer that separated the on-network format of the drawing operations and the internal one it would be easy to add such a field and just always set it to zero during demarshalling, enabling backwards compatibility with older spice servers.

• All the on-wire message structures are packed since they need to be compact and binary compatible between compilers. This leads to portability and efficiency problems when used as internal structures, as many fields are unaligned.

• All integers are defined to be in little-endian format on the network. However, if the client is big endian we need to convert this everywhere. Without a centralized demarshalling this is a lot of work.

• Always writing a full structure to the network means there are some optimizations wrt network size that we can't do. For instance, if some field is not used (for instance the mask) we could just set a flag somewhere and then not send this data at all. Additionally we can get rid of all the 64bit offsets we're currently using in the messages and just send the data inline where the offset was before.

The marshalling layer is based on a machine-readable description of the spice protocol written in a custom language. This description is fed into a code generator during the build which produces marshalling and demarshalling code to be linked into the client and server. As a nice side effect this description is an accurate documentation of the protocol.

The implementation will happen in several stages:

1. Write description of current (unstable) protocol and parser for it
2. Generate code for demarshalling, demarshalling the network format into the current spice message structures
3. Integrate demarshalling into the client (and some in server)
4. Generate code for marshalling structs to the network format
5. Integrate marshalling into the server (and some in client)
6. At this point the network format is isolated from the spice internals and we can start changing the protocol description to make the network protocol more efficient.
7. Write a description of the previous version of the network protocol for backwards compatibility
8. In client, if the server is an older major version, switch to the generated code for the older protocol version

Additionally, when this is done and the PCI compatibility layer is finished the spice message structures are internal only, and we can start changing the representation of them to better match our needs. We at least want to do this:

• Make structs not packed
• Convert offsets to pointers
• Make SpiceRect and pixman_box32_t compatbile (change field order)
• Make Clip references in drawing commands be pixman_region32_t pointers.