Tutorial

Table of Contents

• A simple example
• A portable example
• Custom structures
• Custom implementations

A simple example

The code provided in a_simple_example.cpp demonstrates the simplest possible use case for this library - a basic structure is defined, serialized, and deserialized.

#include <fstream>
#include <cassert>

#include <framework/serializable.hpp>
#include <framework/serializable/mutators/stl_string.hpp>

int main ()
{
    using ::framework::serializable::inline_object;
    using ::framework::serializable::stl_null_string;
    using ::framework::serializable::value;
    
    // Define an object type
    using object = inline_object <
        value <NAME("Field 1"), int32_t>,
        value <NAME("Field 2"), double>,
        value <NAME("Field 3"), stl_null_string>>;

    // Create an object
    object o1 {1, 2.0, "Hello World!"};

    // Write the object to a file
    assert(write(o1, std::ofstream("filename")));

    // Read the object from a file
    object o2;
    assert(read(std::ifstream("filename"), o2));

    // Check the result
    assert(o1 == o2);

    return 0;
}

"filename" output:

0000000: 0100 0000 0000 0000 0000 0040 4865 6c6c  ...........@Hell
0000010: 6f20 576f 726c 6421 00                   o World!.

A portable example

The example above is not portable - an output file produced on one architecture may not be interpreted properly when read on another. To correct this, the endianness of serialized data must be defined - this library provides the mutator types little_endian and big_endian for this purpose.

Note

Unlike Boost.Serializable, which embeds endianness information into the stream (or Archive), this library specifies endianness of individual values. This is largely a reflection of the different intended use cases of the two libraries - there is no guarantee that a protocol will use consistent byte (or bit) packing across any logical message block. Applications may recover this behaviour, if desirable, using custom container types or by redefining the default serialization of fundamental types, provided in base_types.hpp.

This library distinguishes between various types of constructs in a specification - containers types and value types, for example, operate on serializable objects, while mutators types operate directly on underlying data. A mutator type can be thought of as "wrapping" the serialization of an underlying type (ie: another mutator, a string, a vector, a tuple, a fundamental type, ...), defining or altering how that type is serialized. Mutator types generally use the following basic pattern:

// mutator_type <internal_type(s), underlying_type>
mutator_type <int>
mutator_type <mutator_type <int>>
mutator_type <int, std::vector <int>>
...

Note

Some of the mutators defined by this library restrict the underlying type; little_endian and big_endian, for example, require an underlying fundamental type.

A simple portable example using the little_endian and big_endian mutators to serialize data is provided in a_portable_example.cpp:

#include <fstream>
#include <cassert>

#include <framework/serializable.hpp>
#include <framework/serializable/mutators/stl_string.hpp>

int main ()
{
    using ::framework::serializable::stl_wstring;
    using ::framework::serializable::little_endian;
    using ::framework::serializable::big_endian;
    using ::framework::serializable::value;
    
    // Define an object type
    using object = ::framework::serializable::inline_object <
        // Store the first field using little endian byte ordering
        value <NAME("Field 1"), little_endian <int32_t>>,

        // Store the second field using big endian byte ordering
        value <NAME("Field 2"), big_endian <uint64_t>>,

        // Store the wide string as a size-delimited container, storing the size and 
        // characters (see stl_wstring documentation) using little-endian byte ordering.
        value <NAME("Field 3"), stl_wstring <little_endian <uint32_t>>>>;

    // Create an object
    object o1 {1, 2, L"Hello World!"};

    // Write the object to a file
    assert(write(o1, std::ofstream("filename")));

    // Read the object from a file
    object o2;
    assert(read(std::ifstream("filename"), o2));

    // Check the value
    assert(o1 == o2);

    return 0;
}

"filename" output:

0000000: 0100 0000 4000 0000 0000 0000 0c00 0000  ....@...........
0000010: 4800 0000 6500 0000 6c00 0000 6c00 0000  H...e...l...l...
0000020: 6f00 0000 2000 0000 5700 0000 6f00 0000  o... ...W...o...
0000030: 7200 0000 6c00 0000 6400 0000 2100 0000  r...l...d...!...

Custom structures

The preceding examples used inline_object to define the layout of a structure. This template is convenient for it's brevity - constructors, comparison operators, and common accessor methods are all defined. User defined structures may want to include some or all of these features - to illustrate how this performed in this library, the construction of a comparable structure is covered in some detail here.

To begin, we present a simple C structure we would like to transform into a serializable object:

struct object
{
    int x;
    double y;

    double foo () { return x + y; }
};

The structure above (neglecting foo for the moment) may be reformulated as follows:

struct object : serializable <object,
    value <NAME("x"), int>,
    value <NAME("y"), double>>
{
};

The above is sufficient to allow the serializable library to read/write object, interact with fields through the free functions defined in common.hpp, and compare the object through the free functions defined in comparable.hpp. For example, the following sets and compares two such objects:

object o1, o2;

set <NAME("x")> (o1, 1);
set <NAME("y")> (o1, 2.0);
set <NAME("x")> (o2, 1);
set <NAME("y")> (o2, 3.0);

assert(less_than(o1, o2));

The above is clearly unwieldy - construction and comparison are needlessly verbose. To address this, we begin by adding a forwarding constructor:

struct object : serializable <object,
    value <NAME("x"), int>,
    value <NAME("y"), double>>
{
    template <typename... Args>
    object (Args&&... args)
        : object::serializable_base(std::forward <Args> (args)...)
    {
    }
};

Note

The serializable base class here defines how the above arguments are mapped to the associated base class constructors. Generally speaking, one parameter in the above pack is forwarded to each named value in an object, in the order it appears in the object's specification.

The serializable_base typedef is provided by serializable to allow object to easily refer to this base without duplicating the template parameters provided to serializable.

With the above constructor in place, we may reformulate the sample code above more efficiently as:

object o1{1, 2.0}, o2{1, 3.0};
assert(less_than(o1, o2));

Next, common comparison operators need to be added. The defaults (member-wise comparison) are sufficient here and as such these operators may be added trivially through the use of the comparable template:

struct object : 
    comparable <object>,
    serializable <object,
        value <NAME("x"), int>,
        value <NAME("y"), double>>
{
    template <typename... Args>
    object (Args&&... args)
        : object::serializable_base(std::forward <Args> (args)...)
    {
    }
};

The above provides object with common comparison operators through the use of the Barton-Nackman trick. This leads to the final reformulation of the above sample:

object o1{1, 2.0}, o2{1, 3.0};
assert(o1 < o2);

Finally, we return to the sample function foo. Objects similar to the above should generally access members of it's base classes directly rather than relying on free functions like get and set - this allows the class access to protected members of a value type's implementation. The get_base template facilitates this by providing a means of associating a value type's identifier with the corresponding base class type, as demonstrated below:

struct object : 
    comparable <object>,
    serializable <object,
        value <NAME("x"), int>,
        value <NAME("y"), double>>
{
    template <typename... Args>
    object (Args&&... args)
        : object::serializable_base(std::forward <Args> (args)...)
    {
    }

    double foo ()
    {
        using x = typename get_base <object, NAME("x")>::type;
        using y = typename get_base <object, NAME("y")>::type;
    
        return x::get() + y::get();
    }
};

Again, the syntax above is rather unwieldy - the basic pattern is clearly required to retain protected access, however we can clean up the retrieval of the base class somewhat. In particular, a template alias may be used to reduce most of the redundancy above - the macro DEFINE_BASE_TEMPLATE is provided for exactly that purpose. This change leads to the final reformulation of object as follows:

struct object : 
    comparable <object>,
    serializable <object,
        value <NAME("x"), int>,
        value <NAME("y"), double>>
{
    DEFINE_BASE_TEMPLATE(object);

    template <typename... Args>
    object (Args&&... args)
        : object::serializable_base(std::forward <Args> (args)...)
    {
    }

    double foo ()
    {
        return base <NAME("x")>::get() + base <NAME("y")>::get();
    }
};

The above introduced nearly every convenience feature provided by inline_object with the exception of common accessor methods; their construction is neglected here. These methods introduce no new library specific information and as such the implementation of inline_object should suffice - see inline_object.hpp for more details. A simple example of a custom structure definition, together with an appropriate serialization test, is provided in custom_structures.cpp:

#include <fstream>
#include <cassert>

#include <framework/serializable.hpp>
#include <framework/serializable/mutators/stl_string.hpp>

using ::framework::serializable::comparable;
using ::framework::serializable::serializable;
using ::framework::serializable::value;

// Define a new structure
struct object :
    // Provides the standard comparison operations
    comparable <object>,

    // Define the object's specification
    serializable <object,
        value <NAME("Field 1"), int>,
        value <NAME("Field 2"), double>>
{
    // Forward constructor arguments to the base class
    template <typename... Args>
    object (Args&&... args)
        : object::serializable_base(std::forward <Args> (args)...)
    {
    }

    // Defines a convenient 'base' alias for get_base
    DEFINE_BASE_TEMPLATE(object);

    // Example method
    void foo ()
    {
        using x1 = base <NAME("Field 1")>;
        using x2 = base <NAME("Field 2")>;

        if (x1::get() > 0)
            x2::set(x2::get() + 1.0);
        else
            x2::set(x2::get() - 1.0);
    }
};

int main ()
{
    // Create an object
    object o1 {1, 2.0};

    // Write the object to a file
    assert(write(o1, std::ofstream("filename")));

    // Read the object from a file
    object o2;
    assert(read(std::ifstream("filename"), o2));

    // Check the value
    assert(o1 == o2);

    return 0;
}

Custom implementations

Value types provided in this library allow a final optional template parameter that may be used to override the type's default implementation. This may be desirable for numerous reasons - direct access to the underlying type may be required, accessors may need to be protected, invariants may need to be established, and so on. The examples provided in custom_implementation.cpp illustrate the usage of this parameter by providing two such implementations - the first establishes a simple invariant while the second widens an arbitrary underlying value type to an integer, where appropriate.

Note

The implementation override used here accepts a single typename parameter used to provide the structure with the information it may require from the value type's specification. For example, value provides it's implementation with the associated name, underlying type, derived class type, and so on - see value_implementation_wrapper for more details. Custom implementations are strongly encouraged to use these types to avoid effective duplication of portions of the object's specification.

#include <iostream>
#include <fstream>
#include <cassert>
#include <stdexcept>

#include <framework/serializable.hpp>
#include <framework/serializable/mutators/stl_string.hpp>

// Object implementation details
namespace object_impl
{
    // Example of a simple fixed-value object specific field.  This pattern could easily be
    // generalized to a template, but a one-off implementation for a specific object is
    // more illustrative here.
    template <typename T>
    struct version
    {
        // The accessor methods and constructors maintain the invariant version == 5

        typename T::value_type get () const
        { 
            return 5; 
        }

        void set (typename T::value_type const& x) const
        {
            if (x != 5)
                throw std::runtime_error("Invalid object: Version not supported");
        }

        protected:
            ~version () = default;
            version () = default;

            version (typename T::value_type const& x)
            {
                if (x != 5)
                    throw std::runtime_error("Invalid object: Version not supported");
            }

        // Note - this implementation does not use an underlying value type
    };
}

// Widen-value template - stores an underlying arithmetic value type as an integer if possible
template <typename T>
struct widen_value
{
    private:
        enum { widen = std::is_arithmetic <typename T::value_type>::value && sizeof(typename T::value_type) <= sizeof(int) };

        // Note - this implementation redefines the underlying value type
        typename std::conditional <
            widen,
            int,
            typename T::value_type
        >::type p_tValue;

    public:
        typename std::conditional <
            widen,
            typename T::value_type,
            typename T::value_type const&
        >::type get () const
        {
            return p_tValue;
        }

        void set (typename T::value_type x)
        {
            p_tValue = std::move(x);
        }

    protected:
        ~widen_value () = default;
        widen_value () = default;
    
        widen_value (typename T::value_type x)
            : p_tValue(std::move(x))
        {
        }
};

int main ()
{
    using ::framework::serializable::inline_object;
    using ::framework::serializable::value;
    
    // Define an object type
    using object = inline_object <
        value <NAME("Version"), uint8_t, object_impl::version>,
        value <NAME("Value"), short, widen_value>>;

    // Create an object
    object o1 {5, 2.0};

    o1.get <NAME("Value")> ();

    // Write the object to a file
    assert(write(o1, std::ofstream("filename")));

    // Read the object from a file
    object o2;
    assert(read(std::ifstream("filename"), o2));

    // Check the value
    assert(o1 == o2);

    std::cout << "Original bool size: " << sizeof(inline_object <value <NAME("x"), bool>>) << std::endl;
    std::cout << "Widened bool size: " << sizeof(inline_object <value <NAME("x"), bool, widen_value>>) << std::endl;

    return 0;
}

Todo:

• Introduce the differences between container types, value types, and mutator
• Add documentation for "custom_serialization" example
• Add documentation for "adding_mutator_types" example
• Add documentation for "adding_container_types" example
• Add documentation for "custom_serializable_implementation" example