博客

monster.fbs

FlatBuffers 使用

.

来指定嵌套的名称空间/包。


root_type

声明序列化数据的根表（或结构）。

除了基本类型外，

Monster

中包含一个表和一个结构体。

表是在 FlatBuffers 中定义对象的主要方式，由名称（此处为

Monster

）和字段列表组成。每个字段都有一个名称、类型和可选的默认值（如果省略，则默认为

0

或

NULL

）。

表中每个字段都是可选的：它不必出现在线路表示中，您可以选择忽略每个对象的字段。因此，您可以灵活地添加字段，而不必担心数据膨胀。此设计也是 FlatBuffer 的向前和向后兼容机制。请注意：

• 在 schema 文件中只能在表定义的末尾添加新字段。旧数据仍能正确读取，并在读取时为您提供默认值。旧代码只会忽略新字段。如果想要灵活地对 schema 中的字段使用任何顺序，则可以手动分配 id（与 Protocol Buffers 类似），请参见下面的 id 属性。
• 您无法从 schema 中删除不再使用的字段，但是您可以停止将它们写入数据中以达到几乎相同的效果。另外，您可以像上面的示例中那样将它们标记为已弃用，这将防止在生成的 C++中生成访问器，从而强制不再使用该字段。（小心：这可能会破坏代码！）。
• 您可以更改字段名和表名，如果您不介意同步更改代码重命名。

Struct 没有一个字段是可选的（因此也没有默认值），并且不能添加或弃用字段。只能包含标量或其他结构。对于简单的对象，如果您确定不会进行任何更改（如示例中

Vec3

非常明确）。结构使用的内存比表少，访问速度更快（它们始终以内联方式存储在其父对象中，并且不使用虚拟表）。


union

与枚举的许多属性相同，但是使用表名而不是常量的名称。声明的一个联合字段中可以包含对任何这些类型的引用，并且还会生成带有后缀

_type

的隐藏字段，其中包含相应的枚举值，使您能够在运行时知道要转换为哪种类型。

联合体是一种能够将多种消息类型作为 FlatBuffer 发送的好方法。请注意，由于联合字段实际上是两个字段，因此它必须始终是表的一部分，本身不能成为 FlatBuffer 的根。

下面代码生成的结构体在

MyGame::Sample

命名空间中。

namespace MyGame.Sample;

enum Color:byte { Red = 0, Green, Blue = 2 }

union Equipment { Weapon } // Optionally add more tables.

struct Vec3 {
  x:float;
  y:float;
  z:float;
}

table Monster {
  pos:Vec3;
  mana:short = 150;
  hp:short = 100;
  name:string;
  friendly:bool = false (deprecated);
  inventory:[ubyte];
  color:Color = Blue;
  weapons:[Weapon];
  equipped:Equipment;
  path:[Vec3];
}

table Weapon {
  name:string;
  damage:short;
}

root_type Monster;

sample_binary.cpp

FlatBuffers 中有内部定义的

Vector

和

String

。标量可以直接保存，其他成员都需要独立构建后再加入所隶属的结构中。

FlatBufferBuilder

是用于容纳创建 FlatBuffer 所需数据的帮助程序类。


FlatBufferBuilder::FlatBufferBuilder


要序列化数据，通常调用生成代码中的一个

Create*()

函数，该函数依次调用

StartTable/PushElement/AddElement/EndTable

序列或内置的

CreateString

/

CreateVector

函数。这样做是以深度优先顺序在根部建立一棵树。

Finish()

将缓冲区打包以备传输。

FlatBufferBuilder::CreateString

在缓冲区中存储一个字符串，该字符串可以包含任何二进制数据。返回字符串开始处的缓冲区中的偏移量。

  // Build up a serialized buffer algorithmically:
  flatbuffers::FlatBufferBuilder builder;

  // First, lets serialize some weapons for the Monster: A 'sword' and an 'axe'.
  auto weapon_one_name = builder.CreateString("Sword");
  short weapon_one_damage = 3;

  auto weapon_two_name = builder.CreateString("Axe");
  short weapon_two_damage = 5;

CreateWeapon

构建出

Weapon

对象，返回对象的偏移。

weapons_vector

为偏移构成的数组。


FlatBufferBuilder::CreateVector

将偏移数组存储起来。

  // Use the `CreateWeapon` shortcut to create Weapons with all fields set.
  auto sword = CreateWeapon(builder, weapon_one_name, weapon_one_damage);
  auto axe = CreateWeapon(builder, weapon_two_name, weapon_two_damage);

  // Create a FlatBuffer's `vector` from the `std::vector`.
  std::vector<flatbuffers::Offset<Weapon>> weapons_vector;
  weapons_vector.push_back(sword);
  weapons_vector.push_back(axe);
  auto weapons = builder.CreateVector(weapons_vector);

接受数组的

FlatBufferBuilder::CreateVector

将数组值存储起来并返回数组偏移

inventory

。


FlatBufferBuilder::Finish()

通过写入根偏移量完成缓冲区的序列化。

  // Second, serialize the rest of the objects needed by the Monster.
  auto position = Vec3(1.0f, 2.0f, 3.0f);

  auto name = builder.CreateString("MyMonster");

  unsigned char inv_data[] = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };
  auto inventory = builder.CreateVector(inv_data, 10);

CreateMonster

通过

MonsterBuilder

来构造出

Monster

对象。

  // Shortcut for creating monster with all fields set:
  auto orc = CreateMonster(builder, &position, 150, 80, name, inventory,
                           Color_Red, weapons, Equipment_Weapon, axe.Union());

  builder.Finish(orc);  // Serialize the root of the object.

解析缓冲区。


FlatBufferBuilder::GetBufferPointer

获取序列化的缓冲区（在调用

Finish()

之后）。


GetMonster

调用

GetRoot

获得

Monster

对象的指针。

  // We now have a FlatBuffer we can store on disk or send over a network.

  // ** file/network code goes here :) **
  // access builder.GetBufferPointer() for builder.GetSize() bytes

  // Instead, we're going to access it right away (as if we just received it).

  // Get access to the root:
  auto monster = GetMonster(builder.GetBufferPointer());

Monster::hp

和

Monster::mana

通过

Table::GetField

获得字段值。


Monster::name

通过

Table::GetPointer

获得

String

对象的指针。


Monster::pos

调用

Table::GetStruct

得到

Vec3

结构体的指针。

  // Get and test some scalar types from the FlatBuffer.
  assert(monster->hp() == 80);
  assert(monster->mana() == 150);  // default
  assert(monster->name()->str() == "MyMonster");

  // Get and test a field of the FlatBuffer's `struct`.
  auto pos = monster->pos();
  assert(pos);
  assert(pos->z() == 3.0f);
  (void)pos;

Monster::inventory

通过

Table::GetPointer

获取

Vector

对象的指针。


Vector::Get

获取指定索引的元素。


Monster::weapons

通过

Table::GetPointer

获取

Vector

对象的指针

weps

。后者记录了每个

Weapon

对象的偏移，从而能够调用

Weapon::name

和

Weapon::damage

。

  // Get a test an element from the `inventory` FlatBuffer's `vector`.
  auto inv = monster->inventory();
  assert(inv);
  assert(inv->Get(9) == 9);
  (void)inv;

  // Get and test the `weapons` FlatBuffers's `vector`.
  std::string expected_weapon_names[] = { "Sword", "Axe" };
  short expected_weapon_damages[] = { 3, 5 };
  auto weps = monster->weapons();
  for (unsigned int i = 0; i < weps->size(); i++) {
    assert(weps->Get(i)->name()->str() == expected_weapon_names[i]);
    assert(weps->Get(i)->damage() == expected_weapon_damages[i]);
  }
  (void)expected_weapon_names;
  (void)expected_weapon_damages;

Monster::equipped_type

通过

Table::GetField

获得

Equipment

枚举值。

  // Get and test the `Equipment` union (`equipped` field).
  assert(monster->equipped_type() == Equipment_Weapon);
  auto equipped = static_cast<const Weapon *>(monster->equipped());
  assert(equipped->name()->str() == "Axe");
  assert(equipped->damage() == 5);
  (void)equipped;

  printf("The FlatBuffer was successfully created and verified!\n");

FlatBufferBuilder::CreateString

FlatBufferBuilder::NotNested

检查构建过程中是否存在嵌套。


FlatBufferBuilder::PreAlign

向

buf_

填0，预先进行对齐。


FlatBufferBuilder::PushBytes

将数据填入

buf_

中。


FlatBufferBuilder::PushElement

把长度值追加进去。


FlatBufferBuilder::GetSize

返回当前大小。

  /// @brief Store a string in the buffer, which can contain any binary data.
  /// @param[in] str A const char pointer to the data to be stored as a string.
  /// @param[in] len The number of bytes that should be stored from `str`.
  /// @return Returns the offset in the buffer where the string starts.
  Offset<String> CreateString(const char *str, size_t len) {
    NotNested();
    PreAlign<uoffset_t>(len + 1);  // Always 0-terminated.
    buf_.fill(1);
    PushBytes(reinterpret_cast<const uint8_t *>(str), len);
    PushElement(static_cast<uoffset_t>(len));
    return Offset<String>(GetSize());
  }

CreateWeapon

创建一个

WeaponBuilder

，由后者进行构造。


WeaponBuilder::add_name

记录

name

的偏移。


WeaponBuilder::add_damage

保存

damage

的值。


WeaponBuilder::Finish

生成 vtable。

inline flatbuffers::Offset<Weapon> CreateWeapon(
    flatbuffers::FlatBufferBuilder &_fbb,
    flatbuffers::Offset<flatbuffers::String> name = 0,
    int16_t damage = 0) {
  WeaponBuilder builder_(_fbb);
  builder_.add_name(name);
  builder_.add_damage(damage);
  return builder_.Finish();
}

WeaponBuilder

FlatBufferBuilder::AddOffset

插入偏移量。


Offset

用于

uoffset_t

的包装器，可进行安全的模板特化。


FlatBufferBuilder::EndTable

完成了一个序列化对象，方法是生成 vtable（如果是表），将其与现有vtable进行比较，然后写入生成的 vtable 偏移量。

struct WeaponBuilder {
  typedef Weapon Table;
  flatbuffers::FlatBufferBuilder &fbb_;
  flatbuffers::uoffset_t start_;
  void add_name(flatbuffers::Offset<flatbuffers::String> name) {
    fbb_.AddOffset(Weapon::VT_NAME, name);
  }
  void add_damage(int16_t damage) {
    fbb_.AddElement<int16_t>(Weapon::VT_DAMAGE, damage, 0);
  }
  explicit WeaponBuilder(flatbuffers::FlatBufferBuilder &_fbb)
        : fbb_(_fbb) {
    start_ = fbb_.StartTable();
  }
  flatbuffers::Offset<Weapon> Finish() {
    const auto end = fbb_.EndTable(start_);
    auto o = flatbuffers::Offset<Weapon>(end);
    return o;
  }
};

Weapon

WeaponT

为表的对象结构体（T 结尾符），该结构体可以直接进行数据操作。MNN 中采用这种结构体和 Pack 函数完成序列化。

struct Weapon FLATBUFFERS_FINAL_CLASS : private flatbuffers::Table {
  typedef WeaponT NativeTableType;
  typedef WeaponBuilder Builder;
  static const flatbuffers::TypeTable *MiniReflectTypeTable() {
    return WeaponTypeTable();
  }
  enum FlatBuffersVTableOffset FLATBUFFERS_VTABLE_UNDERLYING_TYPE {
    VT_NAME = 4,
    VT_DAMAGE = 6
  };
  const flatbuffers::String *name() const {
    return GetPointer<const flatbuffers::String *>(VT_NAME);
  }
  flatbuffers::String *mutable_name() {
    return GetPointer<flatbuffers::String *>(VT_NAME);
  }
  int16_t damage() const {
    return GetField<int16_t>(VT_DAMAGE, 0);
  }
  bool mutate_damage(int16_t _damage) {
    return SetField<int16_t>(VT_DAMAGE, _damage, 0);
  }
  bool Verify(flatbuffers::Verifier &verifier) const {
    return VerifyTableStart(verifier) &&
           VerifyOffset(verifier, VT_NAME) &&
           verifier.VerifyString(name()) &&
           VerifyField<int16_t>(verifier, VT_DAMAGE) &&
           verifier.EndTable();
  }
  WeaponT *UnPack(const flatbuffers::resolver_function_t *_resolver = nullptr) const;
  void UnPackTo(WeaponT *_o, const flatbuffers::resolver_function_t *_resolver = nullptr) const;
  static flatbuffers::Offset<Weapon> Pack(flatbuffers::FlatBufferBuilder &_fbb, const WeaponT* _o, const flatbuffers::rehasher_function_t *_rehasher = nullptr);
};

Offset

用于

voffset_t

的包装器，可进行安全的模板特化。

// Wrapper for uoffset_t to allow safe template specialization.
// Value is allowed to be 0 to indicate a null object (see e.g. AddOffset).
template<typename T> struct Offset {
  uoffset_t o;
  Offset() : o(0) {}
  Offset(uoffset_t _o) : o(_o) {}
  Offset<void> Union() const { return Offset<void>(o); }
  bool IsNull() const { return !o; }
};

FlatBufferBuilder::AddOffset

FlatBufferBuilder::AddElement



FlatBufferBuilder::ReferTo



voffset_t

  template<typename T> void AddOffset(voffset_t field, Offset<T> off) {
    if (off.IsNull()) return;  // Don't store.
    AddElement(field, ReferTo(off.o), static_cast<uoffset_t>(0));
  }

FlatBufferBuilder::AddElement

插入字段和值。


IsTheSameAs

判断两个标量值是否相等。


FlatBufferBuilder::PushElement

将单个对齐的标量写入缓冲区并返回缓冲区大小。


FlatBufferBuilder::TrackField

将

FieldLoc

结构体插入到

buf_

中。

  // Like PushElement, but additionally tracks the field this represents.
  template<typename T> void AddElement(voffset_t field, T e, T def) {
    // We don't serialize values equal to the default.
    if (IsTheSameAs(e, def) && !force_defaults_) return;
    auto off = PushElement(e);
    TrackField(field, off);
  }

FlatBufferBuilder::TrackField

构造一个

FieldLoc

结构体。


vector_downward::scratch_push_small

将数据插入向量的最低地址中。

  // When writing fields, we track where they are, so we can create correct
  // vtables later.
  void TrackField(voffset_t field, uoffset_t off) {
    FieldLoc fl = { off, field };
    buf_.scratch_push_small(fl);
    num_field_loc++;
    max_voffset_ = (std::max)(max_voffset_, field);
  }

FlatBufferBuilder::CreateVector

调用使用数组指针的版本。

  /// @brief Serialize a `std::vector` into a FlatBuffer `vector`.
  /// @tparam T The data type of the `std::vector` elements.
  /// @param v A const reference to the `std::vector` to serialize into the
  /// buffer as a `vector`.
  /// @return Returns a typed `Offset` into the serialized data indicating
  /// where the vector is stored.
  template<typename T> Offset<Vector<T>> CreateVector(const std::vector<T> &v) {
    return CreateVector(data(v), v.size());
  }

FlatBufferBuilder::ReferTo

  // Offsets initially are relative to the end of the buffer (downwards).
  // This function converts them to be relative to the current location
  // in the buffer (when stored here), pointing upwards.
  uoffset_t ReferTo(uoffset_t off) {
    // Align to ensure GetSize() below is correct.
    Align(sizeof(uoffset_t));
    // Offset must refer to something already in buffer.
    FLATBUFFERS_ASSERT(off && off <= GetSize());
    return GetSize() - off + static_cast<uoffset_t>(sizeof(uoffset_t));
  }

FlatBufferBuilder::CreateVector

FlatBufferBuilder::EndVector

插入数组长度并返回偏移。

  template<typename T>
  Offset<Vector<Offset<T>>> CreateVector(const Offset<T> *v, size_t len) {
    StartVector(len, sizeof(Offset<T>));
    for (auto i = len; i > 0;) { PushElement(v[--i]); }
    return Offset<Vector<Offset<T>>>(EndVector(len));
  }

FlatBufferBuilder::StartVector

  void StartVector(size_t len, size_t elemsize) {
    NotNested();
    nested = true;
    PreAlign<uoffset_t>(len * elemsize);
    PreAlign(len * elemsize, elemsize);  // Just in case elemsize > uoffset_t.
  }

FlatBufferBuilder::PushElement

访问其中的偏移值。

  template<typename T> uoffset_t PushElement(Offset<T> off) {
    // Special case for offsets: see ReferTo below.
    return PushElement(ReferTo(off.o));
  }

FlatBufferBuilder::PushElement

AssertScalarT

检查

T

类型是否为标量。


Align

对

buf_

进行填充。


EndianScalar

得到小端值。


FlatBufferBuilder::GetSize

返回

buf_

的大小。

  // Write a single aligned scalar to the buffer
  template<typename T> uoffset_t PushElement(T element) {
    AssertScalarT<T>();
    T litle_endian_element = EndianScalar(element);
    Align(sizeof(T));
    buf_.push_small(litle_endian_element);
    return GetSize();
  }

FlatBufferBuilder::CreateVector

data

对于空向量返回非 null 以避免未定义的行为。因为最终返回的指针被传递给 memcpy，所以需要它。

  /// @brief Serialize a `std::vector` into a FlatBuffer `vector`.
  /// @tparam T The data type of the `std::vector` elements.
  /// @param v A const reference to the `std::vector` to serialize into the
  /// buffer as a `vector`.
  /// @return Returns a typed `Offset` into the serialized data indicating
  /// where the vector is stored.
  template<typename T> Offset<Vector<T>> CreateVector(const std::vector<T> &v) {
    return CreateVector(data(v), v.size());
  }

FlatBufferBuilder::CreateVector

FlatBufferBuilder::StartVector


以倒序方式插入

  template<typename T>
  Offset<Vector<Offset<T>>> CreateVector(const Offset<T> *v, size_t len) {
    StartVector(len, sizeof(Offset<T>));
    for (auto i = len; i > 0;) { PushElement(v[--i]); }
    return Offset<Vector<Offset<T>>>(EndVector(len));
  }

CreateMonster

调用

MonsterBuilder

inline flatbuffers::Offset<Monster> CreateMonster(
    flatbuffers::FlatBufferBuilder &_fbb,
    const MyGame::Sample::Vec3 *pos = 0,
    int16_t mana = 150,
    int16_t hp = 100,
    flatbuffers::Offset<flatbuffers::String> name = 0,
    flatbuffers::Offset<flatbuffers::Vector<uint8_t>> inventory = 0,
    MyGame::Sample::Color color = MyGame::Sample::Color_Blue,
    flatbuffers::Offset<flatbuffers::Vector<flatbuffers::Offset<MyGame::Sample::Weapon>>> weapons = 0,
    MyGame::Sample::Equipment equipped_type = MyGame::Sample::Equipment_NONE,
    flatbuffers::Offset<void> equipped = 0,
    flatbuffers::Offset<flatbuffers::Vector<const MyGame::Sample::Vec3 *>> path = 0) {
  MonsterBuilder builder_(_fbb);
  builder_.add_path(path);
  builder_.add_equipped(equipped);
  builder_.add_weapons(weapons);
  builder_.add_inventory(inventory);
  builder_.add_name(name);
  builder_.add_pos(pos);
  builder_.add_hp(hp);
  builder_.add_mana(mana);
  builder_.add_equipped_type(equipped_type);
  builder_.add_color(color);
  return builder_.Finish();
}

MonsterBuilder

struct MonsterBuilder {
  typedef Monster Table;
  flatbuffers::FlatBufferBuilder &fbb_;
  flatbuffers::uoffset_t start_;
  void add_pos(const MyGame::Sample::Vec3 *pos) {
    fbb_.AddStruct(Monster::VT_POS, pos);
  }
  void add_mana(int16_t mana) {
    fbb_.AddElement<int16_t>(Monster::VT_MANA, mana, 150);
  }
  void add_hp(int16_t hp) {
    fbb_.AddElement<int16_t>(Monster::VT_HP, hp, 100);
  }
  void add_name(flatbuffers::Offset<flatbuffers::String> name) {
    fbb_.AddOffset(Monster::VT_NAME, name);
  }
  void add_inventory(flatbuffers::Offset<flatbuffers::Vector<uint8_t>> inventory) {
    fbb_.AddOffset(Monster::VT_INVENTORY, inventory);
  }
  void add_color(MyGame::Sample::Color color) {
    fbb_.AddElement<int8_t>(Monster::VT_COLOR, static_cast<int8_t>(color), 2);
  }
  void add_weapons(flatbuffers::Offset<flatbuffers::Vector<flatbuffers::Offset<MyGame::Sample::Weapon>>> weapons) {
    fbb_.AddOffset(Monster::VT_WEAPONS, weapons);
  }
  void add_equipped_type(MyGame::Sample::Equipment equipped_type) {
    fbb_.AddElement<uint8_t>(Monster::VT_EQUIPPED_TYPE, static_cast<uint8_t>(equipped_type), 0);
  }
  void add_equipped(flatbuffers::Offset<void> equipped) {
    fbb_.AddOffset(Monster::VT_EQUIPPED, equipped);
  }
  void add_path(flatbuffers::Offset<flatbuffers::Vector<const MyGame::Sample::Vec3 *>> path) {
    fbb_.AddOffset(Monster::VT_PATH, path);
  }
  explicit MonsterBuilder(flatbuffers::FlatBufferBuilder &_fbb)
        : fbb_(_fbb) {
    start_ = fbb_.StartTable();
  }
  flatbuffers::Offset<Monster> Finish() {
    const auto end = fbb_.EndTable(start_);
    auto o = flatbuffers::Offset<Monster>(end);
    return o;
  }
};

参考资料：

• 
  Tutorial
  
• 
  Difference between char and signed char in c++?
  
• 
  Can I reinterpret std::vector as a std::vector without copying?
  
• 
  FlatBuffers介绍及初步试用
  
• 
  Writing a schema
  
• 
  android FlatBuffers剖析
  
• 
  FlatBuffers学习总结
  
• 
  Flatbuffers使用解析
  
• 
  FlatBuffer Internals
  
• 
  FlatBuffers white paper
  
• 
  how to feed data from binary string ubyte vector of flatbuffer in python?
  
• 
  深入浅出 FlatBuffers 之 Encode
  
• 
  数据序列化组件PB与FB对比
  
• 
  FlatBuffer代码阅读 – 1
  
• 
  Applied SOAP: Implementing .NET XML Web Services by Kenn Scribner, Mark Stiver
  
• 
  Git 协议版本 2 宣布推出：Git wire protocol 的一次重大更新
  
• 
  什么是wire protocol？
  
• 
  What is wire format
  
• 
  Class WireFormat
  
• 
  Wire protocol
  
• 
  XML wire format
  
• 
  flatbuffer的数组小例子（C++）
  
• 
  General rules of passing/returning reference of array (not pointer) to/from a function?
  
• 
  Return array in a function
  
• 
  flatbuffer编码的内存的结构
  
• 
  FlatBuffers反射
  
• 
  caffe模型文件解析_MNN模型转化的主流程
  
• 
  Why cast an unused function parameter value to void?