Adventures in Binary Serialisation

2024-01-07T00:00:00+00:00
The other day I came across an interesting question^{1</a> - instead of faffing around with serde</code></a> and dealing with the compile-time hits, can I just write the binary data from RAM to a file? This is just an exploration of that problem. I thought this would be simple, but I ended up finding some interesting UB and digging into how Vec allocates just a little bit.}
If you find any factual errors with this article - I’ll happily take responsibility, edit it and provide appropriate credit. I’ve tried my best and that’s kinda all I can give. See contact details here</a>.
Because, at least for now, this is just an experience in having fun I’ll be using whatever structs please me2</a> without any real focus on real-world usage. I’ll also be expecting some intermediate to advanced knowledge on programming because I’ll be skipping over some basic explanations. This will also be specific to Rust and other manual-memory management languages.
Plain-Old-Data</h2>
Firstly, I have to work out some data to store, so I’ll start with a simple struct:</p>
struct </span>ExampleData {
</span>    </span>a</span>: </span>u8</span>,
</span>    </span>b</span>: </span>u64</span>,
</span>    </span>c</span>: [</span>u8</span>; 7],
</span>    </span>d</span>: </span>u32</span>,
</span>}
</span>
</span>fn </span>example_data </span>() -> ExampleData {
</span>    ExampleData {
</span>        a: </span>0</span>,
</span>        b: </span>u64</span>::</span>MAX</span>,
</span>        c: [</span>10</span>, </span>11</span>, </span>12</span>, </span>13</span>, </span>14</span>, </span>15</span>, </span>16</span>],
</span>        d: </span>123456789
</span>    }
</span>}
</span></code></pre>
I’ve carefully chosen the types and their positions (but I’ll get more into that later), but all that matters for now is that they’re all stored on the stack3</a></sup>.</p>
Serialising</h3>
Guess what? We’re all done here, because the compiler does all the work! The struct must be somewhere in memory and we just need to work out how to trick the compiler into letting us see the struct as just a series of bytes.</p>
After some real soul-searching4</a></sup>, I managed to find this:</p>
unsafe fn </span>any_as_u8_slice</span><T: Sized>(</span>p</span>: &T) -> &[</span>u8</span>] {
</span>    ::core::slice::from_raw_parts(
</span>        (p as </span>*const</span> T) as </span>*const u8</span>,
</span>        ::core::mem::size_of::<T>(),
</span>    )
</span>}
</span></code></pre>
This takes a reference to a Rust object (which has a finite and known-at-compile-time size), and creates a slice of bytes starting at the address of the reference with the length being the length of the struct. This looks sound to me as we know that the slice will only be valid for as long as the struct (because of lifetimes which are elided here), and we’re only accessing memory that we know belongs to the struct.</p>
If we just print this to stdout, we see something really interesting that rust does that lots of other languages don’t:</p>
[</span>255</span>, </span>255</span>, </span>255</span>, </span>255</span>, </span>255</span>, </span>255</span>, </span>255</span>, </span>255</span>, </span>21</span>, </span>205</span>, </span>91</span>, </span>7</span>, </span>0</span>, </span>10</span>, </span>11</span>, </span>12</span>, </span>13</span>, </span>14</span>, </span>15</span>, </span>16</span>, </span>0</span>, </span>0</span>, </span>0</span>, </span>0</span>]
</span></code></pre>
I deliberately picked some quite distinctive values for the example data - we can see the 8 255</code>s for the u64::MAX</code>, the [21, 205, 91, 7]</code> for our u32</code>, the 0</code> for the u8, the [10, 11, 12, 13, 14, 15, 16]</code> for the u8</code> array. However, there are two main problems here - firstly, the struct’s been reordered and there’s also an extra 4 0</code>s on the end?</p>
The first is a consequence of Rust not having a stable ABI - that is to say that the compiler reserves every right to screw around with the internal representation of anything and everything if it works out it can pack it more efficiently. The second is a consequence of padding - unless we tell it otherwise, the compiler always tries to align</em> the length of the struct to the word length</code> of the PC running the code (here, 8 bytes for a 64-byte word)</del> correction: alignment is actually usually determined by the maximum length of variables inside - here, a u64 so 8 bytes</em>. This just allows reads to be far more efficient for reasons I’m not 100% sure on - I just chalk it up to one of those things that we deal with in exchange for lightning rock magic</strong></em>. correction: some CPU instructions also require alignment for being able to work, not just performance reasons</em></p>
We can fix the first by adding a little #[repr(C)]</code> to just before our struct declaration which tells the compiler to follow the C style which preserves ordering. However, this then balloons our struct to 32 bytes rather than just 24. The only reason you’d do so would be if you were trying to do some form of FFI</code>, but there are far better solutions.</p>
However, if you run this through miri</code></a>, you’ll see that this function exhibits undefined behaviour. This is because we’re accessing uninitialised memory. At first, you might be confused as to why - it’s the padding. Padding is uninitialised memory and so this function is UB (I’ll discuss UB more later, don’t worry). To fix this, we can instead #[repr(packed)]</code>, which removes all padding at a possible cost to performance. That having been said, for future learning opportunities</em>5</a></sup>, we’ll pretend that the struct isn’t packed for the rest of this article.</p>
For now, I’ll write those bytes to a file.</p>
use </span>std::{io::Write, fs::File};
</span>
</span>fn </span>main </span>() -> std::io::Result<()> {
</span>    </span>let</span> eg = </span>example_data</span>();
</span>    </span>let</span> bytes = </span>unsafe </span>{ </span>any_as_u8_slice</span>(&eg) };
</span>    File::create("</span>out.bin</span>")?.</span>write_all</span>(&bytes)?;
</span>    
</span>
</span>    Ok(())
</span>}
</span></code></pre>
Deserialising</h3>
Then, on this end, I can read those bytes back in to prove I’m using different bytes.</p>
use </span>std::io::Read;
</span>
</span>fn </span>read_in </span>() -> std::io::Result<ExampleData> {
</span>    </span>let mut</span> bytes = vec![];
</span>
</span>    </span>const </span>SIZE</span>: </span>usize </span>= </span>16</span>;
</span>    </span>let mut</span> reader = File::open("</span>out.bin</span>")?;
</span>    </span>let mut</span> tmp = [</span>0</span>; </span>SIZE</span>];
</span>    </span>loop </span>{
</span>        </span>match</span> reader.</span>read</span>(&</span>mut</span> tmp)? {
</span>            </span>0 </span>=> </span>break</span>,
</span>            n => {
</span>                bytes.</span>extend</span>(&tmp[</span>0</span>..n]);
</span>            }
</span>        }
</span>    }
</span>
</span>    todo!("</span>What now?</span>")
</span>}
</span></code></pre>
I now have a vector full of data which consists of a pointer to the beginning of the data and the length of that data. I now need to do the reverse of what I did before but this is not nearly as easy.</p>
Raw pointer dereference</h3>
My first instinct here is just to dereference the pointer like this:</p>
let</span> ptr = bytes.</span>as_ptr</span>() as </span>*const</span> ExampleData;
</span>let</span> example = </span>unsafe </span>{ *ptr };
</span></code></pre>
We just have to cast it from a *const u8</code> to a *const ExampleData</code>, which is easy.</p>
The first problem comes from the fact that we can’t dereference the pointer because we’d need to copy out the data. We can fix that with a quick #[derive(Copy, Clone)]</code>. This then appears to work, until we run it through miri</code> and see that the alignment is incorrect and this is undefined behaviour - we’ll have the same problem with the next method so I’ll dive deeper into it there.</p>
Boxing</h3>
Let’s delete the derive and try again by thinking about how we deal with pointers in Rust.</p>
Well, we typically do that with a Box, which holds a pointer to a struct on the heap. We usually make one of these with Box::new</code></a> to get it from the stack to the heap, but if the data is already on the heap, we can use Box::from_raw</code></a>:</p>
let</span> boxed: Box<ExampleData> = </span>unsafe </span>{ Box::from_raw(bytes.</span>as_mut_ptr</span>() as </span>*mut</span> ExampleData) };
</span>return </span>Ok(*boxed);
</span></code></pre>
This is very similar to the way we made a slice earlier, but here we don’t need to specify the size as ExampleData</code> has a constant size. This then also works just fine, because we can dereference the box without needing copy and we’ve got the struct back out from the heap. 🥳🥳</p>
However, if you run this just with cargo run</code>, you’ll see a problem. You won’t see a lovely 0</code> exit code, and instead we get this: error: process didn't exit successfully: 'target\debug\ser.exe' (exit code: 0xc0000374, STATUS_HEAP_CORRUPTION)</code>. This implies that we’ve somehow corrupted the heap, so let’s have a gander at the documentation.</p>
The Rust standard library documentation is incredible for having thorough safety discussions and help for unsafe functions. If we look back at the function documentation</a>, we can see that it has this to say:</p>

Constructs a box from a raw pointer.</p>
After calling this function, the raw pointer is owned by the resulting Box. Specifically, the Box destructor will call the destructor of T and free the > allocated memory. For this to be safe, the memory must have been allocated in accordance with the memory layout used by Box.</p>
Safety</strong></p>
This function is unsafe because improper use may lead to memory problems. For example, a double-free may occur if the function is called twice on the same > raw pointer.</p>
</blockquote>
Here, we can now see our error - a classic double-free. When the function exits, bytes</code> will get freed. Since *boxed</code> goes back to main</code>, it’ll get freed at the end there and we’ve now freed the same memory twice which corrupts the heap. Luckily, the fix here is relatively simple - we just need to add a strategic std::mem::forget</code></a>:</p>
bytes.</span>shrink_to_fit</span>();
</span>let</span> boxed: Box<ExampleData> = </span>unsafe </span>{ Box::from_raw(bytes.</span>as_mut_ptr</span>() as </span>*mut</span> ExampleData) };
</span>std::mem::forget(bytes);
</span>return </span>Ok(*boxed);
</span></code></pre>
std::mem::forget</code></a> basically just tells rust specifically not to run the destructor of the object you give it, therefore not freeing the memory. Here, we use it to make sure that bytes</code> never gets freed. We don’t need to worry about the memory getting leaked because it then gets freed when boxed</code> gets dropped. I’ve also added a Vec::shrink_to_fit</code></a> call to ensure that the spare memory we likely allocated for the Vec doesn’t get lost.6</a></sup>.</p>
correction: the Vec’s allocation may be reused which would then be UB. Instead, use bytes.leak()</code></em>.</p>
UB?</h3>
This all seems to work, but if we run it through miri</code>, we can see that this is UB. Specifically: error: Undefined Behavior: constructing invalid value: encountered an unaligned box (required 8 byte alignment but found 2)</code>, which happens when we create the Box</code> with Box::from_raw</code>. Here, we get to alignment issues. I briefly wrote about alignment earlier, and the part that matters here is that alignment is inherent to a specific struct. In fact, if we look at the alloc</code></a> and dealloc</code></a> methods they take a Layout</code></a> which stores size and alignment, unlike C’s malloc</code> and free</code></a> which just take a size.</p>
If we look at what behaviour is considered undefined (helpfully linked</a> to me by miri</code>), we can see (in the second item) that accessing a misaligned pointer is undefined behaviour. I’ll quickly explain for people coming from C the danger of undefined behaviour in Rust. In C, there are lots of behaviours that people use that aren’t specifically designated by the specification (like integer over/under-flow) which are undefined behaviour. They’re mostly fine there, but Rust UB is a different beast entirely because of how heavily the compiler tries to optimise code. Generally in Rust, no UB is good UB, and if you wrote UB then the compiler reserves the right (and often will, especially in release builds) to do whatever the hell it wants regardless of your intentions. Here, it kinda works but in a larger program nobody</strong> can know what the compiler will do.</p>
correction</strong>: C UB is also very very bad, and I think I just got confused between undefined behaviour and specification-undefined behaviour. the example I mentioned is also wrong - you can’t rely on signed integer over/under-flow, especially in release builds</em></p>
Alignment</h4>
To actually explain the issue, we need to look at the Layout</code>’s of Vec</code> and of ExampleData</code>, which is easy because Layout</code> implements Debug</code> which means we can print it out to get all of its fields.</p>
use </span>std::alloc::Layout;
</span>
</span>fn </span>print_layouts </span>() {
</span>    println!("</span>Vec<u8>:     </span>{:?}</span>", Layout::new<Vec<</span>u8</span>>>());
</span>    println!("</span>ExampleData: </span>{:?}</span>", Layout::new::<ExampleData>());
</span>}
</span></code></pre>
Which then outputs that they’re the same?</p>
Vec<</span>u8</span>>:     Layout { size: </span>24</span>, align: </span>8 </span>(</span>1 </span><< </span>3</span>) }
</span>ExampleData: Layout { size: </span>24</span>, align: </span>8 </span>(</span>1 </span><< </span>3</span>) }
</span></code></pre>
The Same?</h4>
This definitely stumped me for a while until I decided to have a look at the documentation & source for std::vec::Vec</code></a>, and then by clicking on Source</em> I saw that it holds a RawVec</code> buffer and the current length. I’ve added the definition below, simplified to remove the standard library shenanigans:</p>
pub struct </span>Vec<T, A: Allocator = Global> {
</span>    </span>buf</span>: RawVec<T, A>,
</span>    </span>len</span>: </span>usize</span>,
</span>}
</span></code></pre>
RawVec</code> is a private class that Vec</code> uses to wrap lots of the unsafe behaviours, and you can’t actually see it in any documentation. However, you can just click around in the source to find it here</a>:</p>
pub</span>(</span>crate</span>) </span>struct </span>RawVec<T, A: Allocator = Global> {
</span>    </span>ptr</span>: Unique<T>,
</span>    </span>cap</span>: </span>usize</span>,
</span>    </span>alloc</span>: A,
</span>}
</span></code></pre>
alloc</code> is a zero-sized type, and then both cap</code> and ptr</code> have size usize</code> which is 8 bytes. If we add up all of the bytes and check our alignments we can then see that yeah it’s 24 bytes with 8 byte-alignment.</p>
To see where our alignment problem from earlier comes from, we have to dig into Vec::push</code></a>:</p>
pub fn </span>push</span>(&</span>mut </span>self</span>, </span>value</span>: T) {
</span>    </span>if </span>self</span>.len == </span>self</span>.buf.</span>capacity</span>() {
</span>        </span>self</span>.buf.</span>reserve_for_push</span>(</span>self</span>.len);
</span>    }
</span>    </span>unsafe </span>{
</span>        </span>let</span> end = </span>self</span>.</span>as_mut_ptr</span>().</span>add</span>(</span>self</span>.len);
</span>        ptr::write(end, value);
</span>        </span>self</span>.len += </span>1</span>;
</span>    }
</span>}
</span></code></pre>
We can see that it checks for capacity (and reserves space if we don’t have enough which I’ll get to in just a second), and it then writes to a pointer with the new value. That call for reserving is on self.buf</code>, which is the RawVec</code> from earlier which handles all of the allocation so it’s probably where</a> we want to be looking.</p>
pub fn </span>reserve_for_push</span>(&</span>mut </span>self</span>, </span>len</span>: </span>usize</span>) {
</span>    </span>handle_reserve</span>(</span>self</span>.</span>grow_amortized</span>(len, </span>1</span>));
</span>}
</span>
</span>fn </span>grow_amortized</span>(&</span>mut </span>self</span>, </span>len</span>: </span>usize</span>, </span>additional</span>: </span>usize</span>) -> Result<(), TryReserveError> {
</span>    debug_assert!(additional > </span>0</span>);
</span>    </span>if </span>T::</span>IS_ZST </span>{
</span>        </span>return </span>Err(CapacityOverflow.</span>into</span>());
</span>    }
</span>    </span>let</span> required_cap = len.</span>checked_add</span>(additional).</span>ok_or</span>(CapacityOverflow)?;
</span>
</span>    </span>let</span> cap = cmp::max(</span>self</span>.cap * </span>2</span>, required_cap);
</span>    </span>let</span> cap = cmp::max(</span>Self</span>::</span>MIN_NON_ZERO_CAP</span>, cap);
</span>    </span>let</span> new_layout = Layout::array::<T>(cap);
</span>
</span>    </span>let</span> ptr = </span>finish_grow</span>(new_layout, </span>self</span>.</span>current_memory</span>(), &</span>mut </span>self</span>.alloc)?;
</span>    </span>self</span>.</span>set_ptr_and_cap</span>(ptr, cap);
</span>    Ok(())
</span>}
</span></code></pre>
handle_reserve</code></a> just deals with errors, and if we peer through grow_amortized</code> which actually allocates (and strategically ignore the amortisation parts and just look for a Layout</code>), we can see that it actually allocates using Layout::array<T>(cap)</code>. If we then print out that layout (using the number of bytes we need for storing an ExampleData</code> for the array size):</p>
fn </span>print_layouts </span>() {
</span>    </span>let</span> size = std::mem::size_of::<ExampleData>();
</span>    println!("</span>Vec<u8>:     </span>{:?}</span>", Layout::array::<</span>u8</span>>(size).</span>unwrap</span>());
</span>    println!("</span>ExampleData: </span>{:?}</span>", Layout::new::<ExampleData>());
</span>}
</span></code></pre>
We get this out the other end - success! The alignment is different which then explains why our Box</code> creation earlier was undefined behaviour.</p>
Vec<</span>u8</span>>:     Layout { size: </span>24</span>, align: </span>1 </span>(</span>1 </span><< </span>0</span>) }
</span>ExampleData: Layout { size: </span>24</span>, align: </span>8 </span>(</span>1 </span><< </span>3</span>) }
</span></code></pre>
The obvious next question is whether there’s any way to convert between these two without copying (since that’s why we’ve gotten into this whole mess). That having been said, I’ll take a brief aside to explain how to do this if you’re OK with allocating another 24 bytes.</p>
Copying again (but without Copy</code> needed nor any UB)</em></h3>
The following code (according to miri</code> and my own gut feeling7</a></sup>) is unsafe, but sound (meaning that it doesn’t cause UB):</p>
return </span>Ok(</span>unsafe </span>{
</span>    </span>let mut</span> ptr = std::alloc::alloc(Layout::new::<ExampleData>());
</span>
</span>    </span>for</span> b in bytes {
</span>        </span>unsafe </span>{ ptr.</span>write</span>(b) };
</span>        ptr = ptr.</span>add</span>(</span>1</span>);
</span>    }
</span>
</span>    ptr = ptr.</span>sub</span>(std::mem::size_of::<ExampleData>());
</span>    *Box::from_raw(ptr as </span>*mut</span> ExampleData)
</span>});
</span></code></pre>
We create a pointer using alloc</code> with the correct layout. We then write each byte and increment the address of the pointer each time. We then take the pointer back to the start and give it to a Box</code> to be managed. That Box</code> is then responsible for freeing the heap memory we allocated and we also use it to get the data out from the heap back to the stack. Don’t ask me how it does that - I can’t tell for the life of me. There’s probably a better way to do it, but I can at least say that this won’t cause memory leaks, memory corruption or compiler mayhem.</p>
Is there a no-copy solution?</h3>
As far as I can tell, there isn’t a way to tell the Rust compiler that we want to change the alignment of an existing allocation, which then means that (unless the struct has 1-byte alignment) there isn’t a way to do a no-copy solution which annoys me. If you work out a way of doing it - I’d love to hear it.</p>
Unlike the first copying method which required the struct to be Copy</code> and was UB unless the struct was packed, this doesn’t need that and</strong> as far as I can tell, it’s sound.</p>
Heap Data</h2>
That’s all well and good, but what about Heap data like a Vector</code> or a String</code>?</p>
By itself, you’d have to know at compile-time the type, but you could probably get away with just writing the contents of the pointer, and carefully reading them back. However, you’d have to be careful around cases where more space was allocated than used which would be annoying.</p>
The main complications would come from if that heap data was stored inside a struct. You’d no longer be able to just pretend it was all just a series of bytes, and you’d have to put in lots of manual work to not accidentally just write a memory address that would become invalid the second the program finished8</a></sup>, and to read back the correct number of bytes. At that point, you’re better off just using something like protobuf</code></a> or serde</code>.</p>
Closing Thoughts</h2>
Whilst yes, this does work I would never recommend it for any production use where you couldn’t control it very</em> finely. The main problem comes from what I briefly mentioned earlier about the Rust ABI not being stable. If the compiler updates, even if you don’t change your code, the in-memory representation of your struct could change and then you’d never have any way of recovering that written data other than downgrading your compiler version to ‘before it broke’. I’d also be careful about manually writing bytes by hand instead of outputting an existing struct - Rust does some really interesting optimisations which could lead to bit patterns being something completely different to what you expected - like Option<bool></code> which only takes 1 byte. It’s also undefined behaviour for certain types to be in invalid bit states (eg. 0x0 and 0x1 are fine for a bool</code>, but 0x3 is invalid) because they’re often used for those optimisations.</p>
As to why the UB never actually caused any issues, I’ve not a clue. There’s probably someone who could explain more but I’m not that person. Reading padding is undefined behaviour, and I couldn’t say why it didn’t cause any issues here.</p>
Whilst you could make arguments around the use of #[repr(C)]</code> to safeguard against this, there’s another problem - there’s no description of what this data is if anyone’s poking around the filesystem. Most file formats start with something to mark what format they are, but there’s nothing here if you’re trying to read data later. Even if you don’t go as far as to use a self-describing format like JSON</code>, most serialisation methods (as far as I can tell) at least make it clear that they’re storing data. With this, unless you’re a rare programmer who actually writes thorough documentation, I can’t even tell if its a binary or a file.</p>
I’ll finish off by saying that this article was also a bit NIH</a> and if you want to actually do things like this, I’d have a gander at bytemuck</code></a>.</p>
Anyways, that was fun to explore and try and get a better knowledge of UB and binary formats. Enjoy the new year!</p>
Full final code can be viewed here</a>.</p>
errata</em></h2>
I posted a link to this on the subreddit here</a> and received some comments - here are my responses.</p>
u/dkopgerpgdolfg</h3>
This user pointed out one thing which makes me feel like a fool - if we know the length of our struct beforehand (which we do - it’s std::mem::size_of::<ExampleData>()</code>), we can just read straight into the pointer:</p>
fn </span>read_in_straight_to_pointer </span>(</span>file_name</span>: &</span>str</span>) -> std::io::Result<Option<ExampleData>> {
</span>    </span>unsafe </span>{
</span>        </span>const </span>STRUCT_SIZE</span>: </span>usize </span>= std::mem::size_of::<ExampleData>();
</span>     </span>const </span>LAYOUT</span>: Layout = Layout::new::<ExampleData>();
</span>
</span>        </span>let mut</span> ptr = std::alloc::alloc(</span>LAYOUT</span>); </span>//create a pointer to the heap struct
</span>        </span>let mut</span> bytes_read_total = </span>0</span>; </span>//create a counter for how many bytes we've written - this is like the `len` of a Vec, where `STRUCT_SIZE` is the `cap`
</span>
</span>        </span>let mut</span> reader = File::open(file_name)?; </span>//open a file
</span>        </span>let mut</span> tmp = [</span>0</span>; </span>32</span>]; </span>//create a stack for temporary reads - very little performance impact
</span>
</span>        </span>loop </span>{
</span>            </span>match</span> reader.</span>read</span>(&</span>mut</span> tmp)? {
</span>                </span>0 </span>=> </span>break</span>,
</span>                n => {
</span>                    </span>let</span> n = n.</span>min</span>(</span>STRUCT_SIZE </span>- bytes_read_total); </span>//ensure we don't write past the memory we allocated
</span>                    </span>for</span> i in </span>0</span>..n {
</span>                        ptr.</span>write</span>(tmp[i]);
</span>                        ptr = ptr.</span>add</span>(</span>1</span>);
</span>                    }
</span>
</span>              bytes_read_total += n;
</span>
</span>                    </span>if</span> bytes_read_total == </span>STRUCT_SIZE </span>{ </span>//if there's more to read, we don't care
</span>                        </span>break</span>;
</span>                    }
</span>                }
</span>            }
</span>        }
</span>
</span>        </span>if</span> bytes_read_total < </span>STRUCT_SIZE </span>{
</span>            std::alloc::dealloc(ptr, </span>LAYOUT</span>); </span>//if we didn't read enough, then we need to remember to deallocate the block, as Rust won't do it for us
</span>            Ok(None)
</span>        } </span>else </span>{
</span>            ptr = ptr.</span>sub</span>(</span>STRUCT_SIZE</span>);
</span>            Ok(Some(*Box::from_raw(ptr as </span>*mut</span> ExampleData)))
</span>        }
</span>    }
</span>}
</span></code></pre>
I’ve also added corrections from them in the rest of the article relating to alignment, leaking the vec and C undefined behaviour.</p>
u/hniksic</h3>
I’ve added corrections relating to C undefined behaviour.</p>



At least, interesting to me ;) ↩</a></p>
</li>

and some semblance of a structure for you delightful readers. ↩</a></p>
</li>

That is, they’re all in memory in one place without any pointers or memory addresses. ↩</a></p>
</li>

That is, duckduckgo-ing for StackOverflow. ↩</a></p>
</li>

That is, I didn’t realise this properly until I’d written far more of this article (the future bits are also super interesting). If you’ve read more of the article, this is why: if you pack the struct the alignment becomes 1 byte (because of the u8</code>), which then means that it has the same alignment as the vec which makes the initial solution (using Box::from_raw</code> with the bytes.as_mut_ptr() as *mut ExampleData</code>) not unsound any more. ↩</a></p>
</li>

When the Rust vector reallocates, it doesn’t just add one to the capacity, it goes up in a pattern which is designed to amortise the allocation cost to be as low as possible. Because the Box</code> only knows about the length, it won’t know to free the memory that’s in the capacity</code> but not the length</code>; ↩</a></p>
</li>

100% correct 60% of the time ↩</a></p>
</li>

Technically, you could modify the ExampleData</code> from this post with any number of heap-based structures and it might still work. If the memory allocator didn’t de-allocate between you writing out the data (and dropping the old struct), and reading the memory address back I think it’d work? The problem would come from if you wrote on one run and read on another because that memory that it’s pointing to would no longer exist. ↩</a></p>
</li>
</ol>
</section>
Katashift - Serialisation

Adventures in Binary Serialisation
