Katashift - WebVaguely directed ramblingsZola2025-10-18T00:00:00+00:00https://maguire.tech/tags/web/atom.xmlShove2025-10-18T00:00:00+00:002025-10-18T00:00:00+00:00
Jack Maguire
https://maguire.tech/posts/shove/<p>Say hello to <a href="https://github.com/BurntNail/shove">shove</a>, my brand new HTTP file server and content manager!</p>
<p>Shove is an S3-backed HTTP file server that handles live-reloads, partial updates and even basic HTTP auth and it has now replaced caddy as the HTTP server that serves this very blog<sup class="footnote-reference" id="fr-0-1"><a href="#fn-0">1</a></sup>! This blog post will be part explanation, part advertisement and part me being excited about my latest project!</p>
<h2 id="current-functionality">Current Functionality</h2>
<p>Currently, there’s three main commands inside <code>shove</code> - <code>protect</code>, <code>upload</code> and <code>serve</code>, which I’ll briefly explain in that order. I’ll then go over some of the interesting things I did with each.</p>
<h3 id="protect">Protect</h3>
<p><code>shove protect</code> deals with the HTTP Basic Auth that <code>shove</code> has - internally it’s modelled as a bunch of users (each of which has a username, a password and a uuid), and a list of realms (each of which has a pattern to protect and a list of uuids that can access it).</p>
<p>You run <code>shove protect</code> from wherever you’re uploading from (to get easy access to environment variables), add the users and add the realms. The default behaviour for paths which don’t match any patterns is just to allow anyone and everyone access to those files - for example, my blog currently has no access control in place.</p>
<h3 id="protect-1">Protect</h3>
<p><code>shove upload</code> uploads the given directory to the given S3 bucket - as far as the user cares, that’s all it does.</p>
<p>The complexity comes from the fact that it only uploads new files and deletes old files - to achieve this there’s a file in the root directory that stores hashes of each file. <code>shove protect</code> first reads in all the files, then gets that file, checks the hashes and only uploads the files that have changed.</p>
<p>That’s why <code>shove protect</code> can’t upload a ‘current directory’ - it has to have somewhere to put all the files in the bucket where there’s a guarantee the data file (and the auth file) won’t clash.</p>
<h3 id="serve">Serve</h3>
<p>By far however, the most complicated part is <code>shove serve</code> - this is the general lifecycle:</p>
<ol>
<li>Read in the data file from S3</li>
<li>Find all the files</li>
<li>Read them in from S3 and put them into a cache</li>
</ol>
<p>Then, when we need to serve a file it does this:</p>
<ol>
<li>Check for access control - if that fails, then send a <a href="https://http.cat/status/401">401</a>.</li>
<li>Check if that file is in the cache - if so, send it</li>
<li>If it isn’t: grab it from S3, put that in the cache and serve it</li>
<li>If we couldn’t find it in S3, serve a <a href="https://http.cat/status/404">404</a> page.</li>
</ol>
<p>That’s all
<span id="animated-918806491"></span>
<script>
let personToThankForThisSickAnimation="https://codepen.io/zachkrall/pen/MWWGMPx"
let delay = 250;
let outerspan = document.getElementById("animated-918806491");
outerspan.innerHTML = "relatively"
.split("")
.map(letter => {
return `<span>` + letter + `</span>`;
})
.join("");
Array.from(outerspan.children).forEach((span, index) => {
setTimeout(() => {
span.classList.add("wavy");
}, index * 60 + delay);
});
</script> simple<sup class="footnote-reference" id="fr-5-1"><a href="#fn-5">2</a></sup>. The complicated part is the live reloading which goes a little something like this:</p>
<ol>
<li>Check if we need to reload - Tigris has a webhook, or we can just re-fetch the upload data every 60s and see if that file has changed.</li>
<li>Work out which files have changed, and add their new versions to our cache.</li>
<li>Work out which files have gone missing, and evict them from our cache.</li>
<li>Send a notice to all the websockets connected to reload their pages.</li>
</ol>
<h2 id="fun-tricks-stories">Fun Tricks & Stories</h2>
<h3 id="protect-2">Protect</h3>
<p>HTTP Basic Auth was certainly interesting to get working - I wanted to have some form of authentication, and got 90% the way through implementing SCRAM-SHA-256<sup class="footnote-reference" id="fr-10-1"><a href="#fn-10">3</a></sup>, before I realised that this is supposed to be for server->server authentication and no web browser lets you just put in a password for that.</p>
<p>I then redirected that work into my HTTP Basic Auth implementation. I’d originally planned to more closely copy Caddy (which this project is designed to replace for me), but I kinda ended up going ham on the access control. I’d always been annoyed that I had to SSH-in, use <code>caddy hash-password</code>, update the caddyfile and restart caddy to change the auth<sup class="footnote-reference" id="fr-20-1"><a href="#fn-20">4</a></sup> but <code>shove</code> auth just updates an encrypted file in the bucket that gets regularly checked for updates!</p>
<p>On that front, the auth benefits from the livereloading as well - <code>shove serve</code> never changes the auth, so we know that if it’s been changed then we need to update. Technically, if multiple people were to start changing the protections and finishing their work in weird orders, there could be a race condition where work would be lost, but I’m not concerned about this at the moment<sup class="footnote-reference" id="fr-30-1"><a href="#fn-30">5</a></sup>.</p>
<h3 id="upload">Upload</h3>
<p><code>upload</code> was (<strong>comparatively</strong>) simple - we just read in the provided directory, calculate hashes, read in the upload data from S3, and deal with the S3 bits. The only vaguely interesting parts are that I’ve used things like <code>FuturesUnordered</code> to get a load of uploads going at once.</p>
<h3 id="serve-1">Serve</h3>
<p><code>serve</code> is by far the most complicated, because it handles so damn much. This section will be more an explanation of how stuff works, rather than just tricks and stories. With <a href="https://lib.rs/crates/axum"><code>axum</code></a>/<a href="https://lib.rs/crates/hyper"><code>hyper</code></a>/<a href="https://lib.rs/crates/tower"><code>tower</code></a> projects, I always find a fun way to gauge the complexity is to have a look at the <code>State</code> that the services use, and this is mine:</p>
<pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span>#[</span><span style="color:#bf616a;">derive</span><span>(Clone)]
</span><span style="color:#b48ead;">pub struct </span><span>State {
</span><span> </span><span style="color:#bf616a;">bucket</span><span>: Box<Bucket>,
</span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">tigris_token</span><span>: Option<Arc<</span><span style="color:#b48ead;">str</span><span>>>,
</span><span> </span><span style="color:#bf616a;">pages</span><span>: Pages,
</span><span> </span><span style="color:#bf616a;">live_reloader</span><span>: LiveReloader,
</span><span> </span><span style="color:#bf616a;">auth</span><span>: AuthChecker,
</span><span>}
</span></code></pre>
<p>Yeah, that’s not a lot but still considerable, especially considering three of them are custom structs with lots of other functionality. They also happen to be relatively neat lines to draw for explanations and stories about <code>serve</code> so I’ll give a general explanation, and then dive into those three parts (the live reloader, the auth and the pages).</p>
<h4 id="uploaddata"><code>UploadData</code></h4>
<p>The <code>UploadData</code> is the struct that holds all of the hashes for all of the files in S3 and was where this whole project started. Whenever the reload is triggered<sup class="footnote-reference" id="fr-40-1"><a href="#fn-40">6</a></sup>, we check whether any of the upload data has changed and if so, change it. Once I got hashing working, this was relatively simple.</p>
<p>It doesn’t store any of the actual data, just a map which links a path to a hash, and the name of the root directory we’re serving from.</p>
<h4 id="livereloader"><code>LiveReloader</code></h4>
<p>To use the livereloader, sites must include a small snippet in their javascript that creates a websocket connection to the server and reloads the page when it receives a <code>reload</code> message. Technically, they can open a websocket connection to <em>any</em> path as the HTTP server only checks whether the requests are for websocket upgrades and not what path they’re looking for. This works here and for now because I know there won’t be any other reasons to open websockets<sup class="footnote-reference" id="fr-50-1"><a href="#fn-50">7</a></sup>.</p>
<p>But how does that get handled in the server? So, whenever we receive an upgrade request<sup class="footnote-reference" id="fr-60-1"><a href="#fn-60">8</a></sup>, we send back an upgrade response (kindfully handled by <a href="https://lib.rs/crates/soketto"><code>soketto</code></a>), and then create a <code>Sender</code>/<code>Receiver</code> pair for messages. Then, whenever we need to reload the pages (handled by a channel<sup class="footnote-reference" id="fr-70-1"><a href="#fn-70">9</a></sup>), we just send a <code>reload</code> message to all of the senders!</p>
<p>If you’ve had a peek over the source code though, it might look like <code>livereload.rs</code> is doing a fair bit more than that, and it is because we have to deal with closed sockets. That gets dealt with by another thread that pings all of the sockets every 60 seconds to make sure that they’re still alive and stops keeping track of the dead ones. It could be argued that this is excessive, but it’s only a server-side load and I’m happy with it. If you’ve got a better solution, feel free to submit a PR! The code is also slightly complicated because it uses a <a href="https://docs.rs/futures/latest/futures/prelude/stream/struct.FuturesUnordered.html"><code>FuturesUnordered</code></a> to deal with dead senders as their pings finish, rather than needing to wait for all of them.</p>
<p>There’s also some code there for closing sockets when the server finishes.</p>
<h4 id="authchecker"><code>AuthChecker</code></h4>
<p>Shove also has ways of dealing with HTTP basic auth. That’s when you open a webpage and the browser asks for a username and password, rather than the webpage. On the plus side, it’s relatively easy to implement from the server, but also it involves sending the entered password over the wire in plaintext which means that this <strong>must only</strong> be used with HTTPS deployments. In addition, you have to implement things like ratelimiting and it’s a faff.</p>
<p>How do I deal with it? I’ll firstly explain how the auth works, and then how it applies itself. So, when we load a page we firstly get a list of every user that has access to that page, and the hash of their password<sup class="footnote-reference" id="fr-80-1"><a href="#fn-80">10</a></sup>. That function deliberately returns an <code>Option<HashMap></code> so that we can encode three states:</p>
<ul>
<li>A <code>None</code> signifies that there’s no auth needed - in that case we just return the page as normal!</li>
<li>A <code>Some</code> with an empty <code>HashMap</code> signifies that nobody has access. This could be useful for pages that are only necessary for a specific job and that job isn’t always filled.</li>
<li>A <code>Some</code> with a filled <code>HashMap</code> contains the users we need to auth against.</li>
</ul>
<p>Once we’ve got the list of users and confirmed that the page needs auth, we check that user’s IP against a ratelimiter. At any point past here, if we fail then we return a response that tells the browser that the user is unauthenticated and needs to provide the correct password. If they pass that check, we then check the <code>Authorization</code> header in their request - if it isn’t there then we fail them and their browser prompts them for a username and password. We then try to find their user in the list we retreieved earlier. If we couldn’t find it then we fail them, but not before running a hash on a fake password - this ensures that they can’t use the round-trip-time to work out which users are valid and invalid. If we could find their password hash, then we hash what they provided and compare the two. If they match, then we serve the page and if not we fail them.</p>
<p>But how do we know when to apply the auth? As I mentioned earlier, I am not necessarily a fan of the way Caddy does things and I wanted to have some fun here - there’s a whole system of Users and Realms. There’s a list of Users which are just uuid-name-hash combinations, and of Realms which can match on paths (there’s exact match, regex, etc) and then we can link together users and realms for authentication. For me, this system just makes sense.</p>
<h2 id="conclusion">Conclusion</h2>
<p>This project was a blast to work on - just large enough to have some fun complicated parts, but not so large that it becomes painful to work on. It’s also legitimately useful for me which is a nice side effect. This was also my first project deployed with <code>fly.io</code>, which has been incredible to work with - speedy support, great documentation, reasonable prices, and the CLI works very very well.</p>
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn-0">
<p>I honestly don’t understand why more people don’t <a href="https://en.wikipedia.org/wiki/Eating_your_own_dog_food">dogfood</a> their stuff - it’s been so incredible for finding bugs and new features. <a href="#fr-0-1">↩</a></p>
</li>
<li id="fn-5">
<p>Yes I spent far too long getting wavy text working - even with the help of <a href="https://codepen.io/zachkrall/pen/MWWGMPx">a sick codepen</a>, it was certainly interesting learning more about Hugo & Zola and how to get a version that was repeatable on dynamic pages working. <a href="#fr-5-1">↩</a></p>
</li>
<li id="fn-10">
<p>From the RFC alone, I might add. Definitely because I wanted the challenge, not because there were’t any rust crates I could find to do it for me ;). And yes, I am aware of how horrific an idea rolling my own cryptography is. <a href="#fr-10-1">↩</a></p>
</li>
<li id="fn-20">
<p>And I just updated the files through <code>rclone</code> for reference. <a href="#fr-20-1">↩</a></p>
</li>
<li id="fn-30">
<p>But if you are, feel free to open a PR! My first thought would be some kind of lockfile in the bucket which is the first thing created and the last thing destroyed in the <code>protect</code> flow. <a href="#fr-30-1">↩</a></p>
</li>
<li id="fn-40">
<p>Either by a webhook which lives on <code>/reload</code> or on a timer every 60s. <a href="#fr-40-1">↩</a></p>
</li>
<li id="fn-50">
<p>If I ever need some kind of control/management plane, I’ll probably just do it with a raw TCP socket. <a href="#fr-50-1">↩</a></p>
</li>
<li id="fn-60">
<p>A special kind of request that signals to a server: <em>Heyo! I’d love to open a new websocket connection, can I?</em> <a href="#fr-60-1">↩</a></p>
</li>
<li id="fn-70">
<p>That channel gets set off by either a 60s timer that checks for a different content hash in S3, or a Tigris webhook that lives on <code>/reload</code>. <a href="#fr-70-1">↩</a></p>
</li>
<li id="fn-80">
<p>For persistence, this data is stored in an encrypted file in S3. <a href="#fr-80-1">↩</a></p>
</li>
</ol>
</section>
Vent2023-11-11T00:00:00+00:002024-04-29T00:00:00+00:00
Jack Maguire
https://maguire.tech/posts/vent/<p>You know how recipes often have a reputation for rambling sections about the origin of those recipes? Well, let’s just say that I finally understand why.</p>
<p><a href="https://github.com/BurntNail/vent">Vent</a> is my biggest project ever, at time of writing (over 4k LOC of Rust, not including the SQL or the templates 😲 which is pretty damn huge for me) and has been my baby for the last 8 months. It’s finally proper time that I actually show it off and explain the backend (this definitely isn’t a disguised way for me to re-familiarise myself with the whole thing 😉).</p>
<p>You know the saying <em>Vexation breeds innovation</em>? Maybe not? Well, regardless it’s definitely true for this project. I’d just come into possession of a monstrous spreadsheet for managing various events, as well as a OneDrive folder where the photos got dumped. The old spreadsheet was exclusively for managing the details of the events and had no facilities for managing who was attending the event - this was dealt with by a series of newsletters that died out as people got bored and, in the late stages, teams messages which people were told to react with a 👍 to. The photos weren’t organised at all, were a pain to manage and nobody could see them unless you just asked and then they had to be manually dug out.</p>
<p>No more! To fix this, we go to a self-hosted 100% custom solution with <code>Rust</code> using <a href="https://lib.rs/crates/axum"><code>axum</code></a> and <code>postgres</code>.</p>
<ul>
<li><em>Jack, did you need to do this</em>? God no.</li>
<li><em>Jack, was anyone asking you to do this?</em> Also, no.</li>
<li><em>Weren’t there any prebuilt solutions?</em> Alas, I couldn’t bring myself to look when I saw the potential for such an exciting project.</li>
</ul>
<p>The plan for the rest of this is to go over the project as it is now, then to go over the creation and some of my interesting hurdles.</p>
<h2 id="overview">Overview</h2>
<h3 id="technical-details">Technical Details</h3>
<p>As of right now, the project basically just acts as an access-managed front-end for a <code>postgres</code> database serving <code>GET</code> requests using <a href="https://shopify.github.io/liquid/">liquid</a>, and handles all interactive elements via <code>POST</code> requests. For CSS, I’m using <a href="https://getbootstrap.com/docs/5.3/getting-started/introduction/">Bootstrap</a>, which feels 2015-esque but is the easiest way for me to get some vaguely-competent looking CSS. I like doing things myself rather than using pre-built solutions, but I’m willing to limit this project to one new main thing - the sentence above.</p>
<p>If you’re wondering where I’m going to mention what Javascript framework I’m using, the answer is none. I’m a systems dude who’s trying to learn, and I’ll be honest - this is as much a project to learn the ropes of postgres, deployment, back-ends and the web as it is a project to manage events. I’ve tried lots of the java/type-script-y frameworks, and they frankly don’t really appeal to me. I’ve become pretty damn hooked on the Rust programming language due to the package management, documentation, tooling and unique features like discriminated unions which are missing from loads of other languages so I didn’t want to really use any. I’ve tried <em>(trust me I’ve really actually tried)</em>, but I can’t quite get javascript to fit inside my head - I had the most success with <code>Svelte</code> but it just doesn’t work for me ;(.</p>
<h3 id="user-story">User Story</h3>
<p>So, the main way this project works is through events and peoples’ links to events. An event has a number of properties (that came from the original spreadsheet - one of the project requirements was to be able to import stuff from that <em>relatively</em> easy), and then a bunch of links. They all have links to the people participating, the people looking managing the event, and the photos that have been added. Participant-Event links also can be marked as <strong>verified</strong> - this is useful for making sure people have actually attended the event. A manager-level user has to do that after the event, and in my experience the photo comes in very useful for this.</p>
<p>Finally, there’s a rewards system where after completing a certain number of events, you can be eligible for different rewards. There’s also a system in place for two different entry points each of which have their own requirements for the rewards.</p>
<p>The access control works pretty sensibly imho - for example, someone with manager-level permissions can add and remove whomever they please from events to satisfy reality, but a participant-level user can only add and remove themselves. I won’t go into insane detail - just know that there’s 4 levels (Participant, Manager, Admin, Dev), where the admins can add & remove users, and the dev gets stuff that the admin wouldn’t find as interesting like reloading partial templates.</p>
<h2 id="the-story">The Story</h2>
<p>Here’s the story of my application - a few items have been moved around to better tell the story, but it’s mostly accurate and if you want the 100% accurate version then feel free to read the <a href="https://github.com/BurntNail/vent/commits/main/">Commit History</a>.</p>
<h3 id="beginning">Beginning</h3>
<p>This project started with the beginners guide for the <code>axum</code> crate (for this was my first axum project which slowly expanded in capability as I learnt more about axum & async rust), where you get started with a basic GET/POST which worked well for me. I then added a <code>liquid</code> based templating system inspired by <a href="https://fasterthanli.me">Amos Wenger</a> (specifically <a href="https://fasterthanli.me/articles/a-new-website-for-2020">this one</a> iirc), who’s writings I cannot recommend enough. I then slowly by surely added the <em>other things</em> like participants and photos. This part of the project was probably the second most fun, where I could move mountains with little effort - partially because I hadn’t done much yet so any change was big, and partially because I was free to look at all kinds of different approaches before I got entrenched in a pretty specific style.</p>
<p>I also had a few interesting constraints to work within:</p>
<ul>
<li>I wanted to be able to run the whole thing on just the one tiny VPS (a Linode Nanode which has 1GB of RAM, 1 vCPUs and 25GB of storage) I rent whilst still keeping some space for other stuff (like this blog!). This was mainly an attempt to reduce both vendor lock-in as well as costs.</li>
<li>I wanted this to keep backwards compatibility with the existing spreadsheets - that meant making sure that I had lots of import/export tools.</li>
<li>I wanted this to be easily used by lots of more non-technical users - a JSON feed would never be acceptable and I needed to put lots of work in to get something looking halfway decent (a challenge even with the aid of <code>Bootstrap</code>).</li>
</ul>
<h3 id="authentication">Authentication</h3>
<p>I then took it to the person who actually had final say on adoption on this system and he said that I needed to follow all relevant guidelines that he had to, which in this case included making sure that you couldn’t see photos without logging in. My initial thoughts were something along the lines of <em>balls. i don’t really want to do this, because the more user data i store the more impact my screwups have</em>, but I wanted adoption so I just nodded and said yes. Turns out I was still right, just not in the way I thought I was.</p>
<p>At this point I was pretty happy with the general <code>axum</code> ecosystem, and I came across two main crates that seem to be used for logging in - <a href="https://lib.rs/crates/axum-auth"><code>axum-auth</code></a> & <a href="https://lib.rs/crates/axum-login"><code>axum-login</code></a>. <code>axum-auth</code> seemed to have more downloads but was just one rust file to check headers and get them. <code>axum-login</code> seemed to have more features and even <strong>drumroll please</strong> - Session Management, although it had fewer downloads. Generally, I like writing things myself (as has already been mentioned, often for the learning opportunity), but if I’ve learnt one thing in my years of Tom Scott/Computerphile viewership - it’s that you leave password management to other people, preferably open source people with code that’s been explored & checked over. That having been said, it still wasn’t the easiest to work with - there were a few annoyances like having to rewrite the <code>PostgresStore</code> for sessions to work with my system, as well as getting passwords to work right. I started with the <code>0.6</code> version, which has recently been updated with more breaking changes than I’ve had hot dinners (future update coming - <a href="https://github.com/BurntNail/vent/issues/187">issue here</a>). I then used <code>bcrypt</code> to store the <em>hashed</em> passwords.</p>
<p>One fun solution to a problem is signing up. Since I know my entire userbase in advance, I can just import them all via CSV with blank passwords set. My original solution was to just set the password to whatever the user entered on first login, before I realised that I’d have a sea of password resets as people logged into other peoples’ accounts for shits & giggles. I eventually ended up going with emailed magic links to set the password.</p>
<p>Eventually I got all that working, with the access control coming not long after. As I alluded to earlier, there was still a pain point. This was just managing the access control levels across the whole application - I have checks in a whole lot of places to make sure someone’s not trying to be sneaky with direct <code>POST</code> requests to the server and making sure that no PII leaks.</p>
<h3 id="the-modern-web">The Modern Web</h3>
<p>At this point though it was just something running on my machine, or just via direct HTTP from my registrar (originally <a href="https://hover.com">Hover</a>, but I transferred to <a href="https://www.cloudflare.com/en-gb/learning/dns/what-is-cloudflare-registrar/">Cloudflare</a> for cheaper renewal fees) to my VPS (<a href="https://www.linode.com/">Linode</a> seems to give a decent compromise between cost, configuration & ease of use). I <em>needed</em> HTTPS if I wanted people to send passwords through here - there’s nothing particularly important, and I’ve got the password hygiene not to use the same password everywhere but I can’t say that for all my users. Luckily, one of my very good friends (<a href="https://github.com/HandyHat">HandyHat</a>) had experience here, and walked me through the entire setup of <a href="https://caddyserver.com/">Caddy</a> for Cloudflare using Reverse DNS to get HTTPS certificates.</p>
<p>You might’ve noticed that I’ve put all of my eggs into one basket - if Cloudflare goes down then I’m down. I understand the concern, but I also trust that if Cloudflare goes down then the one instance I run of Vent personally will be of little concerns.</p>
<p>I also chose to use a VPS here, rather than a Docker Container running on a <a href="https://www.digitalocean.com/products/droplets">Droplet</a> or something - this is for multiple reasons. First is that I’m definitely not just using the VPS for this so I save the more projects I use. I also find it insanely convenient to quickly fix small issues in things like SQL queries in production without needing some complicated deployment process - I just <code>ssh</code> in, edit the relevant file with <a href="https://micro-editor.github.io/"><code>micro</code></a>, then build the project and restart it in a <code>tmux</code> session I keep around.</p>
<h3 id="adoption">Adoption</h3>
<p>Eventually, I started some user groups on the system and when nothing went wrong (apart from some misspelt usernames which were easily fixed) I went for the full rollout. Whilst some people still don’t mark themselves as planning to attend, it’s now the de-facto system and those people probably check when and where events are on the main page. I’m really happy with how it went.</p>
<p>No first plan survives contact with the enemy though, so I had to rapidly issue a few updates like publicly marking who uploads photos, and improving my logging infrastructure which at that point just forwarded whatever domain specific error it got from whatever caused the error via <a href="https://lib.rs/crates/thiserror"><code>thiserror</code></a> & the <a href="https://lib.rs/crates/tracing"><code>tracing</code></a> ecosystem. You don’t want to know how long its taken to tune the logs 😔.</p>
<h2 id="fun-technical-things">Fun Technical Things</h2>
<p>You didn’t think I’d finish here, did you? Nah - I’ve got a few fun things which I’ll add to as I remember from the making of this project.</p>
<h3 id="threads-not-of-the-zuck-variety">Threads (not of the Zuck variety)</h3>
<p><code>axum</code> has a <code>State</code> extractor you can use with all of your methods, and whilst from what I’ve seen most people don’t use it as a state manager and just use it as various components I’ve got it all tricked out. For example, I publish an ICS but don’t want the faff of re-making the file for every request (which definitely isn’t what I was doing for a while which caused event duplication issues 😉) so I have one persistent file that gets updated. How do I know when to update it? Every time an event gets updated in such a way that would affect the calendar, I call a function on the State which sends a blank message through a channel to the calendar updater.</p>
<p>I’ve also got threads to delete old sessions (hopefully no longer needed in <code>axum-login</code> 0.7?), and send emails to users.</p>
<h3 id="templating">Templating</h3>
<p>I’ve mentioned that I use liquid templating, but given no more insight than that. Currently, the partials get cached but the actual templates don’t (there’s an <a href="https://github.com/BurntNail/vent/issues/180">issue</a> and it’s next on my list after <code>axum-login</code> 0.7). The main reason for this is that <code>liquid</code> templates are entirely synchronous and the compiler is created in an environment with no way to access <code>async</code> APIs other than caching them beforehand 🤷, so if ever anyone looks at my code and gets confused then that’s why.</p>
<p>If you look over my templates, I’ve also got a couple of invariants (like people always being ordered by group in the view all page when logged in) that allow me to commit some <em>horrendous</em> crimes that make me giggle every time I look at them, and sigh every time I have to think about doing something with that part.</p>
<p>Having now got experience with liquid, I can say that it probably wasn’t the right choice - it can be annoyingly limited to work with (something like <a href="https://lib.rs/crates/handlebars"><code>handlebars</code></a> might’ve been better in that respect), but I’m also happy to accept that it’s good enough and that maybe everything doesn’t always need to be perfect.</p>
<h2 id="gotchas-for-future-me">Gotchas for future me</h2>
<p>These are just a few pointers for people ever embarking on similar projects:</p>
<ul>
<li>You need to setup a <a href="https://developers.cloudflare.com/support/page-rules/understanding-and-configuring-cloudflare-page-rules-page-rules-tutorial/">page rule</a> to turn off SSL for <code>DOMAIN/.well_known/acme_challenge/*</code> because that prevents Caddy from setting up the certificate.</li>
<li>Linode blocks email-related ports (presumably for spam/DDOS reasons?) and doesn’t ever say clearly (just in some <a href="https://www.linode.com/blog/linode/a-new-policy-to-help-fight-spam/">random blog post</a>) so you need to contact support to get them unblocked.</li>
<li>If you’re tee-ing JSON logs into a file and then serving that file, you might need to parse that file and reverse all the elements because otherwise all the most recent logs (which are probably the important ones) are right at the bottom.</li>
<li><code>axum::debug_handler</code> compiles to nothing in release mode - cover your project in them because all they’ll do is slightly increase compile times and they’ll massively help in errors.</li>
<li>Use <a href="https://dbeaver.io/">DBeaver</a> - it’s insanely useful for quickly editing database records and doing other database-adjacent things like complete migrations.</li>
<li>People aren’t joking about backups - you wouldn’t believe how easy it is to completely accidentally wipe something important from (or just wipe the whole) production database. It doesn’t matter how many flags you’ve setup in <code>DBeaver</code>.</li>
<li><a href="https://www.hostinger.co.uk/">Hostinger</a> can shut off servers with too high activity - make sure to double check that doesn’t happen to you, because I didn’t get emailed about it or realise until I checked wanting to see the events and saw that production was down because <code>Docker</code> was being screwy.</li>
</ul>