Disaster Recovery
Changelog
- 2025-06-03: Initial version (incomplete)
Complete self-hosted disaster plan
Lots left. Here's the rough order
- Finish disaster recovery section. It's the "failsafe" and needs the least details because it's largely "here are the keys, and get help from someone else"
- Finish the photos section
- Add a "domains" section
- Finish "things you probably don't want to maintain" section
- Finish documenting all the computers (I think all that's left is Snapcast machines)
- Add some warnings about disposing of hardware (the Pis all have the wifi password on them)
- Write some notes about helping get onto our server using my SSH keys
- Put this somewhere accessible (git gist?)
Overview
This document is intended to help you, Maggie, recover our important media in the event I die.
As you know, we "self-host" a lot of stuff on our own servers. If I pass then most of that stuff is not important anymore.
The things, I think, you want to make sure you have consistent access to are:
- Photos - https://photos.roo.lol
- Passwords (including mine) - https://passwords.roo.lol
- Recipes - https://kitchen.roo.lol
- Digital documents - https://paperless.roo.lol
- Network file share
- Maybe TV/Movies (unlikely) - https://plex.roo.lol / https://jellyfin.roo.lol
- Your laptop backups
Each of these has a section below on how to recover their data.
In general, my suggestion is to just go back to hosted services for each of these once you've recovered the data.
We do not host our own email. As you know we use Fastmail. Your account is already an admin so you should be able to access my mailbox as needed.
Log in, click Mail in the top left, click Settings, click Users & Sharing, then you should see our accounts in the main section with an "Open Inbox" section by me.
I have mail rules to send anything not directly addressed to me to the "Services" folder (under Inbox) Finance stuff goes to the Finance folder (near the bottom of the list, NOT under Inbox)
Accessing accounts and computers
You should have access to everything you need from Bitwarden. Even if the servers are off then you should be able to use Bitwarden from your phone and laptop.
If someone is helping you with the servers then the easiest thing will be for them to extract my SSH key from my desktop, zendo and use SSH if they need to remote into any of them.
The hard drive of my laptop is encrypted - it will not be usable (NOTE: I should come back and put the password on a Yubikey in the house)
The SSH key to use is in /home/jbrechtel/.ssh/id_ed25519
My username on each machine is jbrechtel
Bitwarden Emergency
Computers
We have a lot of computers in the house. Here is a description of each of them, including the WiFi access points. I also put labels on them so you can be sure which is which.
| Computer | Location | Notes |
|---|---|---|
| ATT Router | Office closet | Modem - can be a router and access point |
| Biscuit | Dogfood closet | Wireless access point only |
| Nutmeg | Basement guestroom closet | Wireless access point, can be a router |
| Zendo | Office closet | My "desktop" - lives in the closet |
| Splendor | Probably my desk | My laptop |
| Magic | Office closet | Blue computer - this runs Plex and Jellyfin |
| Pandemic | Basement guestroom closet | Black tower - it's our main server |
| Azul | Office closet | |
| Moiraine | Office closet | This is our main router |
| switch-01 | Office closet | |
| switch-02 | My desk | |
| Hive | Probably my desk, maybe your parents | This is a secondary/offsite backup server. See Disaster Recovery below. |
- AT&T Modem (this can also act like a router, see the WiFi Suggestion section below)
- Biscuit (this is in the closet with the dog food) - it's just a wireless access point
- Nutmeg (this
WiFi suggestion
It might not make sense to do this right away if everything is still working, but eventually you'll want to simplfiy the WiFi setup.
Here's my suggestion:
The simplest thing to do is to get rid of almost all of our own stuff - except for one WiFi access point - "Biscuit"
I suggest you do the following
- Turn on the WiFi on the AT&T Modem/Router and set that up to be the main router - it needs to stay upstairs since that's where the fiber is at.
- Put what is currently "Biscuit" in the basement
- Connect "ATTRouter" to "switch-01" (any port on either, doesn't matter)
- Connect "switch-01" to "Biscuit" (use one of ports 1-4 on "switch-01" and the port that has a lightning icon on "Biscuit")
Note that "Biscuit" does not need a separate power cable when connected to "switch-01" if you used the ports I mention above.
As for setting up "ATTRouter" to be the main router - the easiest thing to do is just reset it to the factory defaults. An AT&T tech can help with this. Once it's back to defaults you will probably want to set the WiFi network name and password to something easier to remember and that will be your upstairs network. "Biscuit" will be the downstairs one.
The password for the ATT Router is shared in Bitwarden under "ATT Router"
Recovery
TODO Photos
Passwords
This should be relatively easy. Just a handful of straight forward steps below.
You should be able to access all the passwords from your phone or your laptop even if everything at home is off.
- Open Bitwarden and go to Settings -> Vault Items -> Export vault
- Leave "Export from" set to "My vault" and the format as JSON
- Export vault again and select "Leonard-Brechtel Shared" and leave the format as JSON
- Create a new account on the hosted version of Bitwarden at https://bitwarden.com (you should probably get a paid account, I don't know how they tier the features)
- Sign in to your new Bitwarden.com account go to Settings -> Vault Options -> Import
- Import each of the two files you just exported
- Confirm both your personal and our shared passwords are in the new Bitwarden.com account
- Delete the JSON files and empty your trash so you don't accidentally leave them on your computer unprotected.
TODO Digital documents
This covers all the files at https://paperless.roo.lol
TODO Network file share
Recipes
The software we use, Recipya, lets you download all of the recipes as separate PDF files in a single zip file - but it obviously needs to be running to do that.
If the server is up
- Go to https://kitchen.roo.lol
- Click the "J" in the top right (it does not require you to log in)
- Click Settings
- Click Data
- Change "JSON" to "PDF" in the Export section
- Click the download icon to the right of the now selected "PDF"
If the server is down
You'll need help from someone to start the site and then follow the instructions above.
The site is on the server labeled "pandemic.brechtel" - it's the big black box in the basement.
TV/Movies
Where to put them
Obviously you can just let these go because you won't be uploading them to some online service.
However, if you want to keep them then you'll want to buy a pretty beefy external hard drive to store them on.
At the time of this writing they take up about 11TB of space. 8TB of TV and 3TB of movies.
How to get them
If the servers are still running then you should be able to access them from the network share from your laptop
- Hook up an external hard drive to your computer - you may need to format it
- Open Finder
- Hit CMD+K
- Select or enter the pandemic.brechtel server
- Select select the TV folder
- Copy the files to the external hard drive
- Repeat for movies
These are the raw video files - you can play them by double clicking them and they should open in VLC.
The other things on our servers include (but probably aren't limited to)
- Multi-room audio setup
- Smart home / HomeAssistant (see notes on this below)
TODO Your laptop backups
Things you probably don't want to maintain
Smart home / HomeAssistant
As you know the whole system basically works fine even if the server is off. There are a few exceptions / annoyances but nothing major.
The locks will still work, the camera in Heidi's room will still work, all of the "smart switches" and bulbs and lamps will still work like normal.
I suggest you just turn it off - it will be a pain to maintain it. If you really want to maintain it then check out the r/homeassistant subreddit and ask for help. The community there is generally very helpful in these situations from what I've seen.
Phillip Campbell is also into home automation so he might be able to help.
The things that will stop working if the server is turned off
- Dimming lights with sunset
- Ikea buttons
- Nursery buttons for lights
- The dance party lights will not work anymore
Disaster Recovery (if all the hardware at home is gone)
If something happened to the house and the computer are all fried then you still have two offsite backups. Restoring things becomes a bit trickier but not impossible.
You'll need help from a techy friend.
Backups are stored using the backup software "Kopia" (https://kopia.io/) There are stored in two locations
- Backblaze https://www.backblaze.com/ - the password for this is in Bitwarden and shared with you.
- Hive (at the time of writing, this is at home but I plan on giving it to either Trevis or leave it at your parents)
I use Kopia for storing offsite backups. https://kopia.io/
The information needed to access this repository is in Bitwarden under "Kopia Disaster Recovery Password"
The backup here should have a regular snapshot of everything important at home (NOT movies and TV shows)
Using Kopia, you may need to click "Disconnect" on your existing laptop backup repository. Then you should see "Select Storage Type" and click "Use Repository Token" and enter the token from Bitwarden.
Once that connects (it will take a few minutes before you can view the Snapshots in my experience)
-
Select "Snapshots" in Kopia
-
Change "Local Snapshots" to "All Snapshots"
-
Access the files needed as described below
-
all our photos are in
/nas1/photos.roo.lolin thelibraryfolder. The albums don't exist anymore but hopefully you can recreate them by date and with facial recognition. You could get someone to help you run Immich again from this backup. -
digital docs are in
/services/paperless.roo.lolinmedia/documents/originals -
The files on our "network fileshare" are in
/nas/files -
Recipes are there but you'll need to start the site (in
/services/kitchen.roo.lol) and follow the instructions for it above to get them.