The Backdrop Link to heading
I love computers. I’ve been using them since before I can remember, and I started coding when I was thirteen years old. When I was probably fourteen, my dad gave me a laptop that I think was a family laptop at one point. I very quickly installed Ubuntu on the laptop and my love for Linux began.
Even now as an adult, I have several machines in my house that serve various purposes. I hung on to that same laptop, and probably a year ago I decided to try and revive it to use as a multi-purpose home server. I installed Arch Linux on it, and started setting up some processes to play around with.
For many years, my wife has collected box sets of shows that she likes to watch. I started to get tired of swapping discs out of our janky DVD player every five or six episodes. We also wanted to be able to pick up where we left off if we wanted to watch the same show in a different room. In classic software developer fashion, I decided to turn a five minute problem into several hours of a solution and set up a somewhat complicated media management setup.
The media manager itself was super easy to set up. I originally used Plex, but because of the age of the laptop, it struggled to serve content. I tried Jellyfin and had much more success. I also already pay for Google Drive, so I set up an rclone daemon to trick Linux into thinking that my cloud storage was a network drive. I would like to move away from this soon, but for now it works pretty well. All we had to do was upload our shows to the cloud, and Jellyfin does the rest.
But then we had Thanksgiving.
The Problem Link to heading
Last year it was my parents’ turn to have us, and so before we left for that, I wanted to make sure we could still watch Friends while we were gone. It should be simple enough, all I need to do is set up my router to port forward my Jellyfin server, and then we can use our dynamic DNS to easily access our content. When I did this, I switched my phone to cell data, but I couldn’t access the server. I tried port forwarding a different device, and I was able to reach that one. What gives?
Well, I did some reading around, and ran some tests on my machine. What I found
is that the external traffic was indeed making it to my server. But for some
reason it was not making it back to the original requestor. I was able to
inspect which network interface the traffic was going through. I saw the request
come in on eno1, but the outgoing traffic went through tun0…
Oh, that’s right, I’m running a VPN.
I care very deeply about internet security, and because of that, I will run a VPN on just about any device that will run one. That includes my server. So basically what’s happening is that my router receives a request, and using its NAT will forward that request to my machine. My machine processes the request, and sends the response. But because there’s a VPN running, all outgoing traffic goes through the VPN. When the response gets back to the client, none of the information in the packet makes any sense, and so the client drops it.
I’m going to have to go deep into Linux network routing.
With a sigh, I made some coffee.
The Solution Link to heading
The solution I ended up with is an amalgamation of solutions I pieced together from various articles and StackOverflow answers. I am not a networking expert, but I will do my best to explain what’s happening here. I created a script that runs on boot after my VPN gets connected. I will share that script at the end of this section.
First, I want to add a firewall marker:
ip rule add fwmark 2 table 50
This just establishes that any packet with a firewall marker (fwmark) of 2
will be routed to table 50. These numbers can be pretty much whatever you want
them to be; they don’t have any meaning outside of our script.
GATEWAY=192.168.0.1
ip route add 0.0.0.0/1 via $GATEWAY table 50
ip route add 128.0.0.0/1 via $GATEWAY table 50
ip route flush cache
For this part of the script, $GATEWAY should be the gateway address for your
normal, non-VPN internet traffic. In my case, that is 192.168.0.1, but this
may be different for you.
Lines 2 and 3 here are routing all traffic to my gateway, but because of the
end of the lines, table 50, this logic is only applied to packets that are
routed through table 50. Because of our fwmark rule from before, you might
see where we are going with this. If we can determine which packets came from
outside my network and should be routed back that way, we can mark them with
fwmark 2 and they will be routed through the gateway instead of the VPN.
The last line just makes sure the routing rules apply currently.
PORT=8096 # Default Jellyfin port
iptables -t mangle -A OUTPUT -p tcp --source-port $PORT -j MARK --set-mark 2
Here we are using a tool called iptables which can be a little confusing to
those of us who are not networking experts. Let me break this down a bit more:
iptables \
-t mangle \ # Selects the mangle table
-A OUTPUT \ # Adds an entry to the OUTPUT chain
-p tcp \ # Filters by TCP traffic (HTTP is TCP!)
--source-port $PORT \ # Filters by the port on our machine
-j MARK \ # We want our action to be MARK
--set-mark 2 # Set fwmark to 2
By using this command, we have effictively captured every outgoing packet from
Jellyfin, and marked it with fwmark 2. This means every packet from Jellyfin
will hit table 50 and be routed through the gateway instead of the VPN.
We have one more thing to do:
TUN0=10.0.0.1 # VPN interface IP
iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE --source $TUN0
To try and make this super simple, this rule is essentially saying that for any packet that is outgoing through the VPN interface, modify the packet’s source IP address to be that of our server’s VPN interface IP. This is necessary for port forwarding. Instead of trying to explain how this works on a network level, I will instead reccomend you read this StackExchange answer.
I rebooted my machine, and…
Huzzah! My phone on cell data was able to reach the Jellyfin server! I checked
again on my local network and… no connection. In my script, I put the following
line before the first iptables command we went over:
iptables -t mangle -A OUTPUT -s 192.168.0.0/24 -d 192.168.0.0/24 -j ACCEPT
Basically, in the same mangle table and OUTPUT chain, any local network
traffic should just continue through like normal and not get marked. The -s
and -d are filtering by source and destination IP addresses. If both the source
and destination are the same, this rule applies. This will prevent any sorcery
around what we’re doing in iptables from affecting local traffic.
This is pretty much all I had to do. My server is still available locally, and I’m able to access it remotely. Some of you reading may wonder why I wouldn’t just put my server behind a VPN and then access the entire machine through a secure tunnel. The answer is that I would actually like to do that, and I do have a VPN on my local network that would allow for that. The issue is that not every TV will let me run a VPN client, and it’s much easier on that end to just have an open port to access. The other answer is that I never figured out how to access this machine through my home VPN. I need to try and do some more routing magic.
The Script Link to heading
As promised, here is a (slightly modified) version of the script that I have running on my server today.
#!/bin/bash
# Vars
GREP=rg # change to grep if you don't use ripgrep
# Get the VPN interface IP
TUN0=""
while [ -z "$TUN0" ]
do
sleep 5
TUN0=$(ip addr s | $GREP 'inet (\d+\.\d+\.\d+\.\d+).+tun0' -r '$1' -o -m1 | sed 's/^\s+//g')
done
# Set a firewall rule to route packets marked 2 to the filter table
ip rule add fwmark 2 table 50
# Create a route in the filter table that routes to the eno1 gateway
ip route add 0.0.0.0/1 via 192.168.0.1 table 50
ip route add 128.0.0.0/1 via 192.168.0.1 table 50
ip route flush cache
# Catch LAN traffic and don't mark it
iptables -t mangle -A OUTPUT -s 192.168.0.0/24 -d 192.168.0.0/24 -j ACCEPT
# Allow incoming WAN connections for given ports
PORTS=(8096) # Easy to add more ports here
for port in ${PORTS[@]}; do
iptables -t mangle -A OUTPUT -p tcp --source-port $port -j MARK --set-mark 2
done
# Necessary for port forwarding
iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE --source $TUN0
Note that this script assumes tun0 to be the VPN interface, which may not be
the same for you. I’m sure that there’s an easy way to determine which interface
you want to use from a script, but for my purposes, I know that it will always
be tun0 and so I left it in the script like that. The other part that’s
hard-coded is my LAN address space 192.168.0.0/24. You may need to change this.
The End Link to heading
Thanks for reading! I hope this write-up was able to help you in your quest to serve content from your home server. I struggled to find anyone online whose problem was similar to mine. Most other problems I found were very complex and had a lot of moving parts. Trying to parse through that information was very exhausting. I’m hoping there’s a lot more people like me who wanted a simple solution to their simple problem.
If you enjoyed this write-up, stay tuned for more posts from me. I’m trying to improve my writing skills and share a little bit of what I’ve picked up in my years of learning. I mentioned earlier in this article that I use a dynamic DNS service. My router used to take care of that, but then one day it stopped working. I ended up writing a script to take care of that too. Maybe I’ll share that next time!
If you want to have a conversation, just reach out.
Subscribe to the RSS feed.