As is often the case, when I (or most people, I suspect) start a blog on a subject, it’s something they were already doing and suddenly decided that blogging about it might be fun, or even helpful. What follows is the story thus far….

My home networks since the inception of the common use of post-dialup internet have generally been the minimalist one or two hardwired PCs connected to whatever switch/router combo the internet provider offered. That was plenty at the time. When we moved to our current house in a largely rural neighborhood in late 2010, it coincided with both a desire to execute some home automation and the proliferation of more smart devices, more phones, more laptops, more tablets, more TVs.

The only internet providers available at the time were DSL from the ILEC CenturyLink and the usual suspects for conventional satellite internet, like HughesNet and Viasat. There may be others <shrug>. CenturyLink DSL was adequate for general internet and was also reasonably stable, though there were a few outages during the years we had it.

When I started this post, I had intended to tell the whole story of the network and home automation, because there is a fair bit of crossover between them, but now it occurs to me that one massive blog entry is going to be hard to absorb and enjoy and it’s already too long, so I am going to to split them into two related categories, Home Automation, which I started a freakin’ decade ago and I suppose this one will just be about the network in general.

What follows is roughly in chronological order, but not necessarily strictly so.

The router that came with our DSL service was a decent one, I suppose. I think the first one was a Westell and at some point, they replaced it with a ZyXel. Eventually, I wanted more control, so I bought a Belkin router to put behind it, or maybe the Belkin had a phone jack for DSL? Purchased from BestBuy, if I remember right. The Belkin had 5GHz Wi-Fi, which helped with some devices. The factory Wi-Fi password for this router was a decent one, easy enough to remember, so I’ve kept it a couple of routers later. I also eventually got a Belkin Wi-Fi extender. All the network gear is in a bedroom at the corner of the house and sometimes the signal was a bit weak in the kitchen at the opposite corner of the house, but importantly, where we spend much of our time.

The Wi-Fi extender was not enough to provide decent coverage in the workshop. The shop is roughly 100 feet from the house and it is even near the corner bedroom where the Wi-Fi is located, but the signal was still weak.

With the exception of these two BestBuy purchases, from here on out when I mention ordering something, it is pretty safe to assume I probably ordered it via Amazon. If I didn’t and its important, I’ll comment to the effect.

Anyway, based on some experiences with point to point links between buildings at work, in late 2013, I got a set of EnGenius ENH500 outdoor access bridges. This was an extremely simple and robust system that lasted nine years, through power failures, thunderstorms, ice in winter and Texas sun in summer. They were just never the problem. Since BestBuy was handy, I also got a little Belkin 8 port switch and that was plenty to support a PC and some radio gear in the shop.

Cheaping out on Wi-Fi for the workshop *was* a problem. I had a surplus Cisco WAP that seemed to be a good unit. It had 5GHz Wi-Fi, three antennas for signal diversity and a good reputation otherwise. Mine however, had a tendency to go to sleep. Rebooting it would bring it back, but it would just freeze and stop working again later. I basically dealt with it because the only thing I really needed Wi-Fi for was my phone when I was out there.

If I recall correctly, the CenturyLink DSL was spec’d out at 7.5Mbps download and 2.5Mbps upload. It never met those exactly, but for most of the time we had it, it was pretty solid for 5M down and 2M up. I had little need to work from home then and when I did, it was almost always consoling into a Linux server or some switches and routers, so it was almost all text.

Eventually, however, DSL service began to degrade. I could turn in a ticket and the tech would reset something or replace some card in the DSLAM and get it kinda back up, but after a couple more years, it became a case where dialup internet would have been only a little slower and probably more reliable. I think CenturyLink just didn’t want to invest in upgrading the infrastructure for a handful of houses in a rural neighborhood. There were no new customers to be had.

Terrestrial wireless internet began to look appealing. I don’t remember if we had more than one provider come out, but we did eventually get wireless internet from OneSource in November of 2017.

The installation was what I would classify as marginal. The LTE modem was put on the roof of the house, but at the minimum viable height, just barely shooting over an outbuilding next to the house. Even so, it worked great, 10M/2M as promised and pretty reliable and troublefree except when the growing season was in full tilt and the trees were in full foliage.

One particularly verdant season, it was affected more than usual, but coincidentally, they were upgrading equipment on the tower, which was to require changing the equipment at the house.

The tech that came out put the new modem on a tall roof mount at the peak of the roof, where it gained 8 or 10 feet of altitude. The new equipment was also good for 25M/5M, so I signed up for the upgrade.

I also befriended that tech, which is good not only because he’s a cool guy to hang out with, but it also allowed us both to benefit from my willingness to experiment and help him by testing stuff. It also helps that I’m not Joe User when it comes to this stuff and can converse with him intelligently about issues. And blacksmithing. And firearms.

Somewhere in here, a coworker who had just changed out his whole home network to Ubiquiti gave me a Cisco SG200-26P that he no longer needed. It would sit unused for a while, but would one day become very important. 🙂

OneSource has rocked along since then pretty solid. The old equipment on their tower could only go as high as 10Mbps, but the tower antenna was basically pointed south southeast and we are pretty much due southeast of the tower, so we enjoyed a strong signal, even though the antenna on our end was barely over the roof of the craft shack. The new equipment has higher capacity, but the antennas are pointed due north, south and east, putting us in a comparatively weak crossing of the south and east antennas. Our modem will connect to either. I forget which is which, but connected to one, we get the full 25Mbps, but it’s not as stable and reliable, dropping out occasionally. On the other, we get 10-15Mbps, but it’s more solid. I tend to stay on the solid side because….

Now I work from home almost exclusively. Long story short, my company moved headquarters from near downtown Fort Worth, which is a nice 35-45 minutes drive to Addison, which is a solid hour to 90 minutes in the morning and longer to get home if you don’t leave either well before 4PM or after 6PM. We had negotiated being able to work from home for a number of days, and I often used my “work from home” days to work from the old Fort Worth office on certain days because I had pistol matches in or near Fort Worth, so that was even closer than working from home.

Then came the unspecified virus of unknown origin. <cue dramatic orchestra hit>

Suddenly, working from home was pretty much the only option and 2M upload that wasn’t always 2M was just barely enough. I could join conference calls, but I was the only one without a camera on.

Oh, and then I broke my shoulder and had total shoulder joint replacement surgery, but the virus was a bigger issue.

During this time, my company was in the process of changing to RingCentral telephones and since my gig with the company is the phones, that Cisco SG200 switch mentioned above came in handy as 12 of its 26 port provide PoE and it has a 100W power budget. I could connect all the phones I needed for conducting labs and configuration testing and all the stuff one might otherwise normally do at the office.

When I first heard about Starlink (I am, among other things, a bit of a space nerd) I signed up to be notified when service was going to be in my area. Shortly, I received notice that they had a Beta program wherein one could pay a $99 deposit and end up on a list of possible beta testers in an area. I did that in February of 2021 and tracked the news, but otherwise went on with my life.

In a move to improve Wi-Fi coverage in the house, in June of 2021, I decided to upgrade the old Belkin router. It had no external antennas, so I chose a TP-Link unit with an array of them, hoping to maximize RF flux in the house. I also put the freshly liberated Belkin router out in the workshop as a simple access point and finally had reliable Wi-Fi out there.

I was pretty impressed with the features and benefits of the TP-Link router, including their mesh products. Combined with this router, I could backhaul ethernet to the workshop and have another TP-Link access point out there and they could be the same SSID! No more having to connect to the workshop Wi-Fi out there and reconnect to the house back here. So, by July, I ordered a 2nd TP-Link router of the same model.

Well, it turns out that the router product apparently can’t do the ethernet backhaul *as an access point*. It can be the controller and any of several models of mesh capable access points could backhaul to it over ethernet and use the same SSID, but not two of this model back to back. Poo. Rather than return it, I still deployed it as an access point, but now I renamed the networks HippyHollow for the house and FlyingDog for the workshop, though I still kept the password that my family already knows.

October got busy, partly because Starlink’s delivery estimate narrowed down somewhat to “Late 2021”, so I started tweaking my infrastructure to accommodate what I hoped would be delivery soon. Having studied some of the requirements for Starlink, I knew that the same trees in our back yard that caused us to install the OneSource modem on the far west end of the house would never work with Starlink’s dishy, so I formulated a plan to install the dish, once received, on the roof of the workshop because there are no trees over there. I would then use a VLAN to backhaul it over the point to point bridge to the house and plug it into the router. To that end, I ordered a couple of TP-Link switches with VLAN features. For a $20-30 switch, these are pretty nice units.

Around this time, I saw a Network Chuck video extoling the benefits of the pfSense firewall/router. It occurred to me that having a router with multiple WAN ports could allow me to, at least for a time, keep and use OneSource even after Starlink is in place and my TP-Link router had only one WAN port. So, early in October, I ordered a Netgate 1100 appliance with pfSense+ preinstalled. Sadly, it would be mid November before I could get it deployed, coincidentally while my wife was away on a cruise. That way, if I got stuck on something and killed our internet, she wouldn’t have to suffer, and thus, neither would I. 🙂

The wizard in pfSense works pretty well and getting the router up was easy. It was also easy to break the config enough that it stopped all traffic. It was then also easy to reset it to factory and set it all back up again, having learned my lesson. 🙂 I reconfigured the TP-Link router as a simple access point, like the one in the workshop and kept it in place.

It was around here that I had ordered something that required a signature on delivery. The FedEx guy kept coming to the door while I was home and apparently knocking with a feather so that not even the dogs could hear it and leaving notices that they were about to return my thing to the sender. I called in and probably completely irritated someone that day, but eventually we met up at my door and I got my stuff. The whole experience inspired me to install a doorbell camera, partly to be able to collect evidence, but also because when there is an obvious camera, people tend to act differently, in this case, hopefully try harder to complete a delivery.

Cameras are largely considered to be in the home automation segment, but most of the story of mine is pretty network centric, so I have overlapping parts of the story in both blogs.

Because I knew I would still be upload limited for a while, I wanted to avoid anything that was dependent on the cloud (you know, what we used to call servers, but are now just someone else’s servers), so Ring was right out. I dug around and found that Amcrest had decent products and while <buzzword> cloud </buzzword> (and don’t even get me started on the people who now call conventional servers the “private cloud”) was an option, it was not required; they could record to a local SD card and they could be administered locally. I ordered an Amcrest AD110 doorbell camera and two other cameras suitable for watching our driveway and watching the horses in the barn.

They were pretty easy to connect. The barn camera would have to wait a while, but the doorbell connects to the existing doorbell wiring for power and has some kind of terminator adapter thingy that connects across the existing chime in order to operate it. The camera appeared to come up mostly, but would never successfully connect to the network. A bit of Googling revealed that it requires at least a 16V doorbell transformer in order for Wi-Fi to operate correctly and sure enough, mine was only 10V. Quick trip to Lowes and a quick trip to the attic to swap that out and it came up beautifully. Literally a couple of days later, I am working at my desk and the doorbell motion detector goes of. It turns out to be that the horses got out!

If I am being completely honest, the AD110 does require the largely cloud-based app for configuration of the camera, but it pretty seemlessly integrates, so it’s hard to tell what might be local and what might be cloud. I should test for that.

The driveway camera was more involved because I needed to run an ethernet cable to where I wanted to place it. Happily, most of our attic is pretty open and the wire was easy to run. The camera runs on PoE, so again glad to have that Cisco SG200 switch.

I didn’t have PoE in the workshop/barn, so I had to order a suitable injector. Again the camera came up easily and I have grown to appreciate Amcrest products.

I found that the 32G SD cards that I had readily available for the cameras’ local recording would fill up pretty fast, so I ordered some 256G cards from Amazon. They arrived and I’m not sure what the issue was. They were SanDisk class 10 devices, but they would randomly unmount and recording would stop, sometimes requiring a reboot to restore functionality. [ed: now I know that the write speed of SD cards is important with video] I put the 32G cards back in, but at even pretty modest settings, they would fill and auto overwrite in just a few days, particularly as each alarm event is recorded at a higher rate and I was still getting a lot of video motion detection alarms as I tweaking all that.

I had noticed that one of the recording options was to send to a Network Attached Storage device. I thought that might be a good solution, plus we didn’t really have any appreciably valid data backup strategies for our laptops. I did some shopping and YouTubing and decided that a single drive Synology NAS would probably suit me well, so I ordered a DS120j with a 2TB disk.

Almost immediately upon getting it set up and learning about its features, I found that Synology has a Network Video Recorder app for their NAS called Surveillance Station. That turns out to be way better than just using the NAS to store the camera’s recording files. It is a fully featured NVR and video is stored locally.

However, nothing is perfect. I discovered that Surveillance Station is license bound. The NAS comes with licenses for two cameras. When you have two cameras and go through the procedure to add a third, you are given an option to continue after selecting which of your existing cameras to disable or to cancel. The licenses are not particularly cheap, either. However, I decided that I liked the advantages enough to pay it, and it was still cheaper than a dedicated NVR. So, thinking about where I wanted to put cameras, I ordered a 4-pack of licenses for a little less than $200.

Then is when I learned about the other imperfection. The essentially minimal hardware NAS I had would support a maximum of five cameras. The system resources are the limitation and Synology, wisely I suppose, will not let you over subscribe the hardware. So now I had five usable licenses and one unusable $50 license. Sigh. The next higher rated hardware is a two drive unit that can support 12 cameras and the next one after that does 25. If I feel the need enough, I can migrate my existing disk to it

I know that I can migrate a disk because I have done it once. By then, I had four cameras connected and I was also taking up some file space with several hundred uncompressed photos for astrophotography.

I had fears that I would outgrow my 2TB disk. By December, I ordered a 6TB replacement disk and a 2TB USB 3 external SSD and performed a complete backup using one of the Synology provided tools. It was a very slow process; I left it overnight because I got tired of waiting for it to finish. [ed: yes, the write speed of the 2TB USB drive was a big factor in that slow process] Synology treated the new 6TB disk like it was a new out-of-the-box NAS. Once it completed all its formatting gyrations, I plugged in my backup drive and started the restore process. It took another several hours, but worked flawlessly. The only loss was the footage that would have been live between in end of the backup and the end of the restoral.

It suddenly occurs to me that it was probably slow on backup because Surveillance Station was hitting the drive with 4 new live streams the whole time it was trying to back up the disk. Perhaps I will remember that and take it offline first next time.

Network Chuck strikes again. In my hope to further avoid cloud services for things in my house, I think the benefits of reverse proxy are attractive. In this particular instance, Network Chuck deploys it in the form of a free Kemp load balancer.

I should have made notes setting this up because getting the hardware in place to run a Proxmox hypervisor upon which the Kemp VM can be deployed had it’s own story. I had a fanless PC that I once tried to use as a Windows jump box for ham radio applications out in the workshop. I never could align all the sleep features such that it would not turn off after a couple of hours. Proxmox of course has none of those kinds of features. I think I upgraded RAM but I definitely added a 120GB SSD. BTW, I did try to load the free ESXi hypervisor, but there was something it didn’t like about the machine, but Proxmox loaded and ran just fine.

I did not finish getting the Kemp stuff up and running. Rather than strictly follow the examples in the video where Chuck shows us how to set up a domain name with FreeNom, I wanted to use an underutilized name I already have with a different registrar. I had some trouble getting it to do what was expected and I have not yet finished the process. I will revisit at some point. [ed: plus, Starlink’s use of CGNAT pretty much breaks this particular application of reverse proxy, at least not without outside help.]

More cameras: The driveway camera can *just* see the front gate and I wanted to put a camera out there by the gate for better coverage. My phone has a usable signal from the house Wi-Fi , but being 170ft from the house means power could be an issue.

I ordered a camera that sounded promising, an outdoor Wi-Fi connected camera with a USB power cord and a 12V to USB adapter dongle, the intent being to connect it to the solar charged battery that is already out there for the gate opener. Unfortunately, it would be late March before I got around to setting it up. In the mean time, it also occurred to me that the gate draws nearly nothing unless the operators are actually moving the gates, but the camera is going to draw something 24/7, though due to the nature of the USB connector, I could feel confident it should be 10W or less. As the gate is a kind of security and safety thing for which it would suck to have a dead battery, I decided I would just add a separate battery, solar panel and charge controller for the camera.

The panel is a 25W panel, I’m sure bigger than is needed. I also found a charge controller that has USB outputs, so I don’t even need the power dongle anymore. I’ll repurpose those to our kayaks or something. I already had a couple of battery boxes and elected to rob a lawn tractor battery from a mower and replace it later. [ed: as time has passed, this has been a reasonably reliable setup, but several days of cloudy conditions is enough for the battery to run down enough to lose power to the camera. I am not sure if I need to address that with more PV panel, more battery or both.]

I was not completely surprised to find that the Wi-Fi signal from the gate to the house is ok, but not really solid. My intent was to deploy the old Belkin Wi-Fi extender from years past. I have seen it recently, but I could not find it. I presume it is in a box that I didn’t open. Instead of continuing the search, I ordered a BrosTrend AC1200 to extend the house WiFi into the garage, which is closer to the gate. Another AC1200 will figure prominently into the Starlink story.

The EnGenius WDS bridge between the house and workshop/barn had been in solid service for nearly 10 years when something started getting flaky on one or the other. The received signal on the workshop end dropped significantly, -100 dBm, down from -30 or so. It’s unclear whether that one had a receive problem or the other end had a transmit problem; they would look the same if you just look at signal strength. The unit on the workshop end is fairly near a light that is on all the time. During the spring and summer, there is an absolute bounty of bugs, spiders and geckos feasting on one another, so I thought perhaps some bugs had gotten inside the unit, as I’m sure anyone might suspect given the conditions.

Unfortunately for the story, the unit is well made and the electronics are very well protected from the ingress of dust, let alone critters. It was very clean inside.

Since my plans for Starlink would depend on a reliable link between the house and workshop, I started shopping. My original plan was to stick with what works, another pair of ENH500s, or since it’s been 10 years, whatever may have replaced them in EnGenius’ lineup. They do have a newer model, but they were about $200 a pair, which is not super expensive, but I kept shopping anyway. There were several units that were sold singly for less than $100 a pair but I had just a little instinctive reservation about them; hard to explain. However, I also found that Ubiquiti has units that were physically similar, identical perhaps, to the suspect units. Ubiquiti has a good reputation, so I went for a pair of AirMAX LiteBeam ACs, $140 delivered. They arrived quickly, were easy to mount and easy to configure. With default settings, they were good for 300-ish Mbps, which should be plenty, but I tweaked with settings and got quite a bit more potential data rate out of them.

I was in California for a work telephone project when I got the email: Starlink was ready to ship! [ed: ironically, I was in Hawthorne that day, where Starlink would ship from] I quickly confirmed my order, forked over the balance and also ordered the ethernet adapter that is required to plug it in. The older version had an ethernet jack, but for the new one, it’s an add-on accessory. There is much grumbling about that on r/Starlink.

Since the whole Starlink installation is a week long experience of its own, and as I am writing this, it I have *just* completed it, I’ll continue that story in a post of its own.