Ignorance Is Bliss

One of the many jobs a Synology NAS can do is to serve as a syslog server. Most network gear can support sending it’s logs to a syslog server and in doing so, you can have one place to go to review logs on all those devices.

On the other hand, what you don’t know can’t hurt you, right? Right?!

Seriously, the issues I have discovered in just over 12 hours of using Log Center are arguably not actually serious, but they are bothersome just because now I know.

I configured Log Center to collect logs from my pfSense router/firewall, two Cisco switches and, just for the entertainment value, one of my IP cameras.

Activity from the switches is very light. I verified that it would log a port being disconnected and reconnected and other than an hourly DHCP refresh from each switch, they have been pretty quiet.

The camera is also pretty quiet. It mostly shows login and logout activity from both my laptop and the NAS while playing with some settings and silence since then.

The router, on the other hand, is quite chatty. It also has more granular control over what gets sent to syslog.

Note that I have unchecked Firewall Events. Before doing that, the log was just stupid busy. The firewall blocks a LOT of traffic. I do need to analyze that traffic at some point. Some of the blocked traffic is internal.

The thing that bothers me but probably shouldn’t is the number of DHCP requests from stuff that is obviously online and operating.

Does my camera out by the gate really need to refresh it’s IP every 4 seconds ALL DAY? The camera alone accounts for almost 71% (32,552 out of 46,198) of the log events between midnight and a bit after 11AM when I pulled the log to look at it. Why does a camera that is online and operating have to do that?

I will figure it out….

I Re-re-renumbered My Home LAN

As part of deploying Starlink as my primary internet provider, I had to renumber my LAN. Starlink uses 192.168.1.0/24 as its default client network and that is not alterable, so far as I can tell.

pfSense didn’t know when I started configuring it, so it didn’t argue, but also didn’t work. I finally took a guess that the same IP on both sides of the router could confuse it and changed mine to 192.168.2.0/24 and my stuff started working!

Well, it did until the first time I connected to my work VPN, Cisco AnyConnect. At first, I didn’t specifically notice that the VPN was the trigger. I was working away at something and wanted to check on my camera system to see if the mail had arrived, but I couldn’t get to my NAS. I didn’t panic or anything, but later (without realizing that the VPN was disconnected, I was able to reach the NAS and cameras just fine. Some time later still, I connected the VPN then tried to refresh the camera display and it finally dawned on me that with the VPN connected, I couldn’t get to the NAS. I tried to verify the issue by connecting to something else, one of the switches I think, and: no luck. Disconnect VPN, get right in. Reconnect VPN, get nowhere. Well, except for my router. I suppose the VPN needs the default gateway.

AnyConnect info includes a list of secured routes. It turns out there are secured routes cover most, though thankfully not all, of RFC-1918 address space. All of 10.0.0.0/8 is either in use or allocated internally. Quite a few legacy locations use various ranges within 192.168.0.0/16, especially in the bottom half or so. Finally, there are only a few used within 172.16.0.0/12.

I dashed off a note to the network team exlaining my issue and asking for assistance in finding a subnet I could use without interference, but that note as yet has not been answered. I used my Linux jumpbox, which is a VM in one of our data centers, to investigatively Nmap parts of this space, looking for some vacant space. Running Nmap from home is limited to only the secured routes provided through AnyConnect; running it from the data center should also detect systems that are not specifically accounted for in the VPN.

First run through, it looked like a large block near the top of the 192.168.0.0/16 space should be open. I chose 192.168.250.0/24 as being easy to remember and easy to type.

Even though I was just testing, I was still going to have to update the DHCP reservations I have because the devices I need to be able to reach all have reserved IPs. The process is to edit the LAN IP address, but don’t apply the changes yet, edit all of the DHCP reservations, then apply the changes under both menus.

As soon as you apply those changes, the IP you are using to connect to the router will no longer be valid, so you need to refresh your IP address then log in to the router again.

And it still interferes with the VPN.

Just for the entertainment value, I tried 192.168.1.0/24 again, but this time, since the gateway for Starlink was already connected, pfSense wouldn’t let me use it.

I then went to my next alternative, the 172.16.0.0/12 range. There are a couple in use, but they are at the low end of the range. Ages ago, we had a WAN provider that had assigned the entire 172.28.0.0/14 range to our company within their network. At that time, about half of our branches were numbered as 172.29.X.0/24 or 172.30.X.0/24, where X was the branch number and 29 or 30 revealed whether that location connected via VPN over internet or a point to point T1. It worked well for us. It was fun to realize that my muscle memory could still type 172.29.0.1 really fast, so….

I renumbered my LAN again to 172.29.0.0./24 and FINALLY, all my stuff coexists with the work VPN!

Starlink

I was not fully aware of Starlink as an internet service for a while. I must have started paying attention on or about January 28, 2021, which is the date I received confirmation for signing up for emails about the service.

On February 19th, 2021, I received the email that it was available to order. The email came in around 7PM and it was 10ish before I saw it and jumped on to pay my $99 deposit. In a mere 34,387,200 seconds, it arrived.

Starlink wisely did not email reminders that their dates were sometimes slipping. You have to log in to your account to see the status of your order, which rarely changed. Mine was originally something like “mid to late 2021” then “late 2021” then “first quarter 2022”. In December 2021, it finally said “March 2022”. Since they were willing to narrow the window down to a specific month, I then had the confidence to step up my preparation efforts. The chronology on these various efforts is a bit muddled. I can specify what date I ordered some piece of gear, but for this story, what I did and tried to do are more important than exact dates or times they were tried. It will be like a movie that skips around in the timeline, but it still has a beginning, a plot and an ending. My Starlink kit arrived March 24th, 2022 and it’s working pretty well in my network right now as I write this. This post is the story of making it work.

As detailed elsewhere, I had switched my TP-Link Wi-Fi router for a pfSense router and reconfigured the TP-Link to be an access port. The pfSense router was specifically the Netgate 1100 appliance preloaded with pfSense+.

This appliance has three ports, labeled WAN, LAN and OPT. Most commonly, OPT is used for either a 2nd LAN, like a DMZ, or for a 2nd WAN connection, which was my intent.

I didn’t really start documenting the network until I had pfSense in place, so there aren’t a whole lot of differences between this diagram and the diagram for the network as built.

Even as I look at this now, I see a couple of minor typos in text, but this is a pretty good representation of what was in place around October 2021. I don’t remember for sure, but I think this diagram was inspired by the installation of the TP-Link switch in the house. Before this, it was just whatever little 8 port switch I had, maybe a Belkin.

What is not illustrated well here is the number of Wi-Fi devices in the house. Just as I am writing this, there are 25 Wi-Fi devices that I can list, plus a couple of my wife’s devices that aren’t here right now.

Following r/Starlink and other research indicates that obstructions to the dish are a big issue. Also, though I didn’t know it at the time, the dish was likely to orient itself pointing north. From the house, north is pretty much where all the trees in the back yard are, but there are no such obstructions north of the workshop.

The easiest solution I could think of to have the dish at the workshop but connected to the router in the house via VLAN. One port on the switch at one end untagged in a unique VLAN and one port on the switch on the other end untagged in the same VLAN should be all that was needed.

What follows is a completely non-exhaustive description of what a VLAN is. I could be wrong, but this is what I think I know about them.

A VLAN is a group of ports on a switch that are separated by an identifying tag in the ethernet frame. Terminology can vary between manufacturers, but most commonly, the physical port VLAN membership is “untagged”, meaning the frames going in and out of that port do not themselves have a VLAN tag. Internally, the switch tags those frames with the VLAN number. No matter what the device is or what it knows about VLANs, everything on that port is in the same VLAN. Most often, this is also the “default” VLAN, which is usually but not always VLAN 1.

More sophisticated switches will have a VLAN mode for each port, usually “access”, meaning that port provides access to it’s untagged VLAN number, and “trunk”, meaning the switch will pay attention to the tags in frames going to and from that port. A trunk port can allow all VLANs or just a list of VLANs in or out of the port.

The nice thing about a trunk port is that if you have two or more switches connected by trunk ports in both switches and you have everything configured correctly, ports in both switches configured for untagged VLAN 2, for example, will pass traffic between themselves, yet they will be isolated from the other ports on the switch. It is a simple implementation of this characteristic that I am using to backhaul my Starlink dish over the wireless link between the house and workshop, without anything else in the network seeing, or more importantly responding, to those packets.

Lesson one is that even if a switch advertises VLANs, that doesn’t necessarily mean it will do all this. It took a good bit of experimentation to determine that my TP-Link switches, even if a port is configured for tagged VLANs, that doesn’t mean it is actually a trunk port.

In order to connect Starlink to an existing network, it needs an ethernet port. The previous version of dishy, or more precisely, the previous model Starlink power brick, had an ethernet port that could be used for that. The newer rectangular dish comes with a one box power supply/router that does not have an ethernet port. To plug in, you need an ethernet adapter that is cheap enough that I wish they would have just built it in and charged me $20 more. When I completed the dishy order, I also ordered one of these adapters, but it is not expected to ship until mid April. [ed: shipped April 13]

There are all kinds of dishy hacks all over the internet, but for me, waiting for the official adapter is less painful than risking non-warranty damage to the equipment. What I have done in the mean time is use a Wi-Fi extender that also has an ethernet port. I ordered a BrosTrend AC1200 extender, configured it to extend the Starlink Wi-Fi, but I largely ignore that and use the ethernet. It supposedly runs up to 850-something Mbps in 5GHz mode. The bad part is the that ethernet port is only a 100 Mbps port, so that will cap the maximum speed available from this setup to some fraction of 100 Mbps.

When I took Starlink out of the box, it was very simple to set up and use. Knowing it was temporary wiring anyway, I opened the window into the bedroom where the equipment is, fed the dishy cord through and connected everything up. I then moved the dish as far to teh west as the cord would reach, while still keeping away from trees to the north.

It was about where the dot is.

I powered it up. There was a little interaction with the Starlink app, mostly setting the SSID and password and not much else. After some time, maybe 5 minutes, probably less, it connected well enough to run a speed test. The first one from my phone was not terribly impressive.

It would turn out that speeds on my cell phone are never as good as on a PC. I presume there is a reason that I have not seen as yet. All the tips tend to be for making sure you have a strong signal. <shrug>

At this point, I had not yet received the Wi-Fi extender, so for a little while, my only option was to connect to the Starlink router for high speed or connect to my existing equipment for anything else. It was only for a day or so.

When it did arrive, testing it was easy and setting it up was easy, too. When you power it up, it provides an admin network to connect to and a webserver for configuration. Basically, tell it what network you want it to extend, provide credentials and it’s done.

I tried for a while to connect it directly to the OPT port of the pfSense, but I couldn’t quite figure out how to configure it. I did find that I could just plug it into the WAN port and the router would just get an IP from Starlink and work. It didn’t care that it was a different subnet than OneSource was. I began to suspect, however, that the issue may have been related to teh fact that Starlink comes up with the same 192.168.1.X network that I am using on the LAN side. As much as I hated doing so, it kind of made sense to renumber my network, since Starlink doesn’t give you any control of it’s subnet.

Sadly, that didn’t do it, either. Furthermore, it would soon come to light that the new IP range, 192.168.2.X, conflicts with routes on my work laptop when I connect to the company VPN. If VPN is connected, I can get to my router, but nothing else on the LAN. The biggest effect is that I can’t get to Gnaz to view the cameras and I can’t remote to my non-work laptop to when Cisco Umbrella blocks a website I want to get to. All that means I will need to renumber again, this time avoiding all the routes in the AnyConnect. I think it the LAN and WAN subnets should be different, if for no other reason that clarity, so I won’t be going back to 192.168.1.X.

Somewhere in here, I had broken something on the router that I couldn’t recover and ended up recording all the static DHCP mappings manually and factory resetting pfSense. This time, I configured it with Starlink on WAN and OneSource on OPT and I guess I had cleared something that was keeping it from working or I had a better YouTube video to follow or something, because I got it to work and even switch to OneSource on Starlink fail. Even now, I’m not totally happy with the failback. If the gateway goes offline due to failed pings to its monitor IP and comes back on its own, it will switch back, and I’ve seen that happen a few times naturally. However, if I unplug either provider’s cable to simulate an outage, it doesn’t want to switch back to it when it comes back up. I have to manually set it to down and then back up.

With that part of the puzzle working, I could move on to local testing of the VLAN setup. At this point, the both TP-Link switches were in the same cabinet with Starlink and really everything else. I had Starlink Wi-Fi to the AC-1200, the AC-1200 plugged into port 8 of the workshop switch, a cable between port 7 on each switch to simulate the link to the workshop and the WAN port of the router connected to port 8 of the house switch.

I started with the simplest configuration I could think of and both switches configured the same. Ports 1-6 were untagged on the default VLAN 1. Port 8 was untagged VLAN 2. Port 7 was to be the trunk port, so I first tried untagged VLAN 1, tagged VLAN 2, which should have allowed 7 to carry both VLANs. In short, I tried all four combinations of tagged and untaged VLANs 1 & 2 on port 7.

The two would not pass traffic between port 8 workshop and port 8 house.

I Googled a bunch and finally found a thread where someone explained it well. Those tagged and untagged determine how the local switch treats the ports and VLANs. The packets coming out of the switch do not carry the tagging information at all, so it can’t pass it to another switch. In short, those are not trunk ports.

As I had mentioned a few times in the network blog, I have a Cisco SG200-26P. This switch has 26 ports, 12 of which can provide up to 100W of PoE and it can definitely trunk VLANs as I expected. I just needed to get it a playmate that could the same. I ended up ordering a SG250-08, an eight port switch with essentially the same set of features, except no PoE and, of course, fewer ports.

Even that was not an instant success. The two switches are different enough that the dialog for same setting may not look the same between them. In the 8 port, the port types are “access” and “trunk”, period. In the 26 port, there are “General”, “Access”, “Trunk” and “Customer”. Choosing Access grays out all the default VLAN, etc. General looked like what I probably needed for all but the trunk ports. Customer had a note beside it that made me wonder if it was some kind of auto VLAN, putting each port on a VLAN of its own, like if you were using it in a hotel or multitenant situation.

Long story short, it still didnt quite work, but it was getting close. The MAC address of the AC1200 plugged in to the 8 port began showing up in the MAC address table of the 26 port, but it still didn’t pass any traffic. More Googling lead to a recommendation that all ports be Trunk, but just control what VLANs are available for each trunk. I reconfigured all the previously “access” ports to “trunk” and made sure my tagged and untagged stuff was configured as I thought it should be and it suddenly started working! I was hitting Starlink from the house and it was traversing both switches to get to the WAN port of the router.

It was definitely too late to start moving stuff to the roof, so I would leave it like that to run overnight and through the next day.

The plan was a simple one. Move Starlink, the AC1200 and the 8 port switch to the workshop, with the dish on the roof. Though it was kind of slow going, that all worked out well.

At first I grumbled about having to get on the roof to do this, but it suddenly occurred to me that it just needs to be off the ground. There is no particular advantage in this location between the peak of the roof and where it is shown in this picture. There are no obstacles to clear with a few more feet. I ran the wire into the barn, where the eaves are screened, but not actually sealed and immediately into the loft/attic area above the workshop, which is roughly the left half of this picture. It’s dusty and dirty up there, but I got the job done. I also ran the wire that feeds one of the cameras through teh same path. It has been temporarily run for months. 🙂

Once in place, I find it interesting that the dish orients itself in a more northeasterly direction.

It was a quick thing to change out the old switch in the workshop for the Cisco and connect the AC1200 and Starlink. I waited for the satellite to come up, verified I could connect directly to it and get data then tried from the Wi-Fi out there. Connecting to Wi-Fi, but no internet. Hmmmm.. Oh, yeah, I still need to make the connections in the house. Hooked it up and everything came up!

I plugged the one hanging cord into the switchport it was supposed to go into. It didn’t come up, even after giving it about 5 minutes. I manually down/up’d the WAN interface in pfSense and it came up!

Again, it was too late in the evening to rewire everything in the cabinet. At this point, the Cisco switch had the two ports involving the backhaul and a couple of other ports in use, but the rest of the stuff was still plugged in to the TP-Link switch.

The next afternoon, I took everything out of the cabinet, cleaned it up and recabled with some color coding. It was getting old tracing black wires connected to mostly black equipment inside a black cabinet. When I finished, it looked like this:

Pretty quickly, I noticed that I was connecting to the Wi-Fi, but not getting internet. I had a link light and all the wiring appeared fine. I dug in the switch configs and found that a feature called Smartport showed a different status for the port the AP was plugged into. I moved the AP to 18 and was able to get internet via Wi-Fi on my phone. I tried the laptop and it would connect to Wi-Fi, but no internet. I looked at Smartport and now the port I moved the AP to looked like it’s previous port.

I chased the Smartport stuff in the switch and online for quite a while and got nowhere. I found that if I put the WAN connection back through the TP-Link and also put the AP on the the TP-Link that everyone worked as it should. Again, I had killed pretty much the whole evening troubleshooting this one issue.

The next day, I mentioned it to a coworker. Networking is not his specialty, but in describing the problem to him, something occurred to me. When I plugged everything in, I could get one complete valid connection, then nothing else could get internet. It tickled the back of my mind that this is an intentional behavior sometimes, to prevent users from adding rogue switches or APs to company switches. It took me a a little more Googling to remember that is called Port Security. I found it under Security, not interfaces or Smartport or VLANs. Sure enough, the ports the APs had been on were locked, configured to allow 1 MAC address. I cleared them, set them to allow 50 MACs and normalized all the wiring. The AP works perfectly now.

As it turns out, the Smartport thing could have been a clue. When port security locked a port, Smartport responded to that action; it never was Smartport doing it.

All that has been operating untouched for 24 hours, pretty much as I am typing this. For now, I’ma call it good. Here is the updated drawing.

The next thing should be another simple change when the ethernet adapter arrives. I don’t know what the effect of disabling Starlink’s router will be, other than freeing up an AC1200…

The Obligatory First Catchup Post

As is often the case, when I (or most people, I suspect) start a blog on a subject, it’s something they were already doing and suddenly decided that blogging about it might be fun, or even helpful. What follows is the story thus far….

My home networks since the inception of the common use of post-dialup internet have generally been the minimalist one or two hardwired PCs connected to whatever switch/router combo the internet provider offered. That was plenty at the time. When we moved to our current house in a largely rural neighborhood in late 2010, it coincided with both a desire to execute some home automation and the proliferation of more smart devices, more phones, more laptops, more tablets, more TVs.

The only internet providers available at the time were DSL from the ILEC CenturyLink and the usual suspects for conventional satellite internet, like HughesNet and Viasat. There may be others <shrug>. CenturyLink DSL was adequate for general internet and was also reasonably stable, though there were a few outages during the years we had it.

When I started this post, I had intended to tell the whole story of the network and home automation, because there is a fair bit of crossover between them, but now it occurs to me that one massive blog entry is going to be hard to absorb and enjoy and it’s already too long, so I am going to to split them into two related categories, Home Automation, which I started a freakin’ decade ago and I suppose this one will just be about the network in general.

What follows is roughly in chronological order, but not necessarily strictly so.

The router that came with our DSL service was a decent one, I suppose. I think the first one was a Westell and at some point, they replaced it with a ZyXel. Eventually, I wanted more control, so I bought a Belkin router to put behind it, or maybe the Belkin had a phone jack for DSL? Purchased from BestBuy, if I remember right. The Belkin had 5GHz Wi-Fi, which helped with some devices. The factory Wi-Fi password for this router was a decent one, easy enough to remember, so I’ve kept it a couple of routers later. I also eventually got a Belkin Wi-Fi extender. All the network gear is in a bedroom at the corner of the house and sometimes the signal was a bit weak in the kitchen at the opposite corner of the house, but importantly, where we spend much of our time.

The Wi-Fi extender was not enough to provide decent coverage in the workshop. The shop is roughly 100 feet from the house and it is even near the corner bedroom where the Wi-Fi is located, but the signal was still weak.

With the exception of these two BestBuy purchases, from here on out when I mention ordering something, it is pretty safe to assume I probably ordered it via Amazon. If I didn’t and its important, I’ll comment to the effect.

Anyway, based on some experiences with point to point links between buildings at work, in late 2013, I got a set of EnGenius ENH500 outdoor access bridges. This was an extremely simple and robust system that lasted nine years, through power failures, thunderstorms, ice in winter and Texas sun in summer. They were just never the problem. Since BestBuy was handy, I also got a little Belkin 8 port switch and that was plenty to support a PC and some radio gear in the shop.

Cheaping out on Wi-Fi for the workshop *was* a problem. I had a surplus Cisco WAP that seemed to be a good unit. It had 5GHz Wi-Fi, three antennas for signal diversity and a good reputation otherwise. Mine however, had a tendency to go to sleep. Rebooting it would bring it back, but it would just freeze and stop working again later. I basically dealt with it because the only thing I really needed Wi-Fi for was my phone when I was out there.

If I recall correctly, the CenturyLink DSL was spec’d out at 7.5Mbps download and 2.5Mbps upload. It never met those exactly, but for most of the time we had it, it was pretty solid for 5M down and 2M up. I had little need to work from home then and when I did, it was almost always consoling into a Linux server or some switches and routers, so it was almost all text.

Eventually, however, DSL service began to degrade. I could turn in a ticket and the tech would reset something or replace some card in the DSLAM and get it kinda back up, but after a couple more years, it became a case where dialup internet would have been only a little slower and probably more reliable. I think CenturyLink just didn’t want to invest in upgrading the infrastructure for a handful of houses in a rural neighborhood. There were no new customers to be had.

Terrestrial wireless internet began to look appealing. I don’t remember if we had more than one provider come out, but we did eventually get wireless internet from OneSource in November of 2017.

The installation was what I would classify as marginal. The LTE modem was put on the roof of the house, but at the minimum viable height, just barely shooting over an outbuilding next to the house. Even so, it worked great, 10M/2M as promised and pretty reliable and troublefree except when the growing season was in full tilt and the trees were in full foliage.

One particularly verdant season, it was affected more than usual, but coincidentally, they were upgrading equipment on the tower, which was to require changing the equipment at the house.

The tech that came out put the new modem on a tall roof mount at the peak of the roof, where it gained 8 or 10 feet of altitude. The new equipment was also good for 25M/5M, so I signed up for the upgrade.

I also befriended that tech, which is good not only because he’s a cool guy to hang out with, but it also allowed us both to benefit from my willingness to experiment and help him by testing stuff. It also helps that I’m not Joe User when it comes to this stuff and can converse with him intelligently about issues. And blacksmithing. And firearms.

Somewhere in here, a coworker who had just changed out his whole home network to Ubiquiti gave me a Cisco SG200-26P that he no longer needed. It would sit unused for a while, but would one day become very important. 🙂

OneSource has rocked along since then pretty solid. The old equipment on their tower could only go as high as 10Mbps, but the tower antenna was basically pointed south southeast and we are pretty much due southeast of the tower, so we enjoyed a strong signal, even though the antenna on our end was barely over the roof of the craft shack. The new equipment has higher capacity, but the antennas are pointed due north, south and east, putting us in a comparatively weak crossing of the south and east antennas. Our modem will connect to either. I forget which is which, but connected to one, we get the full 25Mbps, but it’s not as stable and reliable, dropping out occasionally. On the other, we get 10-15Mbps, but it’s more solid. I tend to stay on the solid side because….

Now I work from home almost exclusively. Long story short, my company moved headquarters from near downtown Fort Worth, which is a nice 35-45 minutes drive to Addison, which is a solid hour to 90 minutes in the morning and longer to get home if you don’t leave either well before 4PM or after 6PM. We had negotiated being able to work from home for a number of days, and I often used my “work from home” days to work from the old Fort Worth office on certain days because I had pistol matches in or near Fort Worth, so that was even closer than working from home.

Then came the unspecified virus of unknown origin. <cue dramatic orchestra hit>

Suddenly, working from home was pretty much the only option and 2M upload that wasn’t always 2M was just barely enough. I could join conference calls, but I was the only one without a camera on.

Oh, and then I broke my shoulder and had total shoulder joint replacement surgery, but the virus was a bigger issue.

During this time, my company was in the process of changing to RingCentral telephones and since my gig with the company is the phones, that Cisco SG200 switch mentioned above came in handy as 12 of its 26 port provide PoE and it has a 100W power budget. I could connect all the phones I needed for conducting labs and configuration testing and all the stuff one might otherwise normally do at the office.

When I first heard about Starlink (I am, among other things, a bit of a space nerd) I signed up to be notified when service was going to be in my area. Shortly, I received notice that they had a Beta program wherein one could pay a $99 deposit and end up on a list of possible beta testers in an area. I did that in February of 2021 and tracked the news, but otherwise went on with my life.

In a move to improve Wi-Fi coverage in the house, in June of 2021, I decided to upgrade the old Belkin router. It had no external antennas, so I chose a TP-Link unit with an array of them, hoping to maximize RF flux in the house. I also put the freshly liberated Belkin router out in the workshop as a simple access point and finally had reliable Wi-Fi out there.

I was pretty impressed with the features and benefits of the TP-Link router, including their mesh products. Combined with this router, I could backhaul ethernet to the workshop and have another TP-Link access point out there and they could be the same SSID! No more having to connect to the workshop Wi-Fi out there and reconnect to the house back here. So, by July, I ordered a 2nd TP-Link router of the same model.

Well, it turns out that the router product apparently can’t do the ethernet backhaul *as an access point*. It can be the controller and any of several models of mesh capable access points could backhaul to it over ethernet and use the same SSID, but not two of this model back to back. Poo. Rather than return it, I still deployed it as an access point, but now I renamed the networks HippyHollow for the house and FlyingDog for the workshop, though I still kept the password that my family already knows.

October got busy, partly because Starlink’s delivery estimate narrowed down somewhat to “Late 2021”, so I started tweaking my infrastructure to accommodate what I hoped would be delivery soon. Having studied some of the requirements for Starlink, I knew that the same trees in our back yard that caused us to install the OneSource modem on the far west end of the house would never work with Starlink’s dishy, so I formulated a plan to install the dish, once received, on the roof of the workshop because there are no trees over there. I would then use a VLAN to backhaul it over the point to point bridge to the house and plug it into the router. To that end, I ordered a couple of TP-Link switches with VLAN features. For a $20-30 switch, these are pretty nice units.

Around this time, I saw a Network Chuck video extoling the benefits of the pfSense firewall/router. It occurred to me that having a router with multiple WAN ports could allow me to, at least for a time, keep and use OneSource even after Starlink is in place and my TP-Link router had only one WAN port. So, early in October, I ordered a Netgate 1100 appliance with pfSense+ preinstalled. Sadly, it would be mid November before I could get it deployed, coincidentally while my wife was away on a cruise. That way, if I got stuck on something and killed our internet, she wouldn’t have to suffer, and thus, neither would I. 🙂

The wizard in pfSense works pretty well and getting the router up was easy. It was also easy to break the config enough that it stopped all traffic. It was then also easy to reset it to factory and set it all back up again, having learned my lesson. 🙂 I reconfigured the TP-Link router as a simple access point, like the one in the workshop and kept it in place.

It was around here that I had ordered something that required a signature on delivery. The FedEx guy kept coming to the door while I was home and apparently knocking with a feather so that not even the dogs could hear it and leaving notices that they were about to return my thing to the sender. I called in and probably completely irritated someone that day, but eventually we met up at my door and I got my stuff. The whole experience inspired me to install a doorbell camera, partly to be able to collect evidence, but also because when there is an obvious camera, people tend to act differently, in this case, hopefully try harder to complete a delivery.

Cameras are largely considered to be in the home automation segment, but most of the story of mine is pretty network centric, so I have overlapping parts of the story in both blogs.

Because I knew I would still be upload limited for a while, I wanted to avoid anything that was dependent on the cloud (you know, what we used to call servers, but are now just someone else’s servers), so Ring was right out. I dug around and found that Amcrest had decent products and while <buzzword> cloud </buzzword> (and don’t even get me started on the people who now call conventional servers the “private cloud”) was an option, it was not required; they could record to a local SD card and they could be administered locally. I ordered an Amcrest AD110 doorbell camera and two other cameras suitable for watching our driveway and watching the horses in the barn.

They were pretty easy to connect. The barn camera would have to wait a while, but the doorbell connects to the existing doorbell wiring for power and has some kind of terminator adapter thingy that connects across the existing chime in order to operate it. The camera appeared to come up mostly, but would never successfully connect to the network. A bit of Googling revealed that it requires at least a 16V doorbell transformer in order for Wi-Fi to operate correctly and sure enough, mine was only 10V. Quick trip to Lowes and a quick trip to the attic to swap that out and it came up beautifully. Literally a couple of days later, I am working at my desk and the doorbell motion detector goes of. It turns out to be that the horses got out!

If I am being completely honest, the AD110 does require the largely cloud-based app for configuration of the camera, but it pretty seemlessly integrates, so it’s hard to tell what might be local and what might be cloud. I should test for that.

The driveway camera was more involved because I needed to run an ethernet cable to where I wanted to place it. Happily, most of our attic is pretty open and the wire was easy to run. The camera runs on PoE, so again glad to have that Cisco SG200 switch.

I didn’t have PoE in the workshop/barn, so I had to order a suitable injector. Again the camera came up easily and I have grown to appreciate Amcrest products.

I found that the 32G SD cards that I had readily available for the cameras’ local recording would fill up pretty fast, so I ordered some 256G cards from Amazon. They arrived and I’m not sure what the issue was. They were SanDisk class 10 devices, but they would randomly unmount and recording would stop, sometimes requiring a reboot to restore functionality. [ed: now I know that the write speed of SD cards is important with video] I put the 32G cards back in, but at even pretty modest settings, they would fill and auto overwrite in just a few days, particularly as each alarm event is recorded at a higher rate and I was still getting a lot of video motion detection alarms as I tweaking all that.

I had noticed that one of the recording options was to send to a Network Attached Storage device. I thought that might be a good solution, plus we didn’t really have any appreciably valid data backup strategies for our laptops. I did some shopping and YouTubing and decided that a single drive Synology NAS would probably suit me well, so I ordered a DS120j with a 2TB disk.

Almost immediately upon getting it set up and learning about its features, I found that Synology has a Network Video Recorder app for their NAS called Surveillance Station. That turns out to be way better than just using the NAS to store the camera’s recording files. It is a fully featured NVR and video is stored locally.

However, nothing is perfect. I discovered that Surveillance Station is license bound. The NAS comes with licenses for two cameras. When you have two cameras and go through the procedure to add a third, you are given an option to continue after selecting which of your existing cameras to disable or to cancel. The licenses are not particularly cheap, either. However, I decided that I liked the advantages enough to pay it, and it was still cheaper than a dedicated NVR. So, thinking about where I wanted to put cameras, I ordered a 4-pack of licenses for a little less than $200.

Then is when I learned about the other imperfection. The essentially minimal hardware NAS I had would support a maximum of five cameras. The system resources are the limitation and Synology, wisely I suppose, will not let you over subscribe the hardware. So now I had five usable licenses and one unusable $50 license. Sigh. The next higher rated hardware is a two drive unit that can support 12 cameras and the next one after that does 25. If I feel the need enough, I can migrate my existing disk to it

I know that I can migrate a disk because I have done it once. By then, I had four cameras connected and I was also taking up some file space with several hundred uncompressed photos for astrophotography.

I had fears that I would outgrow my 2TB disk. By December, I ordered a 6TB replacement disk and a 2TB USB 3 external SSD and performed a complete backup using one of the Synology provided tools. It was a very slow process; I left it overnight because I got tired of waiting for it to finish. [ed: yes, the write speed of the 2TB USB drive was a big factor in that slow process] Synology treated the new 6TB disk like it was a new out-of-the-box NAS. Once it completed all its formatting gyrations, I plugged in my backup drive and started the restore process. It took another several hours, but worked flawlessly. The only loss was the footage that would have been live between in end of the backup and the end of the restoral.

It suddenly occurs to me that it was probably slow on backup because Surveillance Station was hitting the drive with 4 new live streams the whole time it was trying to back up the disk. Perhaps I will remember that and take it offline first next time.

Network Chuck strikes again. In my hope to further avoid cloud services for things in my house, I think the benefits of reverse proxy are attractive. In this particular instance, Network Chuck deploys it in the form of a free Kemp load balancer.

I should have made notes setting this up because getting the hardware in place to run a Proxmox hypervisor upon which the Kemp VM can be deployed had it’s own story. I had a fanless PC that I once tried to use as a Windows jump box for ham radio applications out in the workshop. I never could align all the sleep features such that it would not turn off after a couple of hours. Proxmox of course has none of those kinds of features. I think I upgraded RAM but I definitely added a 120GB SSD. BTW, I did try to load the free ESXi hypervisor, but there was something it didn’t like about the machine, but Proxmox loaded and ran just fine.

I did not finish getting the Kemp stuff up and running. Rather than strictly follow the examples in the video where Chuck shows us how to set up a domain name with FreeNom, I wanted to use an underutilized name I already have with a different registrar. I had some trouble getting it to do what was expected and I have not yet finished the process. I will revisit at some point. [ed: plus, Starlink’s use of CGNAT pretty much breaks this particular application of reverse proxy, at least not without outside help.]

More cameras: The driveway camera can *just* see the front gate and I wanted to put a camera out there by the gate for better coverage. My phone has a usable signal from the house Wi-Fi , but being 170ft from the house means power could be an issue.

I ordered a camera that sounded promising, an outdoor Wi-Fi connected camera with a USB power cord and a 12V to USB adapter dongle, the intent being to connect it to the solar charged battery that is already out there for the gate opener. Unfortunately, it would be late March before I got around to setting it up. In the mean time, it also occurred to me that the gate draws nearly nothing unless the operators are actually moving the gates, but the camera is going to draw something 24/7, though due to the nature of the USB connector, I could feel confident it should be 10W or less. As the gate is a kind of security and safety thing for which it would suck to have a dead battery, I decided I would just add a separate battery, solar panel and charge controller for the camera.

The panel is a 25W panel, I’m sure bigger than is needed. I also found a charge controller that has USB outputs, so I don’t even need the power dongle anymore. I’ll repurpose those to our kayaks or something. I already had a couple of battery boxes and elected to rob a lawn tractor battery from a mower and replace it later. [ed: as time has passed, this has been a reasonably reliable setup, but several days of cloudy conditions is enough for the battery to run down enough to lose power to the camera. I am not sure if I need to address that with more PV panel, more battery or both.]

I was not completely surprised to find that the Wi-Fi signal from the gate to the house is ok, but not really solid. My intent was to deploy the old Belkin Wi-Fi extender from years past. I have seen it recently, but I could not find it. I presume it is in a box that I didn’t open. Instead of continuing the search, I ordered a BrosTrend AC1200 to extend the house WiFi into the garage, which is closer to the gate. Another AC1200 will figure prominently into the Starlink story.

The EnGenius WDS bridge between the house and workshop/barn had been in solid service for nearly 10 years when something started getting flaky on one or the other. The received signal on the workshop end dropped significantly, -100 dBm, down from -30 or so. It’s unclear whether that one had a receive problem or the other end had a transmit problem; they would look the same if you just look at signal strength. The unit on the workshop end is fairly near a light that is on all the time. During the spring and summer, there is an absolute bounty of bugs, spiders and geckos feasting on one another, so I thought perhaps some bugs had gotten inside the unit, as I’m sure anyone might suspect given the conditions.

Unfortunately for the story, the unit is well made and the electronics are very well protected from the ingress of dust, let alone critters. It was very clean inside.

Since my plans for Starlink would depend on a reliable link between the house and workshop, I started shopping. My original plan was to stick with what works, another pair of ENH500s, or since it’s been 10 years, whatever may have replaced them in EnGenius’ lineup. They do have a newer model, but they were about $200 a pair, which is not super expensive, but I kept shopping anyway. There were several units that were sold singly for less than $100 a pair but I had just a little instinctive reservation about them; hard to explain. However, I also found that Ubiquiti has units that were physically similar, identical perhaps, to the suspect units. Ubiquiti has a good reputation, so I went for a pair of AirMAX LiteBeam ACs, $140 delivered. They arrived quickly, were easy to mount and easy to configure. With default settings, they were good for 300-ish Mbps, which should be plenty, but I tweaked with settings and got quite a bit more potential data rate out of them.

I was in California for a work telephone project when I got the email: Starlink was ready to ship! [ed: ironically, I was in Hawthorne that day, where Starlink would ship from] I quickly confirmed my order, forked over the balance and also ordered the ethernet adapter that is required to plug it in. The older version had an ethernet jack, but for the new one, it’s an add-on accessory. There is much grumbling about that on r/Starlink.

Since the whole Starlink installation is a week long experience of its own, and as I am writing this, it I have *just* completed it, I’ll continue that story in a post of its own.

A Novel Experience

I have told this story before. I first became a Novice (KA5RZY) ham in the early 1980’s. I upgraded to Technician (N5HRK) in the mid 80’s. I never got over the 13 word per minute hump, so never upgraded to General. Many years passed and I did Other Things ™ and my license lapsed so hard that even the 10 year renewal grace period was years out of date when I got interested in the hobby again around 2011 or so. I had to study and test again to get my license anew. It is a no-code license and between the study tools and my own experiences, it was possible for me to take the tests for Technician, General and Extra in one sitting. I kinda squeaked by Extra (AE5XB), but I did pass. I didn’t like my new Extra call all that much, though it did have a X in it. I searched for a vanity call. For all the searching, the best vanity call for me was my old Technician call, so here I am, N5HRK again.

Also, 10 years have passed and it is time to renew my license and for the first time ever, I did it.

One Orbit Closer

I received my Arrow antenna today. It is trivial to assemble and seems really well designed and built.

Knowing I would be using two radios to achieve duplex operation, I ordered it without a duplexer. However, I failed to order any BNC cabling with it and I don’t have any BNC connectors to make the necessary cabling, so, I packed it down into it’s handy portable bag, ordered what I will need and began the next stage of waiting.

And Sputnik Was His Name-O

I received my MFJ-1866 discone antenna. Assembly was trivial, basically screw all the elements into the right holes, short ones on top, long ones on bottom. It looks as much like Goddard’s early work as an antenna.

I mused that it looked like Sputnik and my wife has already called it Sputnik (as in, “Are you going to move Sputnik out of the bedroom?”

Sputnik it is.

I did screw up my order, though. I had toyed back and forth with what coax I was ordering and ended up with the unintended product in my cart in the end. I ordered a 50′ jumper of 400MAX, which is awesome cable, but it’s 1/2″ diameter stuff and I had intended to get much thinner RG-8X as there will be a 6-8 foot bit of it showing… INSIDE THE HOUSE! It’s not like I’ll never have a use for the 400MAX, so I’m conflicted on whether to jump through the returns hoops, but delays delays delays. I *may* have some RG-58 laying about already, leftovers from a trapped dipole antenna project I started at the old house. There is a box in the storage container that says “ANT TRAPS” that has been misinterpreted by more than one person.

I did connect the discone with a short cable to the SDRdx. Even sitting on the floor in the spare bedroom, it improved reception over the Comet SMA ducky that was used before and I was able to pick up a few 70cm repeaters. Receiving on 2M was a problem, though.

SDRuno

Note the terrible birdies. The receiver and antenna are just a few cubits from my LAN gear, most notably a Cisco PoE switch powering a few Polycom phones. As that is my day job and I work from home, they must remain. However, my personal laptop kinda lives in that room for now and it was the easiest (read: least nanny state) to install the software on. All this will get better when it is moved away from that cluster.

Fun fact, the screen capture above is from my phone, running the Android version of Microsoft’s Remote Desktop, connected to the laptop running SDRuno. Mouse operations are a little weird, but it otherwise works just fine.

Long term, I’m thinking the discone may go into the attic, with the *thin* coax coming down into the kitchen; we have a DirecTV coax coming down in an unobtrusive corner. Then I can kinda shoehorn a little VHF/UHF/SWL station in that corner where we have a PC or two, using the SDR and my FTM-100DR with an antenna switch. RDP lets me work it from largely anywhere else. I can also play with putting the FTM-100 on WIRES-X at that time. The trouble with evil plans is that they just keep comin’.

My Arrow antenna has not shipped yet. The order confirmation said “Your order is estimated to ship within five business days. Orders usually ship within two business days.” Today is the evening of 7th business day, not even counting the day I ordred, so it’s late, but probably not late enough for me to reasonably complain. :/

Sat On A Light

There have been amateur radio satellites in orbit for literally my entire life. Hams were in space only four years behind the Russians, less than a year before I was born. It also was the first satellite deployed as a secondary payload from a primary launch… ejected using a spring purchased from Sears. Remember, it doesn’t *have* to be expensive to work, but if *my* life is in the balance, I’d rather it was. It should at least be a McMaster-Carr spring…

The first time I heard of amateur radio satellites, I presumed that was an exotic part of the hobby that only the ham elite (read: rich hams) would ever have access to. I think a big part of that was the math needed to predict orbits and the presumption that exceptional precision would be needed. Turns out that it’s not particularly difficult, but neither is it trivial. Johannes Kepler figured it out on paper by candlelight. With the advent of personal computers in general and the ubiquitous smart phone in particular, much of that number crunching is a few finger swipes away. Now working satellites seems to be reduced to learning the idiosyncrasies introduced by chasing a slowly tumbling 1/2 watt transceiver 300 miles away while it flies by at 17,000 miles per hour. Piece o’ cake.

I exaggerate the difficulty, though not the conditions. As rough as that sounds, it can be done with as little as a handheld antenna and a dual band HT, assisted by some tracking software.

Armed with some basic tips obtained from various YouTube sources (which I will detail below) and an Android app, I was able to pick up the beacon on a couple of satellite passes with only my existing base station gear, an Icom IC-7100 and a Diamond X-300A. This is a fixed base station antenna optimized for transmitting and receiving towards the horizon.

Undocumented Satellite Beacon Pass

Note that I am chasing the Doppler frequency shift, which is one of the two biggest idiosyncrasies. Satellites are moving fast enough for Doppler frequency shift to matter. Most satellites operate crossband, with an uplink that is either 2M or 70cm and a downlink that is the other band. Obviously both are affected by Doppler shift, but as it is a percentage effect, the higher the frequency, the larger the effect, so 70cm is more affected that 2M.

Having now heard such a birdie tone intentionally, I am reasonably sure that I have accidentally heard them before and perhaps just didn’t realize it was a satellite! How simple must it be if I can do it accidentally?

I have tried a couple times since then to hear QSOs and I can *just* tell there *might* be something there, maybe the tiniest bit of a beat tone tuning the range, but not enough to pull out of the noise.

To give myself the best chance of success, I ordered an Arrow 146/437-10WBP antenna. This is a 3 element 2M and a 7 element 70cm on a shared boom. The BP suffix is the split boom (back pack) version which breaks down easily for transport, so I also got the carrying bag for it. At this writing, it has not yet arrived. Being a handheld antenna, it helps with the other major idiosyncrasy, polarization fading. As the satellite tumbles in microgravity, the antennae are changing orientation and being able to rapidly change polarization is one strategy for coping with that. Antennas built with circular polarization is another.

My plan to start is to try FM using two handhelds. I have 2 each Baofeng UV-5Rs, RD-5Rs and an Icom ID-51, so I can mix and match as necessary.

I also used this as an excuse to picked up an SDRPlay receiver, the SDRdx model. I have nearly purchased an SDRPlay receiver a few times over the last few years, but never pulled the trigger. I’m not sure if it will actually be used for satellites or not, but I imagine I will at least experiment with it. Plus, everything else you can do with an SDR 🙂 Not necessarily only for satellite use, but for very wideband SDR reception, I got an MFJ-1866 discone antenna. This will be my first experience with a discone. This model can do double duty as a 200W transmit antenna for 2M to 23cm as well. It might become the first antenna mounted on the house, as opposed to the workshop.

I’ma Call It Finished

I decided that if I waited until it was perfect, it would never be declared as “finished”.

I managed to put some deep scratches in there that were proving difficult to remove.

This picture was taken with the light at an angle to highlight the scratches, so in the hand, it doesn’t look quite as bad 🙂

The handle is mesquite, sanded to 600 and finished with Danish oil. It has a very nice feel.

Next, I tear into the other blank, though I may fabricate a bevel grinding jig of some sort first.

Sluggy Blogs All The Subjects