Ok, maybe that’s a bit much.

I had a couple of really busy months adding and changing stuff in Home Assistant. A lot of it was adding something, breaking something else, fixing that, adding something new, breaking something else, ad nauseum.

However, at some point, intentionally or not, I ended up kinda leaving it alone for a while, mostly at or after Christmas. I tweaked some dashboards and adjusted a timeout here or there, but really I just didn’t do a lot with it for a while.

My wife had knee surgery. To keep the herd of buffalo that we call Dachshunds from trampling her knee and so that her completely unnatural sleep “schedule” would have the smallest impact possible, she set up shop in the spare bedroom. I put a Tasmota bulb in the lamp and moved one of the Amazon Echo devices in there so that she could control said lamp without necessarily having to get up. The Echo is also handy as an intercom when I am working elsewhere in the house and she needs something.

Trouble is, Alexa would work fine controlling the lamp (or any devices) for a couple of days, then she’d claim there was no such device. Or she would ding like she had turned it on or off, but it didn’t do anything. That would go on for a day or so, then she’d be back to controlling devices again.

I made sure my wife had the Home Assistant app on her phone and a dashboard that had all the general stuff she cares about controlling so whenever Alexa got stubborn, at least she could still control the lamp and the thermostat.

Importantly, Home Assistant always controlled the devices. Alexa, not so much. But Alexa always worked for stuff that didn’t involve Home Assistant. Clue numero uno.

I Googled every combination of ‘home assistant alexa unreliable’ and similar keywords and it seemed like all hits were things about troubleshooting an initial connection between Alexa and Home Assistant. Nothing about Alexa’s moodiness.

During the time I wasn’t doing a lot of work on Home Assistant, I didn’t have cause to connect remotely, via the Nabu Casa cloud, but a couple of days ago, I did. I immediately noticed that a bunch of stuff on my dashboard was grayed out, like the devices were offline. I had notifications and there were 9 (!) updates pending, including an OS update. An OS update was very recently done and I wondered what went so wrong that they had to update the OS again, although that kinda made senseas to why there would also be a bunch of other updates, responding to that one.

I decided that I was definitely not going to do all those updates remotely and I would just wait until I got home to start working on it.

I got home, logged in. All devices look fine, no updates pending.

WTFO?

I opened a new browser tab and logged in via Nabu Casa and I had missing devices and pending updates. I checked the logs from both logins and they both had current entries, but not the same entries. Clue Numero Dos. Then I noticed that a change I had made to my dashboard only the night before was not there on the dashboard via remote. Clue Numero Tres!

I looked for a while for some feature or button that would synchronize the two, but no luck. I cried ‘uncle’ and submitted a ticket to Nabu Casa about it.

Meanwhile, I was chatting with a friend who is also running Home Assistant, the friend who had in fact got me started down this road. He suggested that maybe I had two Docker containers running. Well, he didn’t remember that I had abandoned running Home Assistant as a Docker container originally because there had been so many weird hoops to jump through just to add HACS and secondly because running it instead as a VM meant that I can recover from egregious errors by restoring a snapshot backup. 🙂

In any case, he is not an idiot and neither am I, so I followed his advice and verified that, no, I had no rogue VMs or Docker instances running.

About that time, I got a note back from Nabu Casa. The tech also suggested that, especially from the clue with the logs having entries that were in the same time frame but different, that there must be another instance of Home Assistant running. He detailed some sometimes people spin up a VM or a Docker to try it out, then install to hardware and forget about the Docker and leave it running. He suggested checking the Network menu on both to see what the IP addresses were.

I didn’t have to do that because as soon as I saw the word ‘hardware’ in his note, it reminded me.

Early in December, before Christmas, before my wife’s knee surgery, one of the things I was looking into was moving Home Assistant off the VM and onto a fanless PC I have. The idea was to escape a perceived issue with the USB dongles for Zwave and Zigbee, having recently spent an embarassingly long time getting them working again after what should have been, at most, a mild interruption. In any case, I was able to successfully bring up Home Assistant on this hardware device and, after a fashion, restore a current backup to it. I verified that all my configurations, dashboards, etc. had all moved over. I just didn’t want to commit to moving the USB devices for fear that there would be some issue with the PC then for some unknown reason, I would not be able to bring them back up on the VM. Besides, the snapshot backups are a pretty good reason to stay on the VM for now. So, I shut down the PC and didn’t think anything of it.

Well, spurred by the tech’s phrasing, I verified that the fanless PC was up and that was indeed the instance of Home Assistant that Nabu Casa we connecting to!

I looked into the logs deeply, particularly for anything to do with rebooting and sure enough, there it was. December 13, we lost powere at the house for several hours. The NAS and Home Assistant hosted there showed the reboot on the 13th, as did the Home Assistant running on the fanless PC. The little PC did what I’m sure it was configure to do, boot upon restoral of power. Since it probably booted up faster than the NAS and VM, it was first in line to authenticate to Nabu Casa.

Both systems were probably syncing with Alexa, so whichever had sync’d last had her attention. When she couldn’t find a device by a name I know was there was probably when she was connected to the PC and the names on it hadn’t been changed. Since I never logged into that one, none of those nine updates had been applied yet. The dashboards on *that* system had not been editted.

It’s unplugged now.

And Alexa is going on four days of continuous cooperation.

Update: 9 days and Alexa has been reliable. For this, anyway.

After I finally beat the dongles into submission, it took a little while to discover that many of my devices did not seem to be operating, but in reality, they were just not responding to verbal commands via Alexa.

Complicating things is that I was also dealing with some WiFi trauma while settling in the new Ubiquiti gear. I was suffering a lot of disconnections and I could not be 100% sure that the WiFi devices that I was trying to control may not have just been switching on and off like a crazy monkey. Short version on that seems to have been overzealous roaming defaults with only two fairly distant APs, but that is a story for that blog.

Once the WiFi was settled down, if not solved, it was obvious that Alexa could definitely see all the devices. I could change a device name and it would be detected, but I still could not control any devices with verbal commands. There were a couple of responses but by far, the most common was that there was no <device> in my profile. Sometimes, I would get that the device was not responding. What I would never get is a device that was controlled.

The timing for this failure is abysmal. My wife just had knee surgery and in preparation for that, I made sure the lamp in the bedroom where she would be staying (she must avoid the dogs for a while) had a smart bulb in it that she could control verbally, except that verbal control wasn’t happening.

After trying no shortage of useless stuff and random restarts of various services, I was getting to be on the right track and had tried removing the skill from Alexa. In Googling about that, I stumbled across a forum posting where someone suggested removing one file and restarting home assistant. I chose instead to rename that one file, just in case. Well, praise Jack Bauer because it worked. All the Alexa stuff works again!

Well, I had to re-re-rename some things back to normal, basically clean up the messes I made troubleshooting. I’ve had to do that a lot lately.

The file in question is /config/.storage/cloud. My old one was 3 times the size of the new one that the system created automatically to replace the missing one. It had several blocks that seemed to be associated with Google devices (of which I have zero, so far as I know) but generally, it was just a bigger file with more stuff in it. I’m sure there was redundancy or perhaps one scrambled line in there.

Now that I can tell Alexa to turn on the Christmas tree and she does, of freakin’ COURSE the rotating base for the tree would stop passing power through to the lights. It is always something.

With apologies to Ian Anderson.

Full disclosure. I am still not entirely sure exactly which of the myriad steps finally worked or what order of steps I finally accidentally hit, but the Zigbee and Zwave dongles just came up working again. Furthermore, it’s actually been a few days since that happened, so my memory of it is imperfect as well.

The all critical final step, however, is emblazened in my retinas forever. After doing this SEVERAL times already, removing and adding back the USB devices at the virtual machine level in the Synology NAS, with at least some number of minutes/hours/decades delay seems to let something reset enough that the device is not only detected (which it has been all along) but also works.

Of course, the damage I had done troubleshooting needed to be undone. Some devices had lost their names and almost worked. They had reverted to the default names given when the devices are first interviewed, generally some variation of their brand and model number.

Most of these were pretty easy to figure out, either by being relatively unique (at that point, I had deployed only one Third Reality motion sensor) or by what automation was still associated with the device. Curiously, those automations didn’t work until the name of the device was adjusted. For the afore mentioned motion detector, I had set up a fairly sophisticated timer automation as suggested by Smart Home Junkie and parts of the automation needed to adjusted with new device names as well.

I have all but one or two of those names resolved now, mostly because I just haven’t walked to the doors or plugs in question and verified which was which. Soon.

This power event did inspire me to look into moving Home Assistant over to dedicated hardware. Whether that is really justified depends on whether the advantages of dedicated hardware outweigh the advantages of hosting it as a VM and that really comes down to how it handles USB hardware under unusual conditions like an unplanned power loss.

Raspberry Pi hardware is still very hard to obtain, but I have an Celeron N4100 based fanless PC that is more than adequate to the task. I had a false start (not all the installation methods install identical versions), but did manage to get a suitable Home Assistant image on it and was able to restore a backup of my current configuration. For personal reasons beyond merely being the week of Christmas, I don’t want to tear into the actual move of all these devices to that hardware just this moment. After the holidays, I will revisit it.

On the heels of solving one problem comes another.

We had a scary storm nearby a few days ago.

This is just a few miles north of us and it is just a tiny bit of the storm damage. It took power out for our neighborhood, though only for about two hours. That was long enough for the little CyberPower UPS feeding my network stuff to run down. It was also the first time I have had something hooked up back there that was apparently sensitive to losing power, namely my Home Assistant VM hosted on a Synology NAS.

Before I get into the rest of this story, I’ll say that my UPS does have a USB port, but it is not specifically listed on Synology’s compatibility page. Still, I will secure the proper cable and give it a try. Doing an orderly shutdown of the NAS might have avoided this whole thing. Or not; I suppose that depends on what it takes to finally fix it.

After we got power back, the Home Assistant VM would not finish booting up.

Note that it had been waiting for nearly 6 minutes for whatever it was holding up for. Deep digging seemed to indicate that a database file for something may have gotten corrupted, meta.db. About the time I was figuring this out, I also remembered that I have the Synology doing nightly snapshot backups of this VM!

Though just this moment I don’t remember why, I elected to restore the snapshot from Dec 11. I have also since then locked that snapshot so that it won’t be rolled out. I may need it again.

That process at least got Home Assistant booted up and usable, though it took me a while realize that the Zigbee controller was not up. Zwave was up and working, which was a relief after the previous debacle involving it. I needn’t have worried, though. The attempts to restore Zigbee operation would soon kill ZWave, too.

The details of both seem to indicate that the software can’t find the hardware. Almost surprisingly, however, the hardware is there.

The kernel sees them. So, why can’t Home Assistant?

My first concern is Zigbee because I have more devices on Zigbee and they include the lights in the back yard, used frequently for letting the doggies out. The oldest of these doggies has enough trouble finding the door in the daytime.

For experimental purposes, I also tried another snapshot option wherein you can configure a new VM from one of these snapshot backups. Interestingly, it generates a new bogus MAC address, so DHCP gives the new instance a new IP address. That was slightly unhandy, but easy to fix. The oldest December snapshot, from the 4th, behaved exactly the same, which leads me to believe that this is probably not a filesystem issue, so I have put back the December 11 snapshot and have been working with it.

I tried something kind of radical and deleted the Zigbee Home Automation (ZHA) integration. Home Assistant discovers the dongle like a newly installed piece of hardware:

Unfortunately, it returns an unknown error upon attempting to configure it.

If I instead choose to Add Integration and search for Zigbee, I get these options:

The first option is the same as trying to configure the new hardware. The second option complains that Zigbee integration is not yet set up, but gives an option to proceed to set it up:

It proceeded to look very normal, like it was going to work:

Then, it ground to a halt.

There is something new here, though. It looks like it has the same port in there twice, neither working.

If I click on either of those, I get the same ‘failed to set up/check logs’ message as always, so insanity.

To break the chain, I have ordered Sonoff’s *other* Zigbee device, based on the Silicon Labs chipset. It would seem they had cause to add to the Texas Instruments architecture, whether it was to help assure a diverse supply chain or in pursuit (or retreat) of some features. I hope to know soon.

A few unrelated projects has postponed tackling the plumbing part of putting the other two water meters in service. Meanwhile, I have been tweaking the behaviors and such for a couple other bits.

The heatlamp for the water system has turned itself on every day. It was indeed chilly for several days, so it wasn’t inconceivable that maybe it reached my low temperature trigger of 36F in the garage, but a couple of days over 50 that still had the light on every morning made me start actually checking.

I used one of the Emporia switches (cloud controlled; none of my Zigbee switches monitor power and that is a whole ‘nother conversation) so that I could monitor the power consumption of the outlet and thus detect a burned out bulb. The log showed some curious stuff though. This thing loses contact all the time.

If you look carefully, you can see the pattern. It becomes unavailable, then exactly one minute later, it comes back, appearing to change state, but I think it just renews it’s current state. That goes on as far back as you care to scroll.

However, at 6:00, something turns the switch on, then this same pattern just keeps repeating with the new status.

I checked the automation on the water temperature sensor and the outside temperature sensor, even though I had not automation for *this* switch for outside temp.

Finally, I found it. In the Emporia app on my phone, there was a schedule to turn the switch on at 6AM every day. I had originally deployed this plug out in the workshop for the horse trough and I wanted to ensure it was not accidentally turned off and left off. I disabled the scheduled.

The light was on the next morning.

I deleted the schedule and it finally stopped. 🙂

In related news, I have two CloudFree P2 plugs. These are plugs preconfigured with Tasmota and it turns out they have very extensive power monitoring capabilities and no cloud dependency. I have removed the other Emporia plug that (for reasons I don’t recall) was also deployed for the horse trough and put a CloudFree P2 in it’s place.

As you can see, it gathers a lot of power statistics. Currently, there is a recirculating pump that is running basically all the time. It is important, but not particularly critical. However, I am about to redeploy the deicing heater and if it goes out, particularly during freezing conditions, I need to know about it. The automation I have in place currently will notify me if it draws less that 10 watts for more than 10 minutes which would currently only tell me if something disconnected the pump, which one of the horses used to do on occasion. The pump draws 108W pretty much continuously. The heater pulls 1200-ish watts when heating and about about 1/2W when idle, presumably to work it’s thermostat. I presume I could script something to separate a pump failure from a heater failure, even though they run on the same outlet. I have more than once thought about separating them. At the very least, I could run them on two switch plugs.

The bad news is that I only ordered two of the CloudFree P2 units and one of them turned out to be DOA. I have contacted CloudFree about it (no reply, but it has not even been 24 hours yet) but I also ordered several more. I will have other things to control and/or monitor and especially at $10 apiece (Black Friday 2022 pricing apparently; back to $13 as I write this, but still a great price), they are a great little bit of kit.

I had some difficulties excluding and including my old Jasco ZWave switches to my new controller, but all appears forgiven now. I pulled the air gap power plug thingy on them and that seemed to properly reset them enough for them to be happy.

I dug into the wiring for a couple of 3 way switches in the kitchen. They are wired in the 2nd least favorite way for conversion to the Jasco switches I have had on hand for several years, but with some minor rewiring, I can make it work. I will post that separately one day. The one I really want to change out is a set of lights in our sunroom that is tecnically a 4 way, with one switch in the living room that is almost never touched, but is sure to complicate things, wiring-wise. I really want to deploy my Zooz scene controller where the kitchen located switch for that sunroom light is and use it to control several things.

The barn/workshop automation has to be all Zigbee or Wifi. I don’t think I can reasonably extend the ZWave network out there. That isn’t expected to be a problem; I have good Zigbee and excellent WiFI out there. I have a couple of Zigbee light switches, but I’m not sure how they handle 3 way connections as yet. I may need to deploy some wireless toggle switches for those remotes.

I put a new battery in the Ecolink tilting garage door sensor that I have had since Vera was new, along with putting a ZWave repeater in what I hope was a suitable place to extend the network into the garage, but the switch was last heard from on 16 days ago as I write this. I may try again now that the two light switches are working and helping to provide a little more network stability.

Before I discovered ESPHome, I had gotten a Zooz Multirelay with the idea of using it for the water filter project. As it turns out, a couple of it’s features are not particularly well suited to the needs of the water filter, but they are entirely suitable for automating garage door openers.

Early in my Home Assistant journey, I had planned to connect something to and/or control our self-cleaning whole house water filter.

I had discovered the ESP8266 and it’s pulse_meter sensor and then planned to have a board to connect the sensors for the water meter and later the unfiltered water meter and a separate board for the water heater meter since it is a short distance away, but it is really is a very short distance.

After working with the ESP8266 for a bit, I have decided that there is no reason these functions cannot all be on the same board, thus the Home Assistant device “watermeter” has been repurposed and one of it’s identical syblings “watersystem” has be configured.

It may not win any prizes for awesome miniaturization, but it is packed in pretty tight. On the left is 5VDC power.

On the right, the pair of three terminal connectors are for the unfiltered and hot water flow meters. The voltage divider resistor networks to adapt the meters’ 5 volt output to 3.3 volts are on the bottom of the board. I have not yet connected them, but my confidence that they will operate is reasonably high.

Immediately below those is a Dallas TO92 DS18B20 temperature sensor. There is a 1/4″ hole drilled in the lid of the case to allow it to stick out a bit and measure the ambient temperature. Since all these water system components are installed in an unheated garage and it has gotten below freezing in there before, I need to monitor the ambient temperature and control a heat lamp during winter months.

Next is the pulse input for the main water meter, shown connected in this picture.

Finally, there is a connector intended to connect to the water filter to monitor when it’s cycle runs.

I have covered the details of the configuration elsewhere except for the binary_sensor.

binary_sensor:
  - platform: gpio
    pin:
      number: 15
      # D8 Filter Cycle Sense Switch
      mode:
        input: true
        pullup: true
    name: "Filter Cycle"
    filters:
      - delayed_on: 100ms
      - delayed_off: 100ms

This is the input to monitor the water filter cycle. Since the original water meter, I have learned that the ESP8266 has built in pullup resistors that can be activated, eliminating the need to add a physical resistor. In this application, I may not end up needing it at all. The filter settings are currently there assuming I would be debouncing a switch input.

The water filter has a motor that runs a multiposition valve with a cam and a single sense switch. Investigations indicate that the switch is normally open, with 5V from it’s own controller across it’s terminals when it is at rest. When the filter cycle begins, the switch closes as the valve cam moves. The filter cycle puts the valve into three positions, a flush and a rinse position during the cycle and a normal position when done. I think I can monitor the 5V signal across this switch then either simply time when it has returned to 5V for long enough for the filter cycle to have ended or possibly even count the two stops in the process. I will need to play around with that.

The one thing that might be added is to connect one more GPIO pin to a relay output in order to trigger a filter cycle remotely. There is plenty of room for a couple of wires to run to an external relay. The area in front of of the USB jack needs to remain open in case it is needed to reprogram the board if it becomes unreachable on WiFi.

Funny story there. In arranging the ESP8266 D1 Mini board on the 40×60 perf board, I dithered on whether to mount the USB jack up or down. I decided to mount the board with the jack down, mostly because all the pinout illustrations of the D1 Mini board are shown that way. However, I did not account for how thick the connector is!

I was able to slightly mutilate the connector on a cable enough to fit, though. For future boards, I will need to put a little space under the board.

The ESP8266 boards have eight GPIO pins, though some of them are kinda hijacked early on, they should be available for more general use after the board boots up.

Leaving the bare board hanging by wires is just begging for trouble, so I want to put them in some kind of enclosure. The first ones I ordered turned out to be just a tiny bit too narrow.

Chalk that up to poor tracking of dimensions while shopping. I will keep the little boxes because they may be usable for something else at some point. That and at roughly $0.87 each, even a dozen of them are too cheap to be worth the trouble of return shipping. I think it’s planned that way.

I was more careful with the next case. I first found a small prototype circuit board of 40 x 60 mm, big enough to put the ESP32 board on with some margin around it, then found a case with internal dimensions to fit that board. If I also provide 5V power in some other form besides microUSB, I can mount the ESP8266 in the most space efficient manner on the board and inside the case. This will leave the maxium room for additional components or connections.

I think I will first rebuild the water meter device, adding an on-board temperature sensor and the sensor input and relay output that I had originally intended to put on the big water filter to let us manually trigger a filter cleaning cycle remotely. What would really be cool is if I can find 5V or even 3.3V available inside the filter and eliminate one more wall wart power supply.

While I was ordering stuff, I got too many Dallas DS18B20 in the raw TO92 package suitable for putting directly on a PC board and too many additional ESP8266 boards to connect them to. I also got a variety of little terminal blocks to make it easier to build boards for external connections and make those connections later.

Finally, I got some breakout board things which should help with prototyping all kinds of things with the ESP8266 boards. They breakout all the pins on easily accessible terminal blocks.

Hopefully, all these tools and supplies will soon lead to a bunch more ESP based peripherals to my Home Assistant instance.

One of the many tasks the ESP boards can easily do is read the Dallas Semiconductor (now owned by Maxim) 1-wire devices, particularly the temperature sensors. A popular choice is a unit embedded in a stainless steel probe at the end of a waterproof cable a meter or so long.

The Dallas 1-wire protocol is fascinating and clever, but most importantly (if someone else takes care of the fussy bits) it is really easy to use. In the case of ESPHome and the temperature probe above, it is dead simple to connect and read a temperture, but even cooler, several such devices can share a single pin on the device.

For experimental purposes, I used my ESP32 dev board, which has solderless breadboard friendly pins installed, making it even easier to throw together a test system.

Not shown in the picture is the data bus pullup resistor. Because of the distance it needed to reach, from the 6th pin from the left on the top row to the far right pin on the bottom row, I could better fit the resistor under the MCU board with no jumpers.

After a couple of false starts because the IDE for ESPHome is really sensitive to indentation, I was able to identify the unique ID for this temperature sensor then start bringing in data from it.

dallas:
  pin: 21
  update_interval: 10s

sensor:
  - platform: dallas
    address: 0x693ca0e381b17528
    name: "Testboard Temperature"
    accuracy_decimals: 2
    filters:
      lambda: return x * (9.0/5.0) + 32.0;
    unit_of_measurement: "°F"

At the time I am writing this, I don’t know if pin 21 is special, but it was the pin used in the example I followed, so I kept it.

The first thing you do is install the block with only the ‘dallas’ directive and the pin number. The logs will show the address of any devices it identifies. Then you can add the rest of the configuration, complete with the actual addresses of any and all individual devices. Apparently, one can do a ‘default’ device read, but it will understandably fail if there is more than one device.

I played with the update_interval. For my testing purposes, the shorter to better. One thing I will use this board for is testing communication limits, specifically whether I can expect reliable WiFi connectivity in certain outdoor locations from these boards. Rapid updates of the data will help determine that.

The most interesting new skill learned was the lambda directive. This is a special calculation ‘filter’ of sorts. The hardware device reports in degrees C and lambda is used in this case to convert degrees Celsius to degrees Fahrenheit.

One of the practical uses I see for this type of sensor would be to use these probes for each of our non-kitchen refrigerators, one for the freezer, one for the refrigerator and maybe one for the ambient, if some other sensor doesn’t already provide an ambient temperature. I have found the door switches I like also provide a temperature.

Another could be to put various temperature probes into the HVAC system in the attic, return air temp, duct temp, attic ambient, evaporator coil, etc.

I may add one to the hot water meter to monitor the water tank temperature.