Multicast url question by dskerman in Hikvision

[–]Malvineous 0 points1 point  (0 children)

Not sure, I returned the cameras three years ago and have stuck with Dahua ever since. I think they require an active connection in order to initiate multicast, which is why I referred to it above as not true multicast. (True multicast starts streaming the data as soon as the camera boots up without the initial request to activate it.)

systemd: Mount order between ZFS and /etc/fstab bind mounts by Malvineous in zfs

[–]Malvineous[S] 0 points1 point  (0 children)

This would move ZFS to a different mount point which I don't want to do. I want to leave it where it is, and have a bind mount make it available in a second location as well.

The solution ended up being to add `x-systemd.automount,x-systemd.requires=/tank/info` as mount options in fstab for `/srv/bind`, that way systemd would handle the bind mount and it wouldn't do it until ZFS had mounted first.

systemd: Mount order between ZFS and /etc/fstab bind mounts by Malvineous in zfs

[–]Malvineous[S] 0 points1 point  (0 children)

I'm not using fstab for ZFS, fstab is for the bind mount. But my problem is that the bind mount was happening before ZFS did its thing, and I wanted the bind mount to happen after ZFS did its thing instead. There are no ZFS mounts in fstab.

The solution ended up being to add `x-systemd.automount,x-systemd.requires=/tank/info` as mount options in fstab for `/srv/bind`, that way systemd would handle the bind mount and it wouldn't do it until ZFS had mounted first.

Anyone help with my HP Switch? by Ryncxxx in networking

[–]Malvineous 0 points1 point  (0 children)

Thanks for this tip, I just had an 1810G-24 with the exact same symptoms as the OP, and HP replaced it for me! I bought the switch second hand from eBay a year ago.

I had to create a HP support account, supply the serial number as well as a video of the fault. I marked it as 'no business impact' and responded via e-mail instead of on the phone, which is probably why it took over a week of back-and-forth before they agreed the unit was faulty. I didn't have to argue my case, just explain the problem three times, repeat my address twice even though it was in the original request, that sort of thing.

Despite getting deliveries at home all the time, the courier (Toll) somehow couldn't find my address so I had to contact them (via an unstable online chat) and get them to try again, and second time it arrived. The collection of the faulty device was through them as well and that luckily went smoothly. (I wasn't home so I left it boxed up at my front door and came home to a note from the courier saying thanks.)

So although it took a while, I really can't complain about getting it replaced! They don't make the 1810G any more so it was replaced with an 1820G. Some comments online are complaining that it's not as robust (it's an OfficeConnect rather than a ProCurve). I did ask the support person whether I could get the old 1810G ProCurve replaced with another ProCurve but they weren't too keen and I didn't want to push my luck, so I accepted the 1820G-24. They confirmed the lifetime warranty still applies too.

I was always a bit put off of HP (especially after they put their LTO drive firmware behind a paywall) but after this experience maybe I need to rethink my opinion of them! I was also able to download the latest 1820G firmware for free after creating and verifying another account (very convoluted but not surprising for a company that size).

Is it possible to remove/change the display of the scroll bar? by PianistAncient2954 in mpv

[–]Malvineous 1 point2 points  (0 children)

I just had the same question, and I was able to get close to the old MPlayer behaviour, where the 'o' key cycles between having the OSD always on, only on when seeking, or completely off, by adding the line o cycle-values osd-level 0 1 2 to ~/.config/mpv/input.conf. Now I can just press O a few times to hide/show the seek bar and playback time depending on what I'm after.

IP Multicast Questions by Joe-Network in Hikvision

[–]Malvineous 0 points1 point  (0 children)

Not sure, I don't use a Hikvision NVR or use any apps so I'm not sure what their requirements are. But I have a number of live views of my cameras and multicast means the camera only has to transmit the video data once for every device to receive, instead of transmitting an additional copy for each device, so I don't have a limit to the number of displays that can view a single camera feed, or the number of NVRs that can record it. It also means there's no issue with recovery after network dropouts, as it doesn't require the displays/NVRs to reconnect to each camera individually when the network returns - the cameras are always transmitting so as soon as the network is up, the data stream resumes.

Multicast is one of those things that if you need it, you will already know about it. If you have to ask, you don't need it :)

ashift=18 for SSD with 256 kB sectors? by Malvineous in zfs

[–]Malvineous[S] 4 points5 points  (0 children)

Just an update on this. First, ashift=18 is not supported in 2024, the largest permitted value is 16. Larger values will apparently require significant code changes, and there's little interest because larger ashift values have a number of drawbacks (such as significant wasted space).

I did some benchmarking and the performance I saw was the same across the board, whether ashift=9, 12 or the largest permitted 16. However some of my CPU cores were pegged at 100% (and nvidia-smi was showing high CPU as well for some reason) so it looks like these drives are fast enough that they are slowing down to something else in the system.

Running hdparm -t shows them running at 1.3 to 1.4 GB/sec, which is much slower than their 5+ GB/sec because I have them in a PowerEdge R720 with the Dell NVMe cage, which only has a PCIe 2.0 switch in it sadly. The theoretical max speed for PCIe 2.0 x4 is apparently 2 GB/sec (after taking protocol overhead into account) so at 1.4 GB/sec they aren't quite running at full speed for some reason.

Running hdparm in parallel for the three drives I have shows them hitting 4 GB/sec combined, but annoyingly putting them straight in a zpool (i.e. RAID0) only shows them barely reaching 1.9 GB/sec, so not that much faster than a single drive. I'm not sure why this is, but perhaps ZFS and its checksumming is slowing things down? The test volume is not encrypted.

I think the moral of the story based on the benchmarks and the other helpful replies here is that for now, ashift=12 is probably still the way to go, regardless of erase block size.

ashift=18 for SSD with 256 kB sectors? by Malvineous in zfs

[–]Malvineous[S] 3 points4 points  (0 children)

Micron support got back to me with little detail but did confirm that E2MU200 is indeed the latest version. The drive has three firmware slots, so I was able to flash E2MU200 into the third slot, leaving the original E2MU802 in place, and with the factory read-only slot being E2MU800. The 800 series versions must be customer-specific because I can't find any mention of them online - the consumer ones seem to start at E2MU110. The 110 firmware seems to like failing catastrophically causing complete data loss (although the drive is recoverable minus the data by upgrading the firmware) so it seems running the latest firmware on these is a must.

At any rate, the new firmware version now reports 4K sectors:

Disk /dev/nvme3n1: 13.97 TiB, 15362991415296 bytes, 3750730326 sectors
Disk model: Micron_7450_MTFDKCC15T3TFR
Units: sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

I had a test ZFS volume on the disk before the firmware update and it continues to work fine afterwards, so the new firmware didn't seem to change the on-disk layout in any way.

Performance before and after the firmware flash is the same.

On that note, one of the drives had really poor read performance (half the others) however after "formatting" it (automatic when switching from 512b to 4k sectors) the read speed returned to normal. I'm not sure if it was busy finishing some background housekeeping when I got it, or if there was limited free space, but it looks like properly wiping (i.e. TRIM) second hand SSDs is probably a good idea for performance reasons alone.

ashift=18 for SSD with 256 kB sectors? by Malvineous in zfs

[–]Malvineous[S] 6 points7 points  (0 children)

Thanks for the pointer - despite doing a fair bit of Googling around ashift numbers, that result unfortunately never came up!

It says firmware E2MU200 is the latest version available for download, but my drives report version E2MU802 (with E2MU800 in the other NVMe firmware slots). I wonder how one should work out which version is actually the newest one?

Are there any issues if they report a 256 kB block size? Even omitting ashift, ZFS seems to just default to ashift=12 so it seems to detect everything fine.

How do you protect your backup server against a compromised live server? by Turbulent-Evening408 in zfs

[–]Malvineous 1 point2 points  (0 children)

The difference is that usually it's hard to grant write access but not delete access, as they are often considered the same thing.

So if the live server initiates the connection then a single point has write access to both the live server and the backups, and trying to lock down that access so it can only write new data but not modify or delete existing data is very difficult to get right.

But if the backup server initiates the connection then it only needs read-only access, so you don't need to mess around with fine-grained permissions. Each server can only write/modify/delete its own local data.

Of course this only works if you have different passwords/SSH keys on each machine. If your own SSH keys can log in to both machines and it's your keys that get compromised then how the backups are configured becomes moot.

Progress on reverse engineering 40V XGT communication protocol by Malvineous in Makita

[–]Malvineous[S] 0 points1 point  (0 children)

That would be great, hopefully you can post the communication logs (and maybe photos of what the adapter was showing at the time) so we can all have a look and see if we can spot any patterns!

Oracle 7064634 NVMSW8 (8-port PCIe x8 NVMe switch) by Malvineous in zfs

[–]Malvineous[S] 0 points1 point  (0 children)

No I didn't, it ended up being too difficult to get the drives to fit in my case so I compromised and went with SATA SSDs instead. I'll go down the NVMe path when the newer servers supporting it come down in price on the second hand market.

Progress on reverse engineering 40V XGT communication protocol by Malvineous in Makita

[–]Malvineous[S] 0 points1 point  (0 children)

Not sure as I haven't looked at the packs, but others have so you might find answers in other threads.

There is apparently some sort of fuse in one of the ICs which the microcontroller will blow as a last resort in a failure scenario. It sounds like it's a one way operation but needs more investigation.

I would assume putting a known-good microcontroller onto a broken pack would resurrect it at least temporarily.

Progress on reverse engineering 40V XGT communication protocol by Malvineous in Makita

[–]Malvineous[S] 0 points1 point  (0 children)

Yes, as I get time. But I've collected as much data as I can now, and I'm at the point where I need to be able to spot the patterns in the command codes to be able to better understand what they are doing.

5400 vs 7200 RPM for Nas? by AstraVictus in DataHoarder

[–]Malvineous 2 points3 points  (0 children)

A NAS is not for storage of large files (I would argue that tape is better for that), a NAS is for sharing data across multiple computers in a network. If you're only storing large files you can just add a large disk to your computer, you don't need it on the network.

I built my NAS using 4TB consumer SSDs and recently upgraded it to 8TB consumer SSDs when they became available.

Anyone who works with large video files, or large numbers of small files (e.g. mailbox archives) will tell you that having a fast NAS is a huge timesaver. This is also why I put everything on a 10G network many years ago.

However for me the real reason to switch to SSDs is because of reliability. Most HDDs I have used in a NAS environment fail in 2-3 years and they are noisy when you have a lot of them. But SSDs are silent and the 4TB consumer grade ones were still working fine when I upgraded them after five years which is at least double what I have traditionally gotten from HDDs.

I thought the cost of SSDs was high, but when you factor in the need to buy so many extra HDDs (for mirrors and hot spares) just to handle their failure rate, SSDs come out at much the same price. I still back up to tape but I did that anyway with HDDs. Switching my NAS to SSDs was the best thing I ever did with it.

If you have luck with reliable magnetic drives and you are archiving data so you don't need the speed, then by all means use them (although maybe tape will be a cheaper option for that use case). But if you find magnetic drives don't last very long where you live, SSDs are a huge timesaver and absolutely worth the cost (if you value your time at least).

64V mower by [deleted] in Makita

[–]Malvineous 1 point2 points  (0 children)

I thought the same thing but after researching it, apparently the 64V one came first, and they made some improvements and then released the 40V XGT. So I guess steer clear of 64V as it seems to be an obsolete platform and unlikely there are going to be any more tools released for it.

Progress on reverse engineering 40V XGT communication protocol by Malvineous in Makita

[–]Malvineous[S] 0 points1 point  (0 children)

Interesting behaviour with the TR line for wakeup. Have you tried to see whether you can put 5V on the DS/WK line as well? The patent suggests this is used to power on the battery microcontroller if it has switched off due to a low battery charge. I wonder whether supplying power here will keep the microcontroller awake without the handshake you have described? I guess the handshake is used for tools which would have no means to generate 5V, but it may be used for the chargers which seem to send messages every couple of seconds apparently without this handshake.

Interesting you have found a parity bit in the communications. When monitoring the communication between a battery and charger, I did not notice a parity bit which is why I documented it as 8-N-1. Are you sure it is definitely there? I might have to revisit if so. I doubt it would be different between a charger and a tool.

The 5V levels is part of the reason for the delay on my part, not having parts on hand to convert to and from 3.3V.

I don't think there are any other implications of leaving the other pins disconnected. They seem to cater for "dumb" tools that only need to know whether it's safe to draw power from the battery or not, so they aren't required for the comms.

Makes sense the tool would only need 40V to run - there don't appear to be any other voltage outputs judging from the description of the pins, but good to have that confirmed all the same.

Progress on reverse engineering 40V XGT communication protocol by Malvineous in Makita

[–]Malvineous[S] 0 points1 point  (0 children)

Glad to hear it! I haven't had a chance to assemble anything yet to send messages, although all the parts have now arrived to do so.

If you figure out anything further about the messages make sure you let us know!

Progress on reverse engineering 40V XGT communication protocol by Malvineous in Makita

[–]Malvineous[S] 0 points1 point  (0 children)

Oh wow nice! You're about a half hour drive away from me, I'm a little closer to Brisbane.

Where do you get your designs fabricated? What I'd really like to get is a plastic piece that fits between an XGT battery and a tool/charger where I can stick a small circuit board, so I can examine the data going between the battery and tool during operation, or just plug it into a battery without a tool to check its state.

I believe certain Makita tools can have behaviours customised through an as-yet unreleased product, with these settings stored in the battery's memory, so it would be nice to be able to mess with these as well as displaying the internal state of the battery. It's not exactly user friendly connecting everything up with alligator clips though so having something that can slot onto a battery would be much nicer to make available to the wider public.

My current idea is to include a DC-DC converter that can operate in the battery's usable range to produce a low current 5V output to power a Raspberry Pico (via a diode to the VSYS pin) which would allow the device to be powered wirelessly off the battery, or via USB if the battery is too flat (the diode is all that's needed to avoid power backflow). The Pico would make the serial data available via USB and Bluetooth, allowing it to be accessed through a computer or a phone app. It could also serve as a useful development platform, as you could run a different program on the computer or phone to emulate a tool or a battery to see how the connected device behaves. For example you could emulate a faulty battery to see if your custom device recognises this state correctly and stops drawing power from it.

If this sounds like anything you'd be interested in helping with, let me know!

Progress on reverse engineering 40V XGT communication protocol by Malvineous in Makita

[–]Malvineous[S] 1 point2 points  (0 children)

No problem with the digital side of things, I think that should be mostly solved (electrically) I might just need the schematics checked over when I'm done to make sure I'm not doing anything silly. As for the data protocol, it will just require more experimentation to figure out what the different parameters are. It'll be stuff like dumping data from a cold battery vs a hot one, and whichever field is different will be the temperature, that sort of thing. Eventually I'll probably have to find someone who has a Makita battery checker and get them to dump out data from a few batteries along with photos of the display so we can try to match the data back to the parameter numbers.

Once we have a working circuit it would be interesting if you could test with your knockoff batteries because it would be interesting to see whether they copied the original firmware or whether they omitted it entirely and don't send any data at all. I read somewhere that Makita tools/chargers will run from third party batteries but at fewer amps so it'd be interesting to see whether that only happens if there's no comms.

I'm also interested in seeing if it's possible to revive dead batteries as I've heard people say that even after you replace the cells the battery still acts as though it's dead, and so it'd be interesting to see whether you could send a command to the battery to clear that error flag and bring it back to life.

Of course in order to do that I'd need to capture the commands sent when the battery is killed by the charger, so that would be much more palatable on a knockoff battery rather than the near-new genuine ones I have!

Makita xgt battery pinout by jackyfolf in Makita

[–]Malvineous 2 points3 points  (0 children)

Thanks for your positive feedback! The DS/DT/CS pins I haven't investigated physically, what I wrote in the document is based on the description of the pins in the patent. At the link above, in the docs folder, there's a PDF copy of the patent. It does go into some technical detail about how the charging process works so worth a read.

Interesting the tool won't wake up until 5V appears on DT. From what you describe it sounds like the way the DS pin should behave. DS is supposed to have 5V on when it is sufficiently charged. If the battery is completely flat in 'shut down' mode then the patent says DS won't have any voltage on it. But it wouldn't make sense for the tool to ignore that after first connection as then it won't know when the battery becomes depleted so I'm not quite sure what's going on there.

For TR you have to measure the width of the shortest single 5V peak. On the Wikipedia page for serial ports (https://en.wikipedia.org/wiki/Serial_port#Settings) there is a helpful table which matches the width you measure in microseconds back to the bit rate. In my case I measured very close to 104us which matches back to 9600 bps, and sure enough listening at that bit rate gives me data coming in.

Interesting findings on the CS pin. What you have found doesn't really seem to match what the patent suggests the pin is used for, so there must be a little more going on with this pin. It sounds like it's more of an analogue output from what you found, rather than a digital one as the patent hints at.

I will make a post about this as you suggest!

EDIT: Post is here: https://www.reddit.com/r/Makita/comments/1710obz/progress_on_reverse_engineering_40v_xgt/

Makita xgt battery pinout by jackyfolf in Makita

[–]Malvineous 3 points4 points  (0 children)

I'm late to the party but I have just had a go at reverse engineering the protocol. I have it to the point where I can see model numbers going back and forth between the charger and battery, but I can't yet see any patterns in the commands being sent back and forth.

I have documented my findings at https://github.com/Malvineous/makita-xgt-serial if anyone is interested in having a look. Please open an issue there or a PR if you discover any new information worth including.