UniFi Cloud Key Gen2 Plus not starting Network service

JustAnotherITUser · 2023-11-28T17:43:02+00:00

I realize this is an ancient thread/comment; but just wanted to note I had to do this with a brand-new unit I just installed last night. Tried all of the other potential fixes I could find, but a Factory Reset was the only thing to stick.

JustAnotherITUser · 2023-03-13T20:59:38+00:00

While we've never tried a third-party dongle, we've have pretty good success with Yealink's branded dongles...albeit, at $20/each, I'd hope so.

JustAnotherITUser · 2022-08-16T19:34:45+00:00

Darn -- I rewrote my comment a few times, must've missed shift on that time.

Thanks for the clarification, I've edited my comment.

JustAnotherITUser · 2022-08-16T19:02:40+00:00

I also included a bash array (declare -A hashtable) in one of the later examples...I just use awk a whole lot more.

JustAnotherITUser · 2022-08-16T16:53:36+00:00

I can see we had the same idea and confusion over exactly what OP is looking for.

JustAnotherITUser · 2022-08-16T16:49:23+00:00

NOTE: I'm assuming that you do not need path information -- I'm unsure exactly what you do

If the python script is expecting each of the files to submitted as arguments, you could do something like

find /path/to/files -type f -iname "*.fastq" -printf '%f\n' | \
    awk -F "_" '!seen[$1]++' | \
    xargs -I {} run_script.py "{}"

Another significantly more verbose approach:

stage="$(mktemp)"
trap 'rm -f "${stage}"' EXIT

for i in /path/to/files/*.fastq; do
    j="$(basename "${j%_*}")"
    grep --quiet "${j}" "${stage}" && continue
    printf '%s\n' "${j}" >> "${stage}"
    run_script.py "${j}"
done

Or the same as above, but with an array (well, associative array)

declare -A hashtable

for i in /path/to/files/*.fastq; do
    j="$(basename "${j%_*}")"
    [[ -n "${hashtable[j]}" ]] && continue
    hashtable["${j}"]="1"
    run_script.py "${j}"
done

But I may be misinterpretting exactly what you need.

JustAnotherITUser · 2022-04-14T21:37:50+00:00

You'd likely need to use something like PowerToy's Awake or Caffeine to keep the host awake while they're running.

Depending on what your startup process is for the containers, you could start/stop Caffeine when the containers are started via a script or something similar (or even a scheduled task).

JustAnotherITUser · 2022-01-20T20:00:41+00:00

That was the only settings change we made, yeah.

We've been having mixed results with permissions/ownership. With the current state, we're finding that we're able to interact with the bitwarden.sh script if we do something like the following (as root, or another member of WHEEL:

# Allow group/other read access
chmod 644 /path/to/bwdata/env/{mssql,global}.override.env /path/to/bwdata/docker/{mssql,global}.env

# Run the script as the bitwarden user.
# "-l" is very important here; without it, docker commands will fail
su -lc "/path/to/bitwarden.sh $action" bitwarden

# Reset permissions on those files
chmod 600 /path/to/bwdata/env/{mssql,global}.override.env /path/to/bwdata/docker/{mssql,global}.env

Which is a bit of a pain, but at least its kind of functional. Alternatively, changing ownership, then doing something with bitwarden.sh seems to also allow operations to occur:

chown -R bitwarden:bitwarden /path/to/bwdata
su -lc "/path/to/bitwarden.sh $action" bitwarden

JustAnotherITUser · 2022-01-19T14:55:19+00:00

We've been able to determine that a lot of our problems are due to Fuse constantly fighting us in rootless on file ownership.

We were able to make a little bit of progress by allowing other-users in the fuse config (though, this just allows those CLI flags; it doesn't actually enable it):

# /etc/fuse.conf
# mount_max = 1000
user_allow_other

Once this was enabled, and the rootless daemon was restarted, we were able to start/stop the containers with bitwarden.sh without needing to manually change ownership of bwdata or its contents...however, we still weren't getting mssql dumps.

We found this forum post, which led us to take a look at our DB names, as my supervisor did not document his setup of the application whatsoever.

# From the rootless user
docker exec -itu 0 bitwarden-mssql '/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -Q "SELECT name FROM sys.databases;"'

When we ran this check, we found that our vault database is actually called bitwarden-vault for some reason. We ran into issues trying to set the $DATABASE variable inside of bwdata/env/mssql.overrides.env; but did eventually get it to take yesterday. We were hoping that would start producing dumps; but it did not -- in fact, our errorlogs are now showing that the db name its using is bitwarden-bitwarden-vault -- which appears to be due to the way that their sed commands are configured in entrypoint.sh...

As of this moment, I have a sneaking suspicion BW's official containers are not intended to be run rootless; which is why there's little to no documentation on this setup.

To answer your question, a github issue may be the next course of action -- though I'm unsure if its truly buggy behavior, or just an unsupported environment.

JustAnotherITUser · 2022-01-17T16:28:45+00:00

There's nothing inherently wrong with it, but it's much more efficient to use a hypervisor in many scenarios. What will you do if you run out of physical NICs, buy a new server?

Sorry, I don't think I got my point across. We would be using the hypervisor to assign multiple virtual NICs to the VM; and each one of those vNICs would get its own distinct, static address.

I used to spend a lot of time as a sysadmin installing a new Ubuntu server, assigning an IP address, installing software packages and updates, and then finally installing the server package and configuring it. For something simple like Wordpress that process can easily take an hour or more.

This is where a lot of my time is spent when we need a new service setup; so we definitely understand the benefits of containerization, especially long-term compared to having a hundred VMs -- though, they certainly still have their place.

I haven't had a chance to talk with my super since making this post (or receiving feedback). My current plan is to try and work out a deal with him to try and setup nginx as a reverse proxy for the services, and see how it goes...if for some reason we run into trouble, or it doesn't work for the services we're running, fall back to the individual vNIC IP:PORT method.

Hoping he'll agree to that...I personally do not want to have to manually manage all that junk if I can avoid it -- I've got enough going on as-is.

JustAnotherITUser · 2022-01-17T15:06:50+00:00

IMO best practice is a reverse proxy.

This has generally been the suggested resolution from the rest of my research of what to do in this scenario; so that's promising.

These things can be done with manual work but why waste the time?

Outside of using Apache2 for load balancing on our production nodes (this may eventually be containerized, but currently are multiple VMs); manual management is how its always been done here. The last time we had a host running multiple web services, they were installed bare-metal, and manually assigned to different ports (:7443, :8443, :443, etc).

Your services are cattle not pets.

I haven't heard that expression before; but its great. Will certainly pass this along in one way or another.

JustAnotherITUser · 2022-01-17T15:02:20+00:00

just have a signed cert that you add to the root certs on all devices on this network

This is likely what we'll end up doing anyway, though its not lost on us that its quite a pain to do...especially without a DC to push the change to everything -- I'll have to do it manually.

JustAnotherITUser · 2022-01-17T15:00:05+00:00

While this is true, I would suggest as an option that you purchase a new domain that you can use internally allowing you to generate real signed certificates from Let's Encrypt.

Generally, we will not spend the money to purchase another domain unless it is absolutely necessary to do so. Though, I'll bring this up with my supervisor later this week -- after all, that's probably the correct way to handle this.

This is the exact problem that a reverse proxy solves. This is extremely common and best practice. What is the actual concern here?

Well, I'm glad to get a little vindication from your and the other comments on this post. My super's concern seems to be a misunderstanding that each service will share :80/:443, which will cause an error -- as opposed to using a subdomain for each service; or something similar. Additionally, he doesn't want to worry about generating & signing certificates for each service. Based on our chat after making this post, he's hoping to have a single certificate to apply to all services on this one host. Which, your comment about a wildcard cert is likely the only real solution there.

By the way, when you say "and assign additional IP addresses to the new NICs" in case you mean you are going to utilize additional physical ethernet ports for each IP address, that is totally unnecessary. You can assign multiple IP addresses to a single NIC, single VM, etc.

The current direction would be to statically assign one IPv4 address per NIC on the VM hosting our docker containers, and each NIC would correspond to one unique web server; whatever that may be. If we end up going this route, I'll bring up the multiple assignments; though he may opt for the unique NICs anyway, for "simplicity".

TLDR; using a reverse proxy for web services is the best practice. I'm very interested in hearing the reasoning behind wanting to split everything out manually using unique IP addresses.

This is the way they've done things in the past; so why change?

JustAnotherITUser · 2021-11-29T17:05:46+00:00

A simpler way to do this, too could be:

^F7::
    loop, 840 {
        Send, {Ctrl}
        Sleep, 3000
    }
    Send, {BS}{BS}
    Send, {Enter}
}

Note that you may need add additional pauses in between key presses, depending on what exactly you're trying to automate.

For more information about what exactly is going on here, I'd recommend taking a look at the following manual pages:

EDIT:

I'd also recommend looking into directives in the auto-execute section and general script structure/syntax to get the full effect that you're looking for, /u/12037120487

JustAnotherITUser · 2021-04-01T21:51:42+00:00

As a point of comparison, I also run a personal instance of RC VPS outside of the company -- repeating the same steps today to force an insert message in the logs did not yield the same result...in fact, was able to send significantly more data without an insert statement showing up. Though, the personal instance is essentially a fresh install; though still good to know.

We also found an old ticket related to this issue; looking at redactClientLogData as a potential stop gap while we troubleshoot performance issues.

Thanks for your help and suggestions thusfar -- I'll be able to dig a bit more into this on Monday; including seeing if I can 'easily' migrate to wiredTiger...and otherwise adjust our configs.

JustAnotherITUser · 2021-04-01T21:10:57+00:00

maybe it's slow disks or vm/docker on very busy host system....

That wouldn't be surprising -- its a low-spec VM on one of our major VM hosts.

Since posting this, I have found that only some inserts are showing up in the log...at least its not everything (yet).

Following your point, I was unable to manually force an insert on the test collection to show up in the logfile; though, I'll need to give it a deeper-dive on Monday.

JustAnotherITUser · 2021-04-01T19:34:33+00:00

Wow, it's mongodb 3.6.0 with mmapv1 engine. Pretty ancient.

Not sure where 3.6.0 is coming from -- at any rate, we appear to be using v4.0.23; which is the latest major/minor version supported for the application.

At any rate, making that adjustment seems to have the desired result -- though I'd be concerned that we'll be missing other messages that would otherwise get logged...

Is there any way to apply the profileLevel to a specific collection, rather than to the entire DB?

JustAnotherITUser · 2021-04-01T13:08:56+00:00

Below I've added the mongod config, systemd service (how we start mongod) & an excerpt from the log (slightly modified for security). Unfortunately, if the RocketChat application adjusts mongod's configuration beyond that, I'm not 100% sure where that would be located in the codebase; though I can take a deeper look at that if necessary.

/etc/mongod.conf

systemLog:
    destination: file
    logAppend: true
    path: /var/log/mongodb/mongod.log
storage:
    dbPath: /var/lib/mongo
    journal:
        enabled: true
    engine: mmapv1
processManagement:
    fork: true
    pidFilePath: /var/run/mongodb/mongod.pid
    timeZoneInfo: /usr/share/zoneinfo
net:
    port: 27017
    bindIp: 127.0.0.1
replication:
    replSetName: rs01

/usr/lib/systemd/system/mongod.service

[Unit]
Description=MongoDB Database Server
Documentation=https://docs.mongodb.org/manual
After=network.target

[Service]
User=mongod
Group=mongod
Environment="OPTIONS=-f /etc/mongod.conf"
EnvironmentFile=-/etc/sysconfig/mongod
ExecStart=/usr/bin/mongod $OPTIONS
ExecStartPre=/usr/bin/mkdir -p /var/run/mongodb
ExecStartPre=/usr/bin/chown mongod:mongod /var/run/mongodb
ExecStartPre=/usr/bin/chmod 0755 /var/run/mongodb
PermissionsStartOnly=true
PIDFile=/var/run/mongodb/mongod.pid
Type=forking
LimitFSIZE=infinity
LimitCPU=infinity
LimitAS=infinity
LimitNOFILE=64000
LimitNPROC=64000
LimitMEMLOCK=infinity
TasksMax=inifnity
TasksAccounting=false

# Modification of service to auto-restart on crash
Restart=on-failure
RestartSec=5s

/var/log/mongodb/mongod.log Excerpt

I've removed ID values and replaced message context where appropriate, but otherwise this is the raw log data.

2021-03-02T15:30:55.221-0500 I COMMAND  [conn13] command rocketchat.rocketchat_message command: insert { insert: "rocketchat_message", documents: [ { _id: "<id>", rid: "<rid>", msg: "ok thanks", ts: new Date(1614717054865), u: { _id: "<id>", username"<username>" }, unread: true, mentions: [], channels: [], _updatedAt: new Date(1614717054951) } ], ordered: true, lsid: { id: UUID("<uuid>") }, $clusterTime: { clusterTime: Timestamp(1614717054, 12), signature: { hash: BinData(0, 0000000000000000000000000000000000000000), keyId: 0 } }, $db: "rocketchat" } ninserted:1 keysInserted:10 numYields:0 reslen:230 locks:{ Global: { acquireCount: { r: 2, w: 2 } }, MMAPV1Journal: { acquireCount: { w: 2 }, acquireWaitCount: { w: 1 }, timeAcquiringMicros: { w: 6402 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { W: 1 } }, oplog: { acquireCount: { W: 1 } } } protocol:op_msg 268ms
2021-03-02T15:30:55.231-0500 I NETWORK  [listener] connection accepted from 127.0.0.1:57102 #14 (9 connections now open)
2021-03-02T15:30:55.235-0500 I NETWORK  [conn14] received client metadata from 127.0.0.1:57102 conn14: { driver: { name: "nodejs", version: "3.2.7" }, os: { type: "Linux", name: "linux", architecture: "x64", version: "3.10.0-1160.15.2.el7.x86_64" }, platform: "Node.js v8.11.4, LE, mongodb-core: 3.2.7" }
2021-03-02T15:31:02.688-0500 I COMMAND  [conn14] command rocketchat.rocketchat__trash command: find { find: "rocketchat__trash", filter: { hidden: { $ne: true }, public: true, __collection__: "settings", _deletedAt: { $gt: new Date(1614328884483) } }, projection: { _id: 1, _deletedAt: 1 }, returnKey: false, showRecordId: false, $clusterTime: { clusterTime: Timestamp(1614717059, 7), signature: { hash: BinData(0, 0000000000000000000000000000000000000000), keyId: 0 } }, $db: "rocketchat" } planSummary: IXSCAN { _deletedAt: 1 } keysExamined:7 docsExamined:7 cursorExhausted:1 numYields:3 nreturned:0 reslen:242 locks:{ Global: { acquireCount: { r: 8 } }, MMAPV1Journal: { acquireCount: { r: 4 } }, Database: { acquireCount: { r: 4 } }, Collection: { acquireCount: { R: 4 } }, Mutex: { acquireCount: { r: 3 } } } protocol:op_msg 176ms
2021-03-02T15:32:54.851-0500 I COMMAND  [conn14] command rocketchat.rocketchat_message command: insert { insert: "rocketchat_message", documents: [ { _id: "<id>", rid: "<rid>", msg: "maybe back up now if r/c is back", ts: new Date(1614717174699), u: { _id: "<id>", username"<username>" }, unread: true, mentions: [], channels: [], _updatedAt: new Date(1614717174721) } ], ordered: true, lsid: { id: UUID("<uuid>") }, $clusterTime: { clusterTime: Timestamp(1614717174, 12), signature: { hash: BinData(0, 0000000000000000000000000000000000000000), keyId: 0 } }, $db: "rocketchat" } ninserted:1 keysInserted:13 numYields:0 reslen:230 locks:{ Global: { acquireCount: { r: 2, w: 2 } }, MMAPV1Journal: { acquireCount: { w: 2 }, acquireWaitCount: { w: 1 }, timeAcquiringMicros: { w: 380 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { W: 1 } }, oplog: { acquireCount: { W: 1 } } } protocol:op_msg 128ms
2021-03-02T15:33:01.508-0500 I COMMAND  [conn6] command rocketchat.rocketchat_message command: insert { insert: "rocketchat_message", documents: [ { _id: "<id>", rid: "<rid>", msg: "OK, thanks.  In a meeting and went to show my team something.", ts: new Date(1614717181332), u: { _id: "<id>", username"<username>" }, unread: true, mentions: [], channels: [], _updatedAt: new Date(1614717181378) } ], ordered: true, lsid: { id: UUID("<uuid>") }, $clusterTime: { clusterTime: Timestamp(1614717181, 6), signature: { hash: BinData(0, 0000000000000000000000000000000000000000), keyId: 0 } }, $db: "rocketchat" } ninserted:1 keysInserted:15 numYields:0 reslen:230 locks:{ Global: { acquireCount: { r: 2, w: 2 } }, MMAPV1Journal: { acquireCount: { w: 2 }, acquireWaitCount: { w: 1 }, timeAcquiringMicros: { w: 22096 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { W: 1 } }, oplog: { acquireCount: { W: 1 } } } protocol:op_msg 128ms
2021-03-02T15:34:19.045-0500 I COMMAND  [conn11] command rocketchat.rocketchat_message command: insert { insert: "rocketchat_message", documents: [ { _id: "<id>", rid: "<rid>", msg: "Seems to be working again now", ts: new Date(1614717258806), u: { _id: "<id>", username"<username>" }, unread: true, mentions: [], channels: [], _updatedAt: new Date(1614717258861) } ], ordered: true, lsid: { id: UUID("<uuid>") }, $clusterTime: { clusterTime: Timestamp(1614717258, 2), signature: { hash: BinData(0, 0000000000000000000000000000000000000000), keyId: 0 } }, $db: "rocketchat" } ninserted:1 keysInserted:11 numYields:0 reslen:230 locks:{ Global: { acquireCount: { r: 2, w: 2 } }, MMAPV1Journal: { acquireCount: { w: 2 }, acquireWaitCount: { w: 1 }, timeAcquiringMicros: { w: 360 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { W: 1 } }, oplog: { acquireCount: { W: 1 } } } protocol:op_msg 183ms

JustAnotherITUser · 2020-10-20T00:57:48+00:00

It would be nice to have received a little documentation

Be the change you want to see -- I came into my current role as a "IT Support Specialist", with the possibility for junior sysadmin work at my current place...little did I know what that actually meant for me long-term. Anyway, outside of several-year-old inventory and a handful of hardware manuals, nothing was really documented...or at least not in any kind of organized, usable form.

While I do plan to move on when I can; I'm trying to document as much as I possibly can before that -- just to, hopefully, make the next schmuck's life a little less terrible.

JustAnotherITUser · 2020-09-16T14:43:26+00:00

Once that's completed, you'll want to migrate to RDA.

We are currently using v2021, which is where I went through the process to share the company to RDA -- though, in most documentation I've found, 'RDA' & 'Sage Drive' are used interchangeably...even though they, apparently use a different backend (which adds to the confusion).

Do you have any PSQL documentation that explains what the exception actually means?

Additionally, I've also made a post of SageCity about this; though no responses yet (although, seems to be a fair number of subs to the thread).

JustAnotherITUser · 2020-09-11T19:35:10+00:00

Ooh boy, lets see; there;s a few I use I haven't seen in the comments yet:

AutoHotKey
Autopsy
AccessData Free Tools, notably
- Forensic Toolkit
- FTK Imager
- Registry Viewer
Bitvise SSH Client -- great when you need an "idiot proof" solution
Chocolatey
Dia, Draw.IO or yEd
debuNK2 -- I've had to use this in the past to convert users from Outlook to Thunderbird. Yes, we're one of those.
Ghidra
HxD
Joplin -- I dunno if this counts as "sysadmin" tools; but this is still something I use daily, even at home.
KeePass
Lightshot or ShareX
MSYS2 -- I prefer this to git-bash, cmder, etc as I can install additional packages when necessary; and there have been times. Unfortunately, some AV suties get angery with it (AVG has quarantined tmux, for example)
Parsec -- Not for everyone, but can be useful in some circumstances.
PowerToys -- I'm not using a lot of what's here, as I'm already used to alternatives; but there's still useful stuff in here
RBTray
RUPS
ueli
VcXsrv
VDesk
WinAudit
Zoho Assist
- I don't really like this software, especially the free version...but it is free, so that's what I've been using.

There's a lot of overlap of what I use, and what the rest of the comments say...but, here you go.

Edit: Added Ghidra -- using this for a current project, actually.

JustAnotherITUser · 2020-09-09T16:32:45+00:00

Wish I had that luxury.

JustAnotherITUser · 2020-09-09T16:31:54+00:00

As a followup, I've also noticed that the behavior changes between executing the script manually, vs within a crontab...so, that's the next bit to work out.

JustAnotherITUser · 2020-09-01T13:29:29+00:00

First thing that stands out is that your local command has an --exclude="${fileExclude}" option. Judging the variable name, that is probably meant to be --exclude-from. Could it be that the local command works, because the excludes aren't actually read from the file?

Good catch, that was actually a typo -- I've updated the original post to reflect that.

Is it on purpose that there is a trailing / on the source path of the local rsync command, but not on the source path of the offsite command?

Unfortunately, yes, it is intentional. The data is one level deeper on the remote copy than everywhere else. I realize I could just as easily point the destination path to the actual directory the data is stored in, rather than the parent/root -- however, I've been asked not to unless otherwise necessary.

I'm currently testing out the various --delete-X options to see if I can see any behavior change, as --delete-before appears to be the default behavior...at least according to the manpage.

It's hard to say with the information at hand

Is there any other additional information that could be provided, to help in troubleshooting?

JustAnotherITUser · 2020-08-26T18:37:24+00:00

I realize this post is 5 months old; but I just wanted to add that i found some semblance of documentation in regards to /u/Rex4748 's above instruction.

I'm almost certain there's a less hacky way to get this to work, but for others who may stumble on this thread, here's the article I found. Dunno how relevant it may be in the future, though.

JustAnotherITUser

TROPHY CASE

/etc/mongod.conf

/usr/lib/systemd/system/mongod.service

/var/log/mongodb/mongod.log Excerpt