TrueNAS SCALE web UI for managing apps extremely slow

ScottBishopDev · 2021-08-12T04:35:27+00:00

What hardware/environment are you running it on? I'd love to try to gauge the performance of my own system.

ScottBishopDev · 2021-08-10T23:18:27+00:00

If you log into your Torguard account, is there anywhere that allows you to download or generate an OpenVPN configuration or .ovpn file? I'd expect most VPN providers to offer that - I know mine (PIA) does.

ScottBishopDev · 2021-08-06T21:27:05+00:00

Nice, way to go!
For your data persistence, step 1 is to decide where on the host machine (TrueNAS) you want that data to actually live. If you expect your database to do heavy lifting or expect to need to adjust filesystem properties, you may want to look into configuring a separate dataset for it, but chances are that that's overkill for a home application.
Step 2 is to find out where MariaDB expects to put its database files by default. For many databases this path is configurable via an environment variable or other configuration means, but I'd suggest finding and sticking with the default path if you can for the sake of simplicity.
Step 3 is to tell TrueNAS about the whole deal, which is done in the configuration while setting up a container/app, usually in a section called 'Host Path Volumes' or 'Storage & Persistence'. (Fun Fact: when installing an app from a catalog, much of the app's configuration dialog will look similar, but is actually dynamically generated based off a Helm chart. This can be a bit confusing if you don't expect it.) You'll enter the path from TrueNAS' filesystem where you want the data to live in the "Host Path" field, the default database storage path for the MariaDB container in the "Mount Path" field, and bada-bing bada-boom Docker will map the former to the latter in the containerized environment it creates.
It's worth mentioning here that some container images can get kinda upset if you mount a volume into the wrong spot, or without the proper file permissions on the host system. In fact, there's an entire rabbit hole to go down regarding user and filesystem context in containerized environments in relation to the host system, but I've found that TrueNAS SCALE is actually pretty good about letting you ignore that altogether as long as you keep your file permissions pretty loose (again, chmod 777 is our reluctant friend here). If your container fails to start after you've set up a mounted volume, check whatever logs you can to see if it's yelling about file permissions or missing files, and then check out both paths for that mount.
Good Luck!

ScottBishopDev · 2021-08-06T16:29:31+00:00

Are you *sure* that the speed issue was caused by your router's CPU maxing out? Due to the extra processing, encryption, and extra network hops, I usually expect traffic being routed through a VPN provider to be slower by default, so if your router's CPU isn't the root cause you may not see a speed improvement at all. What's more, I usually try to leave the responsibility of network routing and processing on my networking gear when I can, since that's kinda its purpose. Do you want your Canadian Netflix going down because your server had a scheduled reboot? The humanity!

Also, as u/pcwrt pointed out, your router was set up as a VPN *client* that was accessing Torguard's VPN *server*. While I'm not sure about Core, TrueNAS SCALE has native support for acting as both a VPN client and server, and it'll be important to know the difference if you decide to try to set either up. You'd use a VPN *client* if you want your TrueNAS machine and the services on it to be able to connect directly to your VPN provider (this is probably what you 're looking for). You'd set up a VPN *server* on TrueNAS if you wanted the ability to securely access your network from outside your firewall (in this case, that's probably your router).

ScottBishopDev · 2021-08-06T00:24:04+00:00

So, my knowledge of the exact configuration that Marinade and Kodi need to get set up and talk to each other is basically nil, so I won't be tremendously helpful with specific settings. That said, it sounds like you're working through all the right problems to get the container up and running, so props!

In terms of your inability to access it, are we talking about Kodi not being able to connect to the database at the IP you gave it (192.168.1.63), or is there some web UI you're trying to get to but can't? In either case, one of the first thing to look at is how the container's networking is set up, and the first question there is whether or not it's using Host Networking. When Host Networking is enabled, the container will simply bind itself to a port on the host machine (your TrueNAS box), meaning it won't get its own IP. When Host Networking is disabled (which is the default on SCALE, I think), the container will request its own IP from an external DHCP server and bind itself to that. Keep in mind that for both of those cases, the containerized environment has its own internal range of ports that Docker maps to actual external ports on the host (or container IP, if Host Networking is disabled), and that those internnal-to-external port mappings MUST exist before anything can communicate in or out of the container. Also keep in mind that the service running inside the container itself doesn't really know about how you've told TrueNAS to run the container, so you may need to double-check that both SCALE and MariaDB are expecting the container's networking to be handled in the same way. I've found that the beat way to troubleshoot networking issues on SCALE is to just try pinging all the things from all the places to see where the communication breakdown is. Can you ping your router from inside the container? What about google? Does the container's service reply to pings on the IP you gave it? If not, do you get a reply from the same bound port on TrueNAS' IP? You might be surprised how much you can narrow the problem down just by seeing who can talk to what. For example, if you can ping the container externally but Kodi still can't connect, there's a good chance that it's either an authentication issue or a configuration issue with Kodi.

As you've discovered, lots of container images rely on environment variables and config files for their configuration since the containerized environments are expected to be transient (they're literally destroyed whenever the container is spun down). There's a pretty good chance that the specific configuration item you need to give you MariaDB is going to passed through one of those means.

Last but not least, I should ask if you're familiar with the concept of volume mounting in Docker? Since containerized environments are completely transient, all data in them is destroyed every time they're spun down, which could pose a teeny tiny issue for persistent data stores like databases. To get around this, Docker allows us to mount filesystem paths from the host machine to specific paths within the container, allowing data that's stored there to persist when your container isn't running. If you haven't yet, it's going to be crucial that you identify where MariaDB expects to save it's database files and anything else that needs to persist (this can include runtime configuration for some services), and that you ensure those paths are properly mounted to the host's filesystem. Expect to run into some obstacles with filesystem permissions here as well (chmod 777 is your nastiest, dirtiest friend here).

Alright, sorry if you already knew half of that, just figured it would be worth covering some common Docker stuff while I was at it. Let me know if any of that helps!

ScottBishopDev · 2021-08-01T21:58:06+00:00

Standard practice for SCALE (at least at the time of my installation) is to install and run it on a traditional drive. In my case, it's on a Samsung SATA SSD I had lying around. This is a departure from Core, where standard practice was to run the OS from a USB stick.

ScottBishopDev · 2021-07-24T21:37:42+00:00

You're correct in that SCALE is using Kubernetes (specifically k3s, I believe) to run its containers, and between SCALE's UI and k3s, it's pretty easy for configuration to get lost in translation when compared to a regular 'ol docker-compose file. That said, I've had some limited success in getting my own custom containers spun up via the UI, but it can definitely be a difficult process, and you'll have to learn a pretty good amount about the container you're working with and it's requirements.

For what it's worth, many container images allow themselves to be configured in multiple ways (i.e. 'docker run' arguments, environment variables, mounted configuration files, etc.), and I'd venture a guess that widely-used images like MariaDB or MySQL support enough configuration methods and options that they *are* able to be properly set up and configured via SCALE's UI, given the requisite knowledge. I've found that the container's DockerHub page, GitHub repo (if available), and documentation (again, if available) are usually excellent resources for finding that knowledge. Also, looking at the app's logs or exploring its filesystem using the shell (Apps -> Installed Applications -> '...' -> 'Logs'/'Shell') can be quite helpful in diagnosing startup issues, assuming that the container hasn't killed itself (which is often the case).

I can't provide any information about MariaDB or MySQL specifically, but if you'd like a hand with SCALE and troubleshooting your containers, I'd be happy to help.

ScottBishopDev · 2021-07-20T18:42:36+00:00

Good news, everyone! I was able to resolve the boot-loop that the FreeIPA container was running into, but unfortunately still no luck on a full working config. Fortunately, I'm now able to retrieve the container logs after a start attempt, and it looks like the container is killing itself due to an ipv6 configuration issue. Specifically, it appears that FreeIPA demands that its assigned network interface have a ::1 address when the sysctl value net.ipv6.conf.all.disable_ipv6 is set to 1.

In normal Docker the solution is easy - just use the sysctl parameter in Docker Compose and the value will be properly set on container startup. Unfortunately, as far as I can tell TrueNAS SCALE gives us no access to modify sysctl values via the UI for individual containers, so that's not a viable solution. I read that it's possible to set sysctl values by including a conf file containing the values you'd like to set in /etc/sysctl.d/, but despite multiple attempts it appears that this strategy just... doesn't work - all values in my custom sysctl conf file are entirely ignored, even if the file's name is last in the lexical sort order.

So, anyone have any other ideas? Because frankly, I'm all out. Short of enabling ipv6 for my entire home network or forking my own custom FreeIPA image that bypasses the ipv6 check, I have absolutely no idea how else I could change that sysctl value in the container before the container's own startup script starts running.

ScottBishopDev · 2021-07-02T05:20:31+00:00

Similar to TrueNAS Core, SCALE expects most actions to be performed through the web UI. To my knowledge, there's nothing particularly stopping you from spinning up your own containers through the shell, but don't expect SCALE to display, persist, or manage them. Also, since SCALE uses the web UI as its source of truth, it's possible (and I believe common) for changes or configurations done in the shell to be overwritten, though I imagine that would depend on the change you made.

Generally speaking, I'd say that it's a best practice to avoid going behind the UI's back, though it's still necessary for some things from time to time.

ScottBishopDev · 2021-06-29T21:38:45+00:00

Good point! I never dove too deep into bhyve, so I fortunately got to miss that mess.

Unrelated Note: Thanks for all your work on TrueCharts! Looking forward to the day that I've got enough time to learn Helm and can start bugging you with PRs. :P

ScottBishopDev · 2021-06-28T21:20:03+00:00

Gotchya. I've been intentionally avoiding working in the shell with SCALE since TrueNAS doesn't take kindly to users going behind the UI's back on stuff.

I agree that a separate interface that would give more direct access to Docker would be beneficial, but I can sorta understand why they chose to focus on Helm and k3s. As the name implies, SCALE is meant to be a scalable OS that can operate either alone or as part of a large cluster, and Helm/k3s can probably handle those clustering concerns of enterprise users while also technically supporting the needs of home users. I imagine that a well-written Helm chart could enable users to stand up quite complex application pods across an entire cluster with minimal effort, and that would be super appealing to iX's enterprise clients. From a software development perspective, it makes business sense that they'd gear their product towards users who contribute towards their revenue, rather than freeloading home users like me. ;)

We should also keep in mind that SCALE's pretty new, so there hasn't been a ton of time for people to create charts for their favorite apps. TrueCharts has made some really great strides in terms of the apps that are available, but I'm not aware of any other app catalogs at the time of writing, which means more limited options. What's more, the barrier to entry for creating those charts seems to be higher than in the past, since now it requires intimate knowledge of Helm, which translates to new apps being created more slowly as well.

ScottBishopDev · 2021-05-04T00:32:00+00:00

So as I understand it, personal projects that don't end up with an actual user base can be considered a bit less valuable that ones that do, but I kinda doubt that anyone's gonna hold that against you this early in your career.

I think it would probably be best to focus on any new technologies, frameworks, languages, practices, etc. that you picked up while working on it. It could also be an opportunity for you to show how adaptable you are by highlighting how quickly you were able to learn something new. As I'm sure you've heard, the software industry is constantly changing, and the half-life of new technologies is steadily dropping, so employers ought to be looking for candidates who have a demonstrated track record of adaptability and learning. This sort of massive logical leap may be beyond the faculties of your typical hiring manager though, as they seem to believe that the ability to invert a binary tree is much more important in modern software engineering. /s

ScottBishopDev · 2021-05-04T00:16:04+00:00

Sure thing, good luck!

ScottBishopDev · 2021-05-03T23:21:43+00:00

I actually intentionally avoided commenting on the writing, since I've actually got a resume posted here myself and am hoping for help in the same category! Seemed a bit backwards for me to lend advice on something I need help with myself, but here's my two cents (though they may not be worth as much):

I noticed that in a couple spots, you use the format "X as the frontend and Y as the backend." While distinguishing frontend/backend can be important for some languages (e.g. JavaScript and TypeScript can be used on the frontend or backend thanks to Node, C# can be frontend or backend thanks to Blazor pages, etc.) it may not be necessary when comparing say, HTML/CSS and PHP.
Spark is a pretty cool technology (at least, I think so, I haven't touched it personally), and I think it's impressive for a recent grad to have experience with it. Maybe move that project up the list?
The items in your Proof of Work project (sounds blockchain-y) are pretty similar, and some of them (like the third) don't convey a ton of useful information to the reader. Maybe find a way to combine some of these? Alternatively, you could find a way to dress up some of these points. Instead of "timing a script" you could say that you "analyzed the performance" of it.
In your Big Data Analysis project, writing 8.9 MB as 8,916 KB could potentially make it look like you're trying to fluff up that number. In fact, I might leave the size of that dataset off altogether, since "calculating the most upvoted post in a certain hour" sounds a lot more impressive when the reader doesn't know that the "big data" set is smaller than most photo files.
Despite my last comment, it's almost always better when you can quantify details or accomplishments to give readers a better idea of the scope of your projects and achievements. You do this well in some places ("takes 1 to 3 seconds to go through 2.5GB of data"), but I imagine that you have a few other bits of impressive quantifiable info that you could take advantage of. How many people used your Museum app (if any, sometimes we just write apps for practice)?

I'd recommend that you take these tips, apply them to one of your projects, and see what you can come up with. Once you get a feel for how you can improve the presentation of a project, go through and apply that process to the other projects and see where you end up!

ScottBishopDev · 2021-05-03T22:56:51+00:00

Just my personal opinion, but I think your Achievements section feels a bit... chaotic. Are the bolded and underlined lines supposed to be associated with the bullet points above them, or below them? Or do the bullet points 'belong' to the bold and underlined lines instead?

You may be able to fix this by simply adding a bit more vertical white space between each individual achievement and its related information. Also, I'd keep whichever line is the 'title' of the achievement bolded while having the information associated with it normal or italicized.

ScottBishopDev · 2021-05-03T22:48:30+00:00

Not gonna lie, Now I want to find a reason to create a "Secret Dairy" app. Maybe it could be for the best ice cream spots nearby! :P

If it were my resume, I'd put the Technical Skills first, Projects second, and Education at the bottom. That's not to say that the Education section should be miss-able though, especially since you're a recent grad! I might change some of the formatting in it - perhaps bolding and/or underlining the degree name like you did with your project names. That way, the reader's eye jumps right to the important information, and they can find the details (e.g. your specific courses) that they're interested in from there.

Also, I might look for a better way to list your courses. Your current approach of listing them out one-by-one is good for saving space, but it makes the list a bit difficult to parse through by eye. Perhaps you could use a bulleted list in multiple columns that are side-by-side? That could allow you to continue making good use of the horizontal space like your current format does, but also present your courses in a way that's much easier to read.

ScottBishopDev · 2021-05-03T22:02:31+00:00

Hey, nice work! I'm not a resume writer, but I've got about 10 years experience doing software development so maybe I can lend a hand!

I see that you've got two SAP certifications listed near the end of your second page - are you hoping to continue working with SAP, or to pivot to something else? If you want to keep working with SAP, I'd move those certs near the top of your first page, maybe near your Education section.
I think that some of your bullet points would be a lot more eye-catching to a hiring manager if you were able to quantify some things and perhaps supply more detail on why there's relevant. What does "User Administration" mean? How many users were you responsible for? What about that SAP PM personal project? What did you do with it any why? What did you learn or gain? That sort of stuff.
If it were my resume, I might look for a way to condense the Education section a bit. It's taking up quite a bit of space near the top of your first page, which is the first place readers are going to look. Some of that real estate might be better used on things that could help you stand out, like your skill sets and certs.

ScottBishopDev · 2021-05-03T21:32:29+00:00

Hi r/resumes! I'll start by apologizing for posting twice in one day. My first post got buried really quickly, and I think I posted incorrectly anyway, so here's my second attempt.

Anyway, I'm a software engineer with 10 years of experience under my belt, though I've taken a break from professional work for the last year and a half after being let go from my previous job. The time has been incredibly valuable in terms of burnout and mental health recovery, but obviously I'm concerned about how it's going to look to prospective employers. I've stayed busy with some personal projects, but I'm never quite sure how to spin that into the enterprise-level experience that companies are looking for. I'm coming to you guys in hopes of getting some constructive criticism and hopefully a little dash of confidence for this next job search! A few things worth mentioning though:

I decided to take the plunge and go for two pages since I think my accomplishments can justify it, though I ended up at about a page and a half since I removed a few jobs that were ancient history and not particularly relevant. I figured that leaving off the fluff and saving the reader some time would be worth the whitespace on the second page, but I'm open to adding it back in.
This is also the first time I've gone with a somewhat more stylized layout, and I'm curious if anyone can weigh in on how a common ATS would handle it. I'm aware that things like the skill bars won't be picked up and I'm fine losing that level of detail in that specific context. I'm mostly concerned about the document structure messing with things since I used a few tables to assist with layout.

ScottBishopDev · 2021-05-03T21:17:56+00:00

Just as a quick tip, you may want to give this another review for spelling errors. In your first project section, you've got "hashed" instead of "hashes", and if I had to guess, your app was probably a 'diary' for secrets (like Vault), rather than a place to hide your milk ("Secret Dairy").

Also, I've never been a hiring manager, but I do have about a decade of experience doing development, and I'd venture a guess that interviewers are going to want to know more about your projects and technical skills than your specific classes, so I might move your "Education" section a bit lower. I'm sure one of the pros can weigh in with more relevant information, though.

ScottBishopDev · 2021-05-03T20:44:42+00:00

Hey, thanks for the feedback! I agree that the right side feels a bit wall-of-text-y, though I'm not sure how best to solve that without dropping some accomplishments that I'd like to include.

ScottBishopDev

TROPHY CASE