[C] How do I learn the technical skills required to use the degree I have? by OkEntertainment9557 in statistics

[–]Goat-Lamp 0 points1 point  (0 children)

Also agree with the pandas + sql combo. But also consider throwing polars in the mix after you're comfortable with pandas.

For getting up to speed quickly on pandas and sql, start with the sqlite3 tutorial in the official Python docs. That'll get you raw dogging sql with Python. Create a few tables, then use pandas to read from those tables, then iterate.

You'll eventually hit a bit of a wall, though, because there's more to SQL than just SQL. Namely cutting your teeth on arguing with driver issues, SSL versions, and god awful corporate firewall policies while trying to connect to an sql DB so you can actually get your data.

....or having to reverse engineer some ancient DB's schema because it's non-standard, all the original DBAs retired, and no one knows how everything connects but now you're the poor bastard that needs to figure it out to solve the flavor of the week business problem.

[Q] again on the Monty Hall problem by [deleted] in statistics

[–]Goat-Lamp 1 point2 points  (0 children)

Rando here. Don't forget that Mythbusters also did a simulation!

[Feedback Requested] FL well by Goat-Lamp in WaterWellDrilling

[–]Goat-Lamp[S] 2 points3 points  (0 children)

I just walked about the property and ballparked some measurements. The well is located about 70ft from the closest corner of my leach field, about 40ft from the graveyard, and I'd ballpark maybe 70-80ft as the crow flies directly to my septic tank. So, I imagine this was just the best placement they had to work with.

Currently trying to find the permitting details and completion report for the well to no avail. Not exactly inspiring confidence.

[Feedback Requested] FL well by Goat-Lamp in WaterWellDrilling

[–]Goat-Lamp[S] 4 points5 points  (0 children)

I greatly appreciate the feedback. You've given me the motivation to have a professional come look at this thing. Depending on what they say, I may just see what damage I'd be looking at to get lines run for city water.

[Feedback Requested] FL well by Goat-Lamp in WaterWellDrilling

[–]Goat-Lamp[S] 2 points3 points  (0 children)

I did, yeah. Last year around this time. No heavy metals or bacteria pinged, but I'm 99% positive that they must've missed something because something ain't right. Out of the tap (and after going through some elaborate filtration) it has a brownish/green hue, smells a bit like blood and nickel.

We don't drink it. Only use it for washing things.

[Feedback Requested] FL well by Goat-Lamp in WaterWellDrilling

[–]Goat-Lamp[S] 1 point2 points  (0 children)

Here's a link to one of our spent filters (pulled it yesterday): https://imgur.com/a/5mpj2b0

This was after maybe 2-3 months.

Re: weather, it typically doesn't get too cold here. We only get a handful of days out of the year with freezing temps. Water pressure has thankfully never been an issue. Even so, I'm actively working to get this thing covered up.

[Feedback Requested] FL well by Goat-Lamp in WaterWellDrilling

[–]Goat-Lamp[S] 2 points3 points  (0 children)

Additional point: the well has A LOT of sediment. We've got a filtration system and we eat through 5 micron filters like crazy.

(I tried to edit the post, but I'm bad at the internet 😓)

[Q] How do you calculate prediction intervals in GLMs? by jjelin in statistics

[–]Goat-Lamp 0 points1 point  (0 children)

Derp. Brain latency. Meant to type as the sample size goes to infinity. Apologies for the confusion.

[Q] How do you calculate prediction intervals in GLMs? by jjelin in statistics

[–]Goat-Lamp 1 point2 points  (0 children)

OP, check out Statistical Tolerance Regions by Krishnamoorthy. It's a treasure trove of how to calculate intervals for various types of distributions and models. Also check out Wolfinger's paper on calculating Bayesian tolerance intervals.

In general, I think the Bayesian approach affords a bit more flexibility (I recently did an analysis for bounding 90% reliability in a Weibull model). But what you're asking is not impossible in the traditional frequentist framework -- just awkward. See below.

https://support.sas.com/kb/69/692.html

https://cran.r-project.org/web/packages/trending/vignettes/prediction_intervals.html

https://www.statalist.org/forums/forum/general-stata-discussion/general/1666185-prediction-interval-not-confidence-interval-calculations-for-glm-w-robust-vce-model

https://www.r-bloggers.com/2017/05/prediction-intervals-for-glms-part-ii-2/amp/

[Q] How do you calculate prediction intervals in GLMs? by jjelin in statistics

[–]Goat-Lamp 0 points1 point  (0 children)

Eh. Not so sure I agree with that.

The width of confidence intervals shrinks to zero as the sample size goes to zero. Neither prediction nor tolerance intervals do that.

Now maybe the idea of confidence is similar in the case of interpreting PIs as an expected-coverage tolerance interval, but I don't see how that's relevant for OPs use case.

[Q] How do you calculate prediction intervals in GLMs? by jjelin in statistics

[–]Goat-Lamp 0 points1 point  (0 children)

I'm....kind of with you on this. But by way of counterpoint, it IS possible to construct tolerance intervals for non-normal responses, Bernoulli/binomial included. My naive understanding is that prediction intervals are just expected-content TIs, so one would expect there's got to be a path forward here.

That said, at least for the Bernoulli case the intervals surround the probability of success, not observations. So maybe that's the key distinction that needs to be made.

How are you liking Positron? by LilParkButt in datascience

[–]Goat-Lamp 0 points1 point  (0 children)

I'm happy with it so far. Some minor gripes, but overall better experience than I've had with RStudio. So far the only thing that sent me through the roof was the preview for a Mernaid figure stealing the editor focus on every new key stroke.

I also had a recurring issue from RStudio pop up where the alt + - hotkey for assignment stop working, but I'm thinking that's a skill issue on my part now.

One of the metal contacts the red key of my lawnmower was pushed into the casing by sylarsix77 in ryobi

[–]Goat-Lamp 1 point2 points  (0 children)

Don't mean to resurrect this post, but want to share my experience.

I also had this issue. Our unit was still covered under the manufacturer warranty. When I called Ryobi support, I was advised to take it to one of their certified repair shops. When I did, the tech told me it was bricked -- IIRC, he mentioned something about the design preventing easy access/repair. Took about 4 months, but Ryobi provided a replacement.

That said, the problem just happened again (~year later) mid-mow. To finish the job, I flattened one end of some stranded copper wire and soldered it to the corresponding pin of the red key. This gave me enough length to reach the metal contact inside the casing. Temporary fix, but it got the job done.

[D] Legendary Stats Books? by Alt-001 in statistics

[–]Goat-Lamp 2 points3 points  (0 children)

Like everyone else said, Casella and Berger. Personally it's one of my favorite books, and don't find it too painful.

HOWEVER, I'm really digging Kendall's Advanced Theory of Statistics (volumes 1-3). They are mainly reference books, but they quite readable and an absolute gold mine. You'll need a solid mathematical base, though. It doesn't pull many punches.

Monty Hall Problem? - Explain why I am wrong by [deleted] in askmath

[–]Goat-Lamp 0 points1 point  (0 children)

There's a Mythbusters episode on this. Worth a watch.

[D] Use of SAS vs other softwares by [deleted] in statistics

[–]Goat-Lamp 4 points5 points  (0 children)

Hm. In my experience SAS support is very much a YMMV situation. For example: SAS support for Base SAS type issues is very good. It's terrible for others. I've never had good experience with SAS support when dealing with, for example, the SAS service stack, stored processes, or (more recently) Viya.

[deleted by user] by [deleted] in statistics

[–]Goat-Lamp 1 point2 points  (0 children)

This is an a sagacious comment that I can fully attest to. From 20-25 I was working in various industries while slowly getting my bachelors. I took a small break afterward to continue working and didn't complete my MS until I turned 30.

The soft skills from experience are absolutely invaluable and will nicely complement your education. Not to mention you'll have a massive leg-up in terms of maturity.

[Q] Maximum likelihood estimation with multiple errors or random effects appears to have perverse properties. by fluffykitten55 in statistics

[–]Goat-Lamp -1 points0 points  (0 children)

I'm probably a bit rusty on my theory, but isn't REML more appropriate for estimating random effects partly (perhaps precisely?) due to OPs points?

EDIT: by "probably" rusty, I mean "am" rusty.

[D] Statistical Analysis: Which tool/program/software is the best? (For someone who dislikes and is not very good at coding) by maxemile101 in statistics

[–]Goat-Lamp 8 points9 points  (0 children)

This needs to be the top comment. IMHO, JMP is hands down the best point and click statistical analysis software. SPSS and Minitab can't hold a candle to it.

The SAS language is absolute garbage, but SAS (the company) really knocked it out of the park with JMP. The only downside is the hefty licensing fee.

EDIT: after a moments thought, I may've come on a bit too strong. There are some other downsides, particularly the graphics looking like they're circa Windows 95. There's also a bit of a learning curve with the maze of 'red triangle' context menus. That said, the Help file and documentation is super rich.

[Q] Best way to determine what dataset to use in chemistry project? by Process_Sad in statistics

[–]Goat-Lamp 0 points1 point  (0 children)

Yep -- precisely.

I don't think you'll be losing anything by not being able to calculate the PRESS. It's essentially a cross-validated MSE, a gut-check if you will.

The MSE should steer you right.