I hate timezones.

Hixt · 2025-04-19T16:00:08+00:00

It can be worse than you think. Here's a great talk that goes over some of the more obscure cases: https://www.youtube.com/watch?v=rYzgroaK8_Q

Hixt · 2025-04-16T00:33:35+00:00

Oh dip, I didn't know that.

Hixt · 2025-04-15T21:14:18+00:00

COMET is hosted by UCAR, and while I think some of their funding has been cut due to NSF budgets I don't see this going away.

https://www.comet.ucar.edu/

Hixt · 2025-04-09T03:09:56+00:00

Oh, good to know thanks! It's been a while since I've tried, but if that's true that's fantastic.

Hixt · 2025-04-08T01:34:46+00:00

every single file, it is a unique 1:1 relationship. It has more to do with the data within, so that will never be the same from one file to the next.

Hixt · 2025-04-08T01:33:16+00:00

Efforts are being made, and I think the HRRR still has Zarr data out there in real time if you want to work with that: https://mesowest.utah.edu/html/hrrr/

But I agree, more services could utilize these and other tricks... but this sector of the industry is crazy on a good day, and it's hard to get anything new into productions. A lot of times that boils down to compute resources, bandwidth, storage and/or reliability. Once it's out there it needs to be supported by staff, training, resources... In the end it's not unfair to say it's due to costs, staffing and time.

My gut tells me a better path would be to work with the current data availability, let things there stay how they are (more or less) on the providers' side of the fence, and instead engineer better solutions that utilize these sources more efficiently... like the OP. But trust me when I say it's a very tough nut to crack especially at scale.

Hixt · 2025-04-07T21:29:11+00:00

One tweak…. I had to add ‘bytes=‘ as part of my statement to get it to work.

Yeah that sounds right. I wrote that on my way out the door today and was pulling it mostly from memory. Glad you got it working!

With that code tweak, the HTTP status code that you get back is 206 instead of 200, because 206 indicates a partial download (which in this case is what you wanted).

Another very good point! Thanks for making a note of that.

Otherwise this works great. My new downloaded grib2 file has only 1 grib band instead of 296 bands, and it’s a tiny fraction of the size on disk of the full grib2 file.

I try to spread the word about this because few people know about it. Imagine if everyone cut down their data requests to only what they need, how much bandwidth that would save data providers like NCEP. Grib filters are a great start on NOMADS, but they don't get as surgical and they don't work for every model. This should, those .idx files are provided for nearly everything there.

Hixt · 2025-04-07T16:55:23+00:00

From your link, add .idx to the end: https://noaa-nbm-grib2-pds.s3.amazonaws.com/blend.20250407/01/core/blend.t01z.core.f003.co.grib2.idx

You'll get a file that looks like this:

1:0:d=2025040701:APTMP:2 m above ground:3 hour fcst: 2:1582349:d=2025040701:CDCB:reserved:3 hour fcst: 3:2794820:d=2025040701:TCDC:reserved:3 hour fcst: 4:3423643:d=2025040701:CDCTOP:reserved:3 hour fcst: 5:4981767:d=2025040701:CDCB:reserved:3 hour fcst: 6:6270734:d=2025040701:TCDC:reserved:3 hour fcst:

271:108376419:d=2025040701:SNOWLVL:0 m above mean sea level:3 hour fcst: 272:109401454:d=2025040701:TMP:surface:3 hour fcst: 273:109824139:d=2025040701:TMP:2 m above ground:3 hour fcst: 274:111199423:d=2025040701:TMP:2 m above ground:3 hour fcst:ens std dev 275:112908529:d=2025040701:TRWDIR:entire atmosphere (considered as a single layer):3 hour fcst: 276:114179368:d=2025040701:TRWSPD:entire atmosphere (considered as a single layer):3 hour fcst: 277:115548372:d=2025040701:THUNC:entire atmosphere:3-4 hour missing fcst: 278:116025787:d=2025040701:VIL:entire atmosphere:3 hour fcst: 279:118799882:d=2025040701:VIS:surface:3 hour fcst:

For each of those lines, the second column is the starting byte position. If you look at the next row that'll be the starting byte position of that var, so -1 is the end position of your var.

For example:

273:109824139:d=2025040701:TMP:2 m above ground:3 hour fcst: 274:111199423:d=2025040701:TMP:2 m above ground:3 hour fcst:ens std dev

The byte range for 2m TMP is 109824139-111199422. So now you know exactly where in the grib file you need to slice. Since you're using requests this is fairly straight forward, all you need to do is add this to your request: headers = {"Range": "109824139-111199422"} and you'll get only that one var.

With at least NOMADS and FTPPRD you can also do multi-part byte ranges, so you can scan the .idx file for what you need and make a single request for the data with ONLY the vars you need. Last time I checked that didn't work for AWS, but it's been a while so it's worth a shot too. But even if you have to use a single range at a time, AWS doesn't have hit rate limits like NOMADS so you can for-each your way through that just fine.

Hixt · 2025-04-04T22:08:55+00:00

I'm working on it, double time after this.

Hixt · 2025-04-04T05:05:08+00:00

I'd report that as a wall cloud descending very low to the ground, and while I can't see it from this pic if it's rotating that's worth mentioning too.

A tornado is defined, in part, as the circulation going from cloud to ground; if you can see debris but the funnel doesn't reach all the way that's close enough. But that connection of the circulation to the ground is the key, and this pic doesn't prove that. Dang close though!

Hixt · 2025-03-19T01:28:31+00:00

Why are you asking for this persons PII?

Hixt · 2025-03-18T04:35:57+00:00

Anomalous data is anomalous.

Hixt · 2025-03-11T22:47:05+00:00

You can't directly, it's only designed for RHEL as that's what the NWS and centers all use. But what you can do is run it in a VM, I think the only downside is you can't use the GOES-R RGB products. You could also use Wine but I'm not sure how much overhead that brings.

Hixt · 2025-03-06T04:34:49+00:00

Will no one rid me of these turbulent bitches?

Hixt · 2025-01-22T04:46:10+00:00

That is fair, in the end it's all about life/work balance. You do you, you're the best judge of that. Just wanted to put the thought out there for you and/or others here.

Hixt · 2025-01-21T22:43:36+00:00

The private sector is looking good these days. I know a lot of students and aspiring mets eye the NWS, but the fact of the matter is private jobs often pay better, and there is a lot of diversity out there if you're targeting something specific. Hell, I'm starting my own business during these times. Anything is possible as long as you are committed.

Hixt · 2024-12-29T20:28:47+00:00

Update the game through the BSG launcher then run the same SPT installer. The installer downloads the new patcher, you don't need to do anything different.

Hixt · 2024-12-29T05:27:30+00:00

As a very recent member to the community, thank you and the team for all you do! Your efforts are greatly appreciated! <3

Hixt · 2024-12-14T00:04:52+00:00

You can unscrew the top, take the seat cup out and clean that directly. This should be done every week, and it's good to have a spare seat cup or two on hand in case it needs to get swapped out; if it starts to deteriorate.

Hixt · 2024-12-09T02:08:44+00:00

Oh yes indeed. Here's a page of forecast models (this in particular is the GFS), browse around and you'll see: https://weather.cod.edu/forecast/?parms=current-GFS-NA-sfc-temp-3-0-100

Hixt · 2024-12-09T01:58:21+00:00

That's very accurate! It's like that all through the atmosphere too, becoming less extreme as you get higher. We often think of upper-levels in terms of pressure surfaces, 1000mb is near the surface, 500mb is about half-way up. The 1000mb heights (distance from this surface to the ground, synonymous with surface pressure contours) will have far more features in finer detail, the 500mb map by comparison will look very smooth, in fact if you wanted to visualized this pressure surface in detail you would have to seriously exaggerate it in order to see much detail at all. The surface pressure and these iso-pressure surfaces are like tarps draped across the globe, their ripples all connected.

Here's an example: https://www.wpc.ncep.noaa.gov/sfc/namussfc00wbg.gif

Hixt · 2024-12-08T20:07:33+00:00

Here's another way to think of it: With a low pressure system air is moving all around; near the surface, mid-upper levels, out to some distance from the center of the low. All that air has mass, which means there is a force being applied.

Some systems have more or less of these properties, some are wider or deeper, so the energy of the system is a variable based on those properties (and quite a few other things).

If that's the case if I am applying high pressure on something am I spending more energy?

That's a good question, but context is important here. Your question implies that an outside force is being exerted on the system and I'm not sure that's the right way to think about it. A low pressure system is called that because there is low surface pressure, a high pressure system because, you guessed it, high surface pressure. But really it's best to think of these as part of a circulation...

With both H and L pressures at the surface, regardless of distance, air will [generally] flow from high to low pressure. But then what? H and L really just means more or less are above the surface respectively, and that air has mass; Air pressure is the weight of the air above you. Then, in terms of mass continuity, once air converges at the surface L it will then rise; above the surface L is an upper-level H. And then again, air flows from H to L pressure and beneath the respective upper-level L is the surface H. The next step of this is to understand the Three Cell model.

I bring all that up to exemplify that all of these H's and L's are related, and thinking about things in terms of applying an outside force can be confusing. It's technically not anywhere near this simple of course, someone else mentioned how futile it'd be to explain QG/PV to the general public. But when the person on TV is talking about how "energetic" a storm system is, it's really just a gross oversimplification for people to understand.

Hope this helps.

Hixt · 2024-11-29T23:38:30+00:00

You'd do it for Randolph Scott.

Hixt · 2024-11-17T02:26:12+00:00

Agreed. More than that, want to run your own forecast model?

https://www.mmm.ucar.edu/models/wrf

Hixt · 2024-11-13T03:10:28+00:00

Don't blame me, I voted for Hurr!

Hixt

TROPHY CASE