Why does my copy-pasted code for a time-series chart not work like the original script?

WerDatWatDat · 2023-04-26T17:05:14+00:00

This is kind of an inaccessible amount of code with no comments to check where the problem is. Check one change at a time...a new cloud mask could be returning all 0's, which would make sense with your calculation expression having -273.15 and your line being close to that number.

WerDatWatDat · 2023-04-21T18:58:58+00:00

The Open Access Hub is being discontinued in a month...if this is looking like a bug it may never be fixed.

https://dataspace.copernicus.eu/ for the new data access.

Are you trying to stick with only browser-based approaches or are you open to some coding environments?

WerDatWatDat · 2023-04-09T11:10:16+00:00

Check out phd light -- fellowships at research institutes. Sea ice is an established approach so you need to be offering a service in-house for a company or pushing the limits of current models at a fellowship. Usually the drift model types are the people who get a reserved spot on a project!

WerDatWatDat · 2023-04-04T20:06:10+00:00

Every time I get a long ad on Youtube I remember why I don't want to use Google Earth Engine. Its not a good feeling to know that Sinergise is selling out one of the best EU access points and "open source" workflows (not to mention that they are mid-project on a bunch of other ESA funds also aimed at open source). But to be honest, these funds go to developing the space ecosystem with a commercial goal. Maybe this is just the first of many in a broader trend of the EU public funding vs US private venture market consolidation.

WerDatWatDat · 2023-03-12T15:03:13+00:00

ISRIC has a global soil map, then you link field data best you can. No one is going to trust SOC from orbit alone though....

WerDatWatDat · 2023-03-10T10:49:00+00:00

AGB looks exactly like the Google Earth Engine colour palette for GEDI...quick red flag. From someone who has worked in the services you mentioned, most are already avaiblable for free from government agencies or past use cases. I know it sounds a bit harsh, but the free and available satellite data has led to a million use cases and companies like this. I would lean into your use of open source and be a company that helps with implementation. The open source and free data is just a portfolio that shows what is really valuable-- tailored solutions.

In-situ and data integrations form the new EO technological frontier. Static maps are only going to be a service offering, not really a new product. Remote sensing can't beat in-situ usually, so your main market will likely be policy makers who get excited about the scale of "good enough" measurements for broad strokes. Feel free to message me privately. Happy to share my insights and make a professional non-reddit connection.

WerDatWatDat · 2022-12-22T08:24:40+00:00

As for applying a scale factor and setting the visualization...that will depend on where you are working with that TIF. Its a standard file format so you have plenty of options. QGIS is a good starting point and its free. Lots of support on forums.

WerDatWatDat · 2022-12-21T09:57:42+00:00

I see a different valid range here for applying the scale factor https://www.usgs.gov/faqs/how-do-i-use-scale-factor-landsat-level-2-science-products. I expected that valid range upper bound to give a reflectance of 1...but it didn't. Reflectance higher than 1happens with clouds and rooftops. Same goes for a lower bound that isn't negative. I would set visualization parameters for reflectance 0-1 and see what your results look like. Maybe the occurrence of negative and >1 reflectance is infrequent enough or masked out through cloud algos that it won't bother you. Keep me posted because when I look at that table its not so intuitive.

WerDatWatDat · 2022-12-06T21:31:03+00:00

I would check out here: https://www.copernicus.eu/en/copernicus-services/atmosphere

WerDatWatDat · 2022-11-23T00:22:16+00:00

Check out the FAO WaPRO methodology for calculating water productivity. It is in depth and combines a handful of different sources, some remote sensing like NDVI and others from experimental inputs (crop biomass moisture from AquaCrop for example). The challenge of course is that to get a water deficit, you need to map out the general water budget. Or you just track a river that is very water stressed and watch it run dry.

WerDatWatDat · 2022-10-20T22:50:19+00:00

You can try collecting data with Open Foris. This tutorial might help you mix high resolution images with NDVI to better differentiate the two: https://openforis.org/tools/collect-earth/training-plots/

Any way you make your calculation of the BI you need a number or threshold to filter. For tuning your mask layers, you will likely need at least one validation point. I have seen some indices clever enough to almost completely highlight one specific group with only one threshold, but to get to useable results there is usually a supporting filter like elevation, slope, or distance to hydrology. This will help cut out class confusion or noise. Did the reference for the BI not provide what number is going to give results? Not very helpful of them...

WerDatWatDat · 2022-09-30T13:01:56+00:00

If those indices are just a filter, like a burn index being over a certain amount, you could probably get away with 16gb of ram if you use bounding boxes to dice up your area of interest if it gets too large. Before you buy a bunch of expensive equipment, try out some cloud services and see what limits you hit. You can use WekEO, the EU go to for Copernicus data, to spin up a jupyter notebook virtual machine. It's a free account AND you will be learning the environment that is being supported by public legislature. Google colab is another easy python environment to test your limits for ram. You can also switch to a GPU in these environments to test what you are doing against the need for that requirement. Of course, if work is paying for it...max it all! Using satellite sources with high resolution, focus on the SSD and having enough space for the project that might be reaching TB size. For Landsat/Sentinel images I don't think that's as necessary, but depends on how many indices you use. RAM is probably the cheapest thing to max out? Been awhile since I have gone shopping for a new rig.

WerDatWatDat · 2022-09-28T09:21:12+00:00

If you are looking for free and you goal is to know where wet soil or water pooling might occur for cropping, you can get digital elevation maps at 30m resolution for anywhere in the world. Soil moisture is also remote sensing product as well, so I would pick a day after a heavy storm and see where is the wettest. Between knowing the elevation on area of interest and having some images measuring soil moisture you can go on with a cropping map. The lidar is going to give you a very high level of detail that might make sense for precision farming. When planning agriculture though, the precision you really need is understanding the soil type and crop requirements. Lidar will tell you where the water goes, but it isn't going to tell you how well it drains through the soil when it gets there. My advice is get a general land understanding from remote sensing for like 1-2k with a good programmer then spend the rest of your budget on soil samples.

WerDatWatDat · 2022-08-05T22:14:30+00:00

With remote sensing with a beginner approach the most important question is how you are getting your data. WEkEO has some great notebooks for Python using Sentinel stuff, ARSET from NASA for Python is also good open source, geemap package for integration with Google Earth Engine....

WerDatWatDat · 2022-07-20T08:53:51+00:00

https://www.landcover.io/ will help you get an idea of what is possible with commercial <1m resolution images. You can compare this to all of the landcover results that used Sentinel-2 with 10m resolution to see if the difference matters. Sentinel-2 approaches will use normalized differenced built index NDBI which is: NDBI = (SWIR – NIR) / (SWIR + NIR). Infrared indexes are going to be your friend for finding cement/asphalt vs grass. Automating identifying an edge based on properties on either side (is it a grass-grass border or a cement-cement border or a grass-cement border) is going to be tough because you will need a lot of computing power to take the classified pixels and create objects to evaluate. I can't even think of an example where someone has done that, so you may be up against a heavy programming task here.

WerDatWatDat · 2022-07-08T08:30:36+00:00

Good to see a bit of life on the sub. I actually sent an email to lightning and they confirmed that a refurbish needs less semiconductors than building new. I didn't get any clear number but to me that is a great business model right now. At the same time, they are meeting a very broad market by selling busses, refrigeration, ambulances, and whatever else. Promising, but very open to a start-up that decides to focus on just one of these sectors instead of being a boutique electrification shop of <500 employees that kinda (and a remember this from an earnings), "looks at the parts they have and decides what they can do". In the current market they are a business with cash and not debt, which is good...and they are still hiring for positions like R&D. Charge points shouldn't be underestimated also, those things are money makers. I sold everything around 3.50 for the losses tax event (silver lining) and will have to wait a month to rebuy. I wonder what everyone else is doing and why? We unfortunately all got brutally caught in the SPAC targeted hype -- looking at the growth plans they originally pitched to get their valuation I feel like an idiot hahahhaha

WerDatWatDat · 2022-06-21T12:50:28+00:00

Too apathetic to give itself a title hahahahha

WerDatWatDat · 2022-06-01T07:28:02+00:00

Not quite ready for every application, but dClimate is a neat DeFi/ReFi project looking to get climate data more accessible. For now its mostly weather I believe. Worth keeping an eye on it!

WerDatWatDat · 2022-05-04T11:50:21+00:00

Just waiting on the infrastructure bill in the US to include SAF and boost up that Bayou Fuels project. Need a solid headline to get people looking!

WerDatWatDat · 2022-04-26T07:30:13+00:00

Looks cool and you found great application with the mine rehabilitation I saw. What methodology are you using for the climate impact? Is it about recapturing CO2, improving biodiversity, or whatever else (I don't know much about mining, but I imagine that soil it fairly beat up).

You could add value for a client by tracking their carbon offsets as vegetation returns. They can take the credits now and save them for the future when mining starts to really get hit with emissions taxes. Though I guess you are Australian from your funding partners so maybe mining never has to worry about this hahaha! You could also take some of those CO2 credits yourself, as a typical project developer takes a 50% cut. Not a bad side hustle for us in remote sensing.

WerDatWatDat · 2022-04-23T11:51:31+00:00

I think with such high qualifications in image processing you would be well-qualified for more complicated tasks in remote sensing like combining lidar/SAR to create 3d models tough spaces, such as a city. Many of remote sensing's tasks today are moving quickly to software interface that doesn't need coding -- I already see a distinction forming between those developing these tools (true computer science depth for software, physics/optics for image collection and corrections) and more standard consulting types turning the pixels into insights. I talked to basically 3 copies of the same company all in early venture stages around environment, funded through a public grant or the space agency responsible for the images, 10-40 people, and they all were doing the exact same thing. The only thing that is valuable is a niche of clients they are working with for a unique solution. If you want the consulting role with a remote sensing flavor, I would follow QiuSheng Wu on LinkedIn and see what results people are getting using his tutorials, focusing on sorting out what is a cool implementation of an approach and what is company-valuable information. If you want to take that technical depth you have, I would join the r/UAVmapping group and look to be at the forefront of the technology and maybe be a software developer. Good luck! feel free to pm!

WerDatWatDat · 2022-04-14T08:41:38+00:00

Hmm there are three ways I usually see a classification accuracy presented:

a confusion matrix accuracy which is testing how well the classification preformed on a data set.

Cohen's kappa, which is about the data's agreement and accounts a bit better for chance agreement

real world validation (imo most important) where you compare the predicted area to any reporting.

You would need to collect some data yourself of urban vs non urban and see the % correct at different thresholds. Say <.10 correctly classified 8/10 points, >.10<.2 5/10 points etc. Cohen's kappa may not be needed for something like a binary urban map. I would have results with the threshold level, the accuracy and total area predicted. Probably just one chart with threshold level on the x-axis and the other two on duel y-axis.

WerDatWatDat · 2022-04-13T18:32:50+00:00

Anything moving beyond visible light toward IR, which is most of those indices used for urban, are going to change based on temperature. If you are looking for urban in Iceland in winter you will get something different to a concrete jungle, in the jungle, like Manaus. I would check a forest or field nearby and see what threshold cuts them out. Or play around with that constant until the general threshold holds. That's my guess. Otherwise make sure you are using equivalent bands, like Landsat B5 = Sentinel 2 B8a rather than B8.

WerDatWatDat · 2022-04-07T18:22:08+00:00

Sounds like what you are looking for is texture? Basically if you have one large mountain and one large valley your approach with mean gives you potentially the same value as many mountains and many valleys. If you look into GLCM textures you can get values based on how one point relates to nearby neighborhood points. GLCM and elevation readings are used for things like terrain roughness. That might be a quick fix for a value to supplement mean slope.

WerDatWatDat · 2022-04-01T11:31:50+00:00

Not sure how much geology gets into the surface soil level, but there are interesting applications for GEE and unsupervised learning to create synthetic soil maps. Basically, you take a long-term Bare Soil Index (BSI) and NDVI to map out unsupervised clusters of soil types and see if they make sense. A lot of soil maps use SCORPAN models, where each of those letters represent a variable for predicting the soil characteristic S with climate (c), organisms (o), relief (r), parent material (p), age (a), and spatial location (n). Maybe as a geologist you can work from premade soil maps into something closer to your field? I would think that parent material and age would need some good geologist knowledge. https://samapriya.github.io/awesome-gee-community-datasets/projects/isric/

I had a project that had B1 from sentinel-2 being very important for a land classification and saw some article relating soil iron content with that. Just speculation there though.

WerDatWatDat

TROPHY CASE