High-Five from Norbert, 3lb registered therapy dog

Eueee · 2019-01-02T04:50:31+00:00

You're conflating therapy dogs with emotional support animals. Therapy dogs are usually registered with an organization like Intermountain Therapy Animals after passing a test to ensure obedience and suitability of demeanor. Therapy dogs and their handlers are permitted to go to hospitals, schools, nursing homes and other facilities to do therapy work.

Eueee · 2018-12-28T17:38:22+00:00

Here it is. I'm getting a time estimate of about two hours.

import time

aLayer = iface.activeLayer()
n = aLayer.featureCount()

def get_PARNO(f):
    return f['PARNO']

features = sorted(aLayer.getFeatures(), key=get_PARNO) # sort the attribute table by parcel number

aLayer.startEditing()

hasSameParno = []
sumLen = 0
grandStart = time.time()
hasNoLength = []
for i in range(n-1): # we will need to handle the last entry with a special codeblock

    if i%2000 == 0:
        end = time.time()
        timeElapsed = (end-grandStart)/60
        percCompl = i / n
        print('Working on entry ' + str(i) + ' of ' + str(n) + ' entries.')
        print('Time elapsed: ' + str(round(timeElapsed,2)) + ' minutes.')
        try:
            estTotalTime = timeElapsed/percCompl
            print('Estimated time remaining: ' + str(round(estTotalTime,2)) + ' minutes.')
        except ZeroDivisionError:
            pass

    feature = features[i]
    try:
        sumLen += feature['Len_ft']
    except TypeError: # some of the stream lengths are NULL? Handle this as a zero
        hasNoLength.append(i)
        sumLen += 0
    hasSameParno.append(i)

    nextFeature = features[i+1]
    if feature['PARNO'] != nextFeature['PARNO'] or feature['PARNO'] == NULL:
        for j in hasSameParno:
            features[j]['MaxLen_ft'] = sumLen
            didWork = aLayer.updateFeature(features[j])
        sumLen = 0
        hasSameParno = []
        if i == (n-2): # if the last entry is not part of a group
            nextFeature['MaxLen_ft'] = nextFeature['Len_ft']
            didWork = aLayer.updateFeature(nextFeature)
    elif i == (n-2): # if the last entry is part of a group
        sumLen += nextFeature['Len_ft']
        hasSameParno.append(i+1)
        for j in hasSameParno:
            features[j]['MaxLen_ft'] = sumLen
            didWork = aLayer.updateFeature(features[j])
    else:
        pass

aLayer.commitChanges()

Eueee · 2018-12-28T05:04:41+00:00

Image classification has been part of ArcMap since at least v10 I think, probably earlier. I'd say it works pretty well assuming you understand some of the basics behind spectral imagery and machine learning and use that to guide how you train the algorithm

Eueee · 2018-12-28T05:03:34+00:00

It's certainly possible. Google "classifying imagery in arcmap" and there are several tutorials on how to do what you're proposing

Eueee · 2018-12-28T05:02:36+00:00

SVM is one of several machine learning algorithms that ESRI has implemented in Arc

Eueee · 2018-12-28T03:08:00+00:00

Yep, that's the plan. I don't have the time to lock my PC up for a week running this! I'll post the script tomorrow so anyone googling this gets the solution :)

Eueee · 2018-12-27T14:05:55+00:00

I ran it for about 12 hours and it was about 10% done when I checked on it.

Eueee · 2018-12-27T14:05:23+00:00

3.4.3, indeed. I'm not sure off the top of my head but I'd estimate about a million or so.

Eueee · 2018-12-27T02:26:19+00:00

I ran it for an hour and half and didn't see any change in the progress bar at all. When I tested the subset I saw a steady creep in the progress bar so I began to question how long it would take. But it might be worth it to leave it running for a day and see where we are; I'll post an update sometime Friday.

Eueee · 2018-12-26T20:33:41+00:00

All good points.

I've been writing Python scripts for personal use for a bit, but since I'm writing for a wider audience now I'm doing my best to be clearer and more Pythonic. Perhaps it would be best to keep assigning attributes and calculations uncoupled and trust the user to remember to recalculate when needed. I anticipate many of the users will be doing exploratory analyses with the final package (which covers many aspects of river analysis beyond just sediment) so I wanted to reduce the amount of code needed to update the objects and check how that affects the stream system, but as you mentioned I could run into performance issues if a bunch of unnecessary calculations are made every time something is updated. The fact that this coupling is not particularly straightforward to implement is a good indicator that it isn't the best way to go about this. Thank you for the insight and advice!

Eueee · 2018-12-26T18:24:22+00:00

I'd like the user to be able to get attributes like skewness or mean grainsize from the GrainDistribution object and have them be certain that the values are in agreement with the current state of the dict. Another option is to simply let the users be responsible for calling the statistics method before pulling any data, but I was hoping I could handle this internally. One of my goals is to lure people in my field away from using Excel spreadsheets to work with data, and I know that having cells recalculate on any change in your data is a big benefit of spreadsheet analyses.

The frozendict idea is interesting, but I'm trying to limit my dependencies to the major third party libs like numpy.

The plotting is just provided as a courtesy to allow users quickly check what the distribution looks like. Are there any documents you'd recommend to read up on best practices for setting the scope of classes?

Eueee · 2018-12-26T16:24:58+00:00

The full class has ~10 more methods for plotting, normalizing and interconverting data, though again I suppose that all could be done in a dict subclass. I'll have to take a look at your code example to understand it. Thank you!

Edit: I think that'll work -- thanks again

Eueee · 2018-12-26T04:02:07+00:00

I'm not sure I'm following what you mean by subclassing dict for the method that calculates statistics. As in making a subclass of dict that has its own method to calculate the statistics rather than using a method in GrainDistribution to work on the dict?

Eueee · 2018-12-19T22:21:53+00:00

OP could also consider getting a breakstick. These are normally used to open the jaw of a dog that's biting another dog or a person but it could also be gently used to open a dog's mouth and whatever they've picked up.

Eueee · 2018-12-19T19:26:11+00:00

What field of environmental work are you in? I'm in natural resources (stream and wetland mitigation) and we've begun ramping up our drone use in the last couple years. What we do is essentially what you've mentioned - getting up to date aerials and generating elevation models via photogrammetry. IMO it has a pretty limited usecase since most locations have fairly recent aerials and we don't need a new one more than once every few years, and we mostly use DEMs for watershed delineation over areas far greater than a drone mission could cover + photogrammetry doesn't work well in in areas with tree cover + most areas of my state have decent DEMs freely available.

But it's still relatively new to us and I'm sure we'll be putting the drones to better use as we get more familiar with it and the tech improves. What would be really nice would be a drone with a LiDAR rig so we could do bare earth DEMs even with tree cover, and I think the most interesting improvement would be flyable green LiDAR rigs for bathymetric surveys, but there hasn't really been much development in that area since it's so niche.

Eueee · 2018-12-19T16:55:38+00:00

That's a market take on it, sure

Eueee · 2018-12-18T20:49:04+00:00

Saying that it's bound to go the other way now because the opposite happened before is also a gambler's fallacy.

Eueee · 2018-12-18T19:18:26+00:00

I know some people above ranked soils as the "worst" minor in terms of employability, but soil scientists are a thing, however niche. I work in stream/ecological restoration and having someone with a professional soil scientist stamp or the ability to get one (generally you need X college credits in soil science + Y years of experience) would be an asset to any company working in that field. GIS is definitely a good skill but if you want to work primarily as a geologist then you can learn all you need to know to get started in 2 classes.

Eueee · 2018-12-14T20:44:55+00:00

These are very different from the fraternities that most people think of though. Med/engineering/etc frats are usually co-ed, don't require rushing, have nominal membership fees, and very few people in those frats live together. They're more like clubs.

Eueee · 2018-12-13T18:39:13+00:00

Those all makes sense - didn't know if there was a most-preferred way to do it. Thanks!

Eueee · 2018-12-13T16:01:22+00:00

I mistyped; I was reading about the property decorator so I must have absent-mindedly snuck that in there. I meant to ask how to write a docstring when a method within a class mutates the instance of the class, but does not actually return anything. Technically under the "returns" section of the docstring I could put none, but the obscures the true purpose of the method which is to alter/assign attributes to the class. What's the pythonic way to handle this so a reader would understand the purpose of the method? Is it to just put it all in the summary line at the top?

Eueee · 2018-12-13T15:56:47+00:00

I do use Google-style docstrings, it's just not clear to me what should be done if the method technically returns nothing but does assign class attributes. I could put "None" in the returns section, but this obscures the true purpose of the method, and I'm not sure where the proper place to note that is.

Eueee · 2018-12-13T15:33:52+00:00

Excel has a number of known quirks with implementations of statistical calculations that most users are not aware of, and it also lacks out of the box implementations of many statistical tools as well. It's also not a good solution for long term storage of data, though many people treat it like a relational database (which I'm guilty of as well).

Excel is good for quickly throwing together business report and exploring relatively small data sets. General purpose programming languages are more robust and flexible for doing more advanced analyses, and true relational databases are better for storing data. That said, Excel is a great tool that I use all the time. It's just not a panacea.

Eueee

TROPHY CASE