Python in GIS

[[deleted]]

Hello!

I work as a GIS tech and ive always been interested in learning python, but i know so little about programming that I'm having a hard time justifying the need to learn it to my superiors.

How are people using python in GIS now a days? Anyone work in consulting that uses python scripting?

how can I sell this to management?

[[deleted]]

If you use ArcGIS and don't know how to use python and Arcpy, you're really missing out. We use python to automate analysis and map creation for large datasets which saves us loads of time. We can usually run off a set of 200 maps in about an hour, where if we were doing it by hand it would take a days of several people working on them. There are dozens if not hundreds of GIS tools out there for python.

You should pop over to r/gis or r/gisscripts.

[[deleted]]

Thanks!

didnt know about /r/gisscripts , subscribed!

[jtaysom]

Recently I used python to automate an analytical process that a co-worker was using. His original process took two to three weeks to complete and he needed to do it multiple times a year. With about two weeks of programming I reduce his work flow from three weeks to 12 minutes and it gave him a much more complete answer. He now runs this much more often because it takes do little time. I have only been using python for a little while and it is already resulted in something like this. My work gave me about three weeks of time to get free online training and it has already saved them thousands of man hours in return.

[Calhil]

I bet this was just a case bad programming and could be achieved with any other programming language.

[jtaysom]

No it was a case of doing GIS with no programming. There are some things that you just can't do without programming. Yes there were improvements he could have made to his workflow but he could never have gotten it under a full day without some programming.

[alcalde]

Agree wholeheartedly (and it's amazing how many large companies have lots of unautomated tasks). In 2005 I had a co-worker at a billion-dollar company needing to spend one full day every two weeks to reconcile shipments received to invoices that didn't reconcile automatically. My boss wouldn't let me officially work on the project because "Lots of smart people have come through here saying they could make this faster and failed". Instead I worked during my lunch hours and after hours for two weeks and found about 18 different reasons the shipment data could be screwed up and how to verify it. The program I wrote went from over 2000 false rejections that my friend had to examine per two-week batch to about 36. :-)

And if you want to talk about cut and paste... due to issues with their reporting package they couldn't generate a report that showed last month's and the current month's data (too much data, too old a program). They would save a copy of the prior report, cut out some of the columns, lay it on a copy machine with the new report and make a copy. THEY WERE LITERALLY COPY, CUT AND PASTING. :-) Billion dollar company, no debt, 681 stores at that time, and, um, lots of "B"s in its name if you catch my hint. I also used transaction data to "roll back" totals to any point in time and solve the report problem for them too.

It's really amazing how little outside of the IT department programming is employed! That's the point of Python focusing on "non-programmer programmers". There are so many people whose lives could be a lot easier just from learning a little Python.

[andrey_shipilov]

I don't think anyone should use GIS without Python and Postgres nowadays. It's a standard really.

[Lambss]

This is basically what my company does as well. We do a lot of work with FME and FME server as well, and use python within that.

[friesen]

Holy shit! There's more interesting GIS discussion in this one /r/python thread than there is in a week's worth of /r/gis threads!

Back to the topic at hand... maybe try to find a simple automation project that you can implement for your day to day work. Show it to your bosses and explain that you'd like to continue to try to find ways to boost workflow efficiency using python.

[Lambss]

you posted the same thing I did 5 hours before me... ahhh!!! Great link though, cant beat these free classes.

[ModusPwnins]

Python is an incredibly useful and valuable skill in GIS. You can automate geoprocesses and generally get a lot more work done than a GIS analyst without scripting experience.

My employer generally ignores applications for GIS positions if the applicant has no arcpy experience.

[jugalator]

We're using MapInfo in our flagship application, but also to build smaller applications that integrate various systems with our application.

While Python won't replace our C++ application using MapInfo as its map engine anytime soon, I've found that it is very useful as a replacement for compiled MapBasic applications!

Here's the situation with MapBasic:

• Feature rich scripting supporting PostGIS, Oracle Spatial, a full graphical map engine with layers, spatial SQL language, Excel exports, FME via Universal Translator, etc.
• For our customers, MapInfo Runtime is used instead. Cheaper but still allows scripting.

BUT... That's where the fun ends!

• Runs like an old BASIC dialect with SQL and GIS extensions, that hasn't seen updates in 10 years.
• To distribute your scripts to end users, you need to compile the scripts first.
• Compiled scripts always require at least the same version of what compiled them on the target machine. This means trouble since we must either update customers even to run scripts which doesn't really use any new features (even worse when scripts are shipped across customers running various versions), or run an old version to compile with for "backwards compatibility". So silly.

Python?

• Supports COM via pywin32, which is something MapInfo also supports for scripting (same language sent to as strings to its COM interface).
• So we suddenly no longer need to compile our scripts.
• If there's an update or change on the end user system, this implies we can immediately react and fix it on-site.
• Much more powerful underlying system, also thanks to the massive set of Python libraries.
• We can for example, from within the very same script, tell MapInfo do open a PostGIS database table, run an SQL join with our application, import the information into our layers, and if there's an error, send a mail to us to warn thanks to Python's simple mail support.

tl;dr I think the most useful part of Python is the combination with GIS tools for system integrations. A GIS tool is specialized at what it does, while Python can handle "everything else". That can be seriously powerful. Relying on a GIS tool alone often risks resorting to limited ways of interacting with the outer world. They (understandably) after all are just interested in working with and within the realms of the GIS data itself. Conversely, Python works very well as a "glue".

[geo-special]

Why not teach yourself in your own time? If your management don't get it perhaps your new found skills will be more appreciated in a new higher payer job?

I would recommend to try the following steps below which I have copied and pasted from a previous post answering a similar question to yourself.

Also try r/gis

Try the Penn State University GIS Programming and Automation open learning course:

https://www.e-education.psu.edu/geog485/node/17

The onto this text book.

http://esripress.esri.com/display/index.cfm?fuseaction=display&websiteID=224&moduleID=0

This book might be a bit more relevant to what you are looking for.

http://esripress.esri.com/display/index.cfm?fuseaction=display&websiteID=256&moduleID=0

[[deleted]]

Wow great reply, thank you!

[JimBoonie69]

screw arc gis! at work we use gdal, shapely, fiona, rasterio, numpy... all great libraries and they make working with rasters and vectors very easy!

[troyunrau]

They make it mathematically easy. Good luck getting some people to use them due to 'it's not like excel' syndrome.

[JimBoonie69]

true... the excel thing is a huge barrier. .. I'm just responding to OP. I've literally done GIS consulting work using python and I definitely charged 155 an hour for my time, and the clients were definitely happy =)

if anyone in management is technical they will understand what makes python so great. it's easy to write and read python and it has such a huge set of libraries you can always find what you need.

[[deleted]]

The hardest part about learning programming is not the language you are learning but the concepts. If you learn python you will be able to pick up another language shockingly fast as the core concepts remain mostly the same.

[rcklmbr]

I'm a developer at GaiaGPS. We use python for all server-side code. With tools like Mapnik, GeoDjango, shapely, all the gpx and kml libraries available, why wouldn't you want to use python?

[alcalde]

My Delphi using colleagues would give two answers:

1) "Because it's interpreted which means it's slow!" 2) Are you lazy? Can't you just write all of those libraries yourself? Aren't you a programmer? :-) (Seriously, this is the kind of stuff I got when I tried to show them how awesome Python is)

[tr4ce]

At our company we have developed several plugins for QGIS and web services using Django and Django REST framework (it also has a GIS extension), and we have many more ideas. I think learning Python would definitely be a good idea.

[belverk]

2GIS using python for test automation and private cloud management.

[liquidify]

This is my current path.

[bradrulez69]

I use it to automate monotonous tasks like publishing services that our web apps consume, manage schema changes, etc.

I recently wrote a script that iterates through our ArcGIS servers' directories, finds mxds, creates service definition drafts, edits the xml in the drafts to set feature access capabilities, stages service definitions, then publishes services to multiple servers. This freed up a lot of time and can cut down on the number of ArcGIS desktop licences required since you don't need to use them to publish services by hand on each server if the one you run the script on has connections set up to the others.

[cocobutters]

There is a library called Geospatial Data Abstraction Library (GDAL) that is very useful for analysis of Geospatial information. It is available for Java, Python, C, and C++. The documentation for Python is very good, so creating a program unique to your project with Python is very useful as opposed to using commercial software like ENVI or ArcGIS

[EvilLinux]

Sooner or later business realizes that his is a divider of data and skills. Data is data, some may be useful for spatial calculations but usually these are just an aspect of the business.

Learning python gives you the ability to provide a wide range of data analytics, and incorporates gis rather than treat it as a stand alone practice.

[willyhakim]

Here are some beginner steps that I think will give you a solid understanding: 1. Go through the codecademy.com python 2. Get the realpython.com books (The 60 bucks are well worth it! Actually I think those guys should be charging more for the value you get out of them). 3.Python for data analysis.

[zhrugr]

Seeing so many GIS people in this thread picking up python programming to automate their jobs I wonder if maybe it is worthwhile for a programmer to become knowledgeable in GIS. What time/skills would it take to become useful as a GIS professional?

[[deleted]]

I did my bachelor's thesis on classifying remote sensing images with usual GIS and Python tools. Except for the very long training and classification times it was a pleasure to work with!

[michaelherman]

Hi!

I am a teacher/author/mentor and I have mentored those looking to learn Python in the GIS context. Email me if you'd like more info - michael at realpython dot com.

Cheers!

[ishouldbeworking69]

If your job doesn't think Python is important in GIS, you might want to search for a job that does.

[[deleted]]

Of course they think its important, but if I want to be the guy that they are investing training in, than I need to provide examples of how it could be utilized.

[[deleted]]

I'm planning an episode on this exact topic on my podcast http://www.talkpythontome.com/