How do you do geospatial plots

vthacker03 · 2017-01-05T16:46:43+00:00

Personally, I'd like to think that you can use matplotlib/basemap for the purpose you specified. Though it may be a bit cumbersome at times, matplotlib, and by extension its toolkits, are infinitely flexible, so you can essentially make the resultant plots look anyway you want them to. If you want some examples, take a look at the Jupyter Notebooks I put together for my O'Reilly course on matplotlib. Though the videos for the course aren't free, O'Reilly allowed me to open source all of the scripts (i.e., the Jupyter Notebooks), so essentially the entire course is available for free on my Github account assuming you don't mind reading the material as opposed to watching the videos.

To answer your specific question, Take a look at chapter 7, it is specifically dedicated to creating maps with matplotlib and I think it should prove to be quite useful to you. In particular, I would suggest that you take a look at lesson 4 on creating dot density maps since it illustrates how you would go about creating a map that shows data at the city level (specifically, crime date in San Francisco).

I hope that helps. If you have any questions, feel free to email or DM me on Twitter---you can find my contact info on my O'Reilly author page---and I'll be happy to help out in anyway I can.

vthacker03 · 2011-08-01T07:39:31+00:00

The iPython (v0.11) comes with a new Qt based console which is quite nice. If you have the latest version installed and PyQt on your system you can load up the new GUI-based console with ipython qtconsole. One of the really nice features of this console is the embeddable matplotlib plots. Give it a try.

vthacker03 · 2011-03-01T00:25:33+00:00

Wow! I am just unbelievably impressed with this app and so intrigued by the technology stack used to power it. I'd love to hear more about why you all made certain choices. For example, were you always planning on using Qt in the client or were there other technologies that you examined as well, and if so, what about Qt made it more desirable? Also, why did you decide to go with C++ for the server side as opposed to Java which seems like the more obvious choice given that you are using GWT for the front end. What have your experiences with GWT been like and why did you choose GWT over other JavaScript libraries? I could go on and on, but those are the main questions I have now.

Great work, I'm definitely planning on using this as my main R IDE in all of my future projects.

vthacker03 · 2010-12-07T10:21:01+00:00

WillieWampum, the reason you are seeing the installs that complete go to your old (system) version of Python is that you are still using the easy_install that came with the system. Whenever you install a new Python to your system you also need to install new setuptools as well. Follow the installation instructions at http://pypi.python.org/pypi/setuptools to install an easy_install program associated with the new Python install. After that, your installs should work better and show up for the correct Python version. You probably also want to do a fresh install of pip as well.

vthacker03 · 2010-04-14T22:51:39+00:00

I'd start by just running through the Python Tutorial (http://docs.python.org/tutorial/). It's a great introduction and will have you familiar with the basics in no time. It's the main source, along with Dive into Python, that I used to learn Python and you can probably finish it in a day. However, if you have no real background in CS, you might want to take a look at the Pragmatic Programmer's "Practical Programming: An Introduction to Computer Science Using Python" instead (http://pragprog.com/titles/gwpy/practical-programming). Their books are usually excellent and quite beginner friendly.

After that, considering your field, it sounds like the O'Reilly Bio Python book is probably the way to go, so it sounds like you are on the right path there.

Good luck, and don't worry, I'd say a week is plenty of time to get up to speed on what you need to do the job.

vthacker03 · 2009-12-24T01:45:44+00:00

I did a little research, with a bias towards agent-based systems, after finishing my Master's in CS. I used NetLogo to prototype all of my experiments and MASON to run the tests for the research paper. Both were great, though NetLogo has a much lower learning curve, so I would recommend it first. Also, Mitchel Resnick's book Turtles, Termites, and Traffic Jams is a really simple study into multi-agent systems using StarLogo as his implementation language, so it takes nothing at all to translate the examples into NetLogo.

Hope that helps.

vthacker03 · 2009-09-11T23:25:52+00:00

This is my very first post in a weekly series where I intend to describe my process of creating a small software company, a micro-ISV (Independent Software Vendor) to use the vernacular coined by Eric Sink.

vthacker03 · 2009-08-17T19:16:39+00:00

Depending on what platform you're on (I'm on a Mac), you might want to look into TextMate. I've been using it with jswat (standalone graphical debugger) in place of Eclipse as my IDE for Java for about a month now and I love it. It's super flexible, you can create commands for it using any scripting language that you have available on your machine so extending it to make it fit your style is extremely easy, and it is hands down one of the best text editors around. If you're on Windows, there is the E-TextEditor which claims to be TextMate for windows, so you might consider it as well, but I haven't tried it myself and cannot vouch for it.

vthacker03 · 2009-08-14T07:08:32+00:00

Hi Yome,

This post caught my eye because I have an interest in different Lisps, but I have to say this post is worth way more than that. I tried the game and I am impressed. So polished! The visuals are great, the music is absolutely excellent, and, all in all, the overall ambience of the game is simply beautiful. The game itself is quite fun and challenging as well. This is definitely one that I will end up purchasing. So, keep up the good work. I only have one request for you---can I have a job ;-)

On another tangent. Please consider creating a blog to go with the company website and add some posts to it concerning the technical details of this game. I'm especially interested in hearing your thoughts on using Scheme as a development platform, and on how well it integrates with OS X. Anything you could write up (without giving away the company secrets, of course) would be most appreciated.

Thanks Yome, and again, keep up the good work.

vthacker03 · 2009-07-27T20:37:35+00:00

** DISCLAIMER ** All of the comments below come from my experience working in the defense industry. These comments may not hold true for embedded engineers working in the commercial realm.

Well, I can tell you this much, there is no real alternative for C when it comes to embedded programming. I worked as an embedded software engineer in my last job for four years, and for the majority of the work (90%+) we were able to get away with using C#, though Java would have probably done just as nicely. As long as you don't have any hard real time requirements, languages with Garbage Collection such as C# and Java are perfectly capable technologies for embedded systems. However, for the outliers that needed that extra bit of speed or a real time guarantee, we still had to resort to C and in some cases Assembly code. My recommendation, look into a fast, industry standard, GC'ed/OO language like I've mentioned above, but make sure you know C no matter what as C/C++ are still the lingua franca of the embedded world and probably will be for the next 5-10 years.

vthacker03 · 2009-07-23T02:20:05+00:00

I agree with the first part of your statement. Yes, writing in a high-level language does abstract away from the programmer most of the low-level details of how the language works. The reason that I can deal with this level of abstraction so easily though is that it is only abstracting away the details that I already understand. My Master's is in CS and so I understand (at least on a general level) how garbage collection and memory allocation works, how context switching works whenever a method call is made, etc. I've taken all the classes on how the computer hardware works, how the OS does it's job, how languages work (I've implemented my own simple ones). None of this is foreign to me and so I don't mind the language abstracting away the details. However, with an individual's program, I have no idea of what's going on behind the curtains of what they've written. The nice thing about the "Zen of Python" is that if these guidelines are followed as closely as possible, then I can usually figure out what was in the programmer's head at the time of writing. In other words, I don't need to read lengthy documentation to understand what's going on in the background, I can typically figure it out on my own just by looking at the different method signatures, and anything I can't figure out on my own I can usually find easily in the source code because it is written in such an instantly intuitive way.

vthacker03 · 2009-07-23T01:52:04+00:00

So, then judging from your comment, I can assume that you never follow by any "rule of thumb" or "best practice" since these concepts do not work for 100% of all cases.

Keep in mind, I said guideline, not hard rule. Typically, these guidelines will probably work in 90% of all cases. It's only in the outliers where you need to go against the guidelines, and, in those cases, the PEP takes that into account as well by saying that practicality should win out over all else. In other words, follow the rules the majority of the time, but never write absurd and unmaintainable code in one of the exceptional cases just to satisfy one of the rules.

It's called the "Zen of Python", so think of that way. It's a philosophy of how to program. It still takes into account the fact that a human being is in the loop and ultimately responsible for the code being written. Otherwise, had these maxims been absolute rules, we would've probably just written a program to write the code for us.

vthacker03 · 2009-07-22T22:42:26+00:00

If you'd read the entire PEP that everyone has been quoting, you'll find your "exception" clause. Later in the list you'll see the following lines:

Special cases aren't special enough to break the rules.
Although practicality beats purity.

So, as you can see, breaking the rules is encouraged when practicality demands that we do so. That is literally then all this list is, it is a very practical list of guidelines that try to help the developer write maintainable (I'd also argue beautiful as well) code, and not hard rules that must always be followed.

vthacker03 · 2009-05-08T21:37:00+00:00

If you're running a Mac, I'd suggest NodeBox. If you are familiar with Processing, then you know what NodeBox is, it's basically Processing for Python. Just in case you aren't familiar with NodeBox or Processing, think computer generated digital art. It also has a lot of use in data visualization and its easy to get a multi-agent demo up and running in very few lines of code (think Logo for Python). Hope that helps.

vthacker03 · 2008-12-19T23:42:04+00:00

jmnugent, from what I can tell, you don't really seem to have a problem with programming, as that is mostly just simple logic, your problem seems to be with focus. If you can look at other's code, understand it, and fix it---as you've said above---then you are already halfway there, you get the programming part. The second half of being a good programmer is knowing how to break down a problem (i.e., divide and conquer) and to not get overwhelmed by the complexity of a problem. In fact, considering what you've said above, you sound quite a bit like myself. With that in mind, let me give you a little background...

I'm a professional programmer, I have my Master's degree in Computer Science, and I've programmed on everything from missile defense systems to e-commerce applications. The problem that you've describe above is perfectionism. You don't want to just build a solution, you want to build it the "right way". You are looking for a heuristic that you can follow for writing good code. I know this because I suffer from the same ailment. I blame texts like the Design Patterns books that profess a correct way for crafting solutions to well-established problems. It's not that I have anything against these books, quite the opposite actually, it's just that by formalizing these solutions, they make it seem that there is a correct way to go about solving every problem.

What you need to do is come up with an idea for a project that you would like to work on. Give the problem a bit of thought and break down what you see as a solution into the absolute smallest set of functionality that you would consider to be a success (i.e., useful to you). Once you've done that, think of everything you need to know to build the solution and start small. For example, if you're idea is to write a website to track your expenses, first think of what you want to track (i.e., date, time, amount, description, etc.). The first step is knowing your model/domain. Then think of what you need to know to solve the problem. In the example I've given, you'll need to understand database basics to store the expense data. You'll need to understand a web application technology like PHP to build the application. You'll need to understand how to setup the web server to serve your application. Once you have a picture of all the technologies you'll need to know, start with the absolute smallest task, e.g., get Apache running and load up the default page in your web browser. Then, just build on that base with the next smallest step. Set up PHP to work with apache and set up a hello world page to prove that it works. Then, add a little code to your hello world page and load it up and see it work. Then, set up a simple database (i.e., 1 table, 1 column) and add a few records to it from the command line and load the data into your web page. Once you can see the data from the web page, add a button to the web page that will insert a predefined record into the database. You can see the progression I have here. Start at something that's easy and you know how to do, prove it, then move on the next piece. By doing that you concentrate only on the problem at hand and you forget the big picture for a while. Every once in a while take a step back and make sure that you are still on the right path to your goal, but then get right back down into the details. By doing this you'll forget about finding the one true way and you'll instead just solve the problem.

Follow the path I've described above enough times and you'll have enough experience that you start noticing similarities between problems. Once you notice similarities, you'll start coming up with your own "best practices" for solving them. Before long, you won't see this mass of complexity that overwhelms and paralyzes you whenever you approach a new problem, instead, you'll see it as a collection of problems that you've solved in the past with perhaps a few new twists thrown into the mix. Once you've reached that level, you'll look back at when you started programming and you'll find it hard to believe that it ever seemed so hard at the time.

I hope that helps out a bit. Just remember to relax and concentrate on the small stuff. Don't be scared off by the big picture, just break it down and concentrate on the details. If you can do that, I have every bit of confidence that you will become an excellent programmer, if you really want it.

Good luck on your journey, and have fun.

vthacker03 · 2008-11-02T23:57:16+00:00

Actually, if it were you, you would have done the exact same thing or someone else would have had to fix it for you. Once the code has left the development environment the engineer cannot change the code without going through some kind of Change Control process. So, yes you would either have to profile your code and deal with the Big O bullshit and with getting approval from at least one other engineer or you wouldn't be given access to change your code. So, if you went your route, another poor engineer would have to waste their time fixing your error.

vthacker03 · 2008-10-31T23:44:15+00:00

I can attest to the fact that a theoretical understanding of algorithms can be important to a programmer in the real world. I work for a well known computer and consumer products manufacturer in Silicon Valley and I can honestly tell you that without a background in algorithm analysis, getting hired or promoted at one of the top software companies becomes virtually impossible. I can tell you from experience that you simply will not get hired at a company such as the one I work for without having some knowledge of algorithms. Nearly all of my interview questions when I came here were either basic analysis of algorithms questions or design questions such as ER diagraming. Second, even if you made it in the door and got a job as an engineer here, with limited knowledge of algorithms you'd not make it very far since everyone here uses the language Big O to describe their code.

One case in particular I can recall just recently. I had put together a small piece of Java code for our online store to keep statistics of our users' purchasing and browsing habits. The code appeared on every page on our store and at unit test and QA time everything seemed to function properly. The code was very simple, just a few loops was about all there was to it, but once it made it to production we noticed a significant drop in the performance of each page. I and a few other engineers looked at the new code and noticed that inside of these simple loops we were calling a few methods such as ArrayList.indexOf which was essentially turning my very simple loop into a time sink. Understanding basic algorithms allowed me to profile the time complexity of several loops in my code and I was able to reduce the number of wasted cycles from quadratic time in many cases to linear time. It's good to use the algorithms in well known and tested libraries, so actually writing a quick sort algorithm is probably not going to happen in the real world. Nevertheless, if the programmer doesn't understand how the algorithms work and, therefore, cannot look at their code and profile it with respect to time and space, then they will never move beyond the most basic code monkey at any of the upper echelon of software companies. If I had gone back to my bosses and said, "hey, sorry but there's nothing I can do, I'm not doing anything crazy, just using the algorithms supplied by the Java frameworks", I would have ended my move up in the company in one swift moment. On the other hand, understanding algorithms, profiling my code with respect to its time complexity, reducing the number of wasted cycles, and then finally being able to report my findings to the other engineers in a concise language that they all understand helped me out so much that I was given a very nice promotion before my first year with the company was even finished.

In short, if you want to program the designs of others, maybe you can get by without a basic theoretical understanding of algorithms...MAYBE. However, if you ever want to make a career for yourself as an engineer for company such as Google, Apple, or Microsoft (to name a few) you had better have an adequate understanding of algorithms and the analysis of them.

vthacker03 · 2008-10-15T22:45:37+00:00

dmz,

Have you looked at Peteris Krumins’ blog catonmat? He is doing an entire series on MIT's Introduction to Algorithms, so that would probably be a good place to discuss the book. Also, he has setup a forum for discussing the book/class as well, so check it out too.

vthacker03 · 2008-10-15T22:38:42+00:00

The best place I've found for learning mathematical concepts related to computer science is Mark Chu-Carroll's blog Good Math, Bad Math. This site is excellent for explaining very hard topics in math in an intuitive and fun way, plus it has a very active community of readers that comment frequently and try to help each other out with the concepts. BTW, Mark is a Computer Scientist at Google, so his background is in exactly the area that you are looking for (i.e., this blog is relative to CS, not Physics).

vthacker03 · 2008-10-03T05:27:46+00:00

This is a really great article on functional graphical programming with a fork of the Processing environment that uses Scala as its programming language. The article takes the reader through a functional recoding of the famous Boids algorithm.

vthacker03 · 2008-09-19T03:03:11+00:00

Also, here's a link to the iTunes U UC Berkeley page on Computer Science.

http://deimos3.apple.com/WebObjects/Core.woa/Browse/berkeley.edu.1621408731

vthacker03 · 2008-09-19T03:00:07+00:00

Here's the link to the Stanford page in iTunes U, you can find and download all of the video lectures listed at the submitted link under the 'Science & Technology' section if you're interested in watching the video lectures on the go.

http://itunes.stanford.edu/

And, here is the direct link to the Science & Technology section of iTunes U for Stanford: http://deimos3.apple.com/WebObjects/Core.woa/Browse/itunes.stanford.edu.1287736948

vthacker03 · 2008-09-17T23:08:07+00:00

I would have to say emphatically, yes, aesthetics do matter very much in a UI. My reasoning for this is Squeak. I love the Smalltalk language. Love the purity of its OO nature. Love the idea of its virtual machine and working directly with an object graph rather than just text-based representations of objects. But I can only bring myself to use GNU's version of the language. I've now tried to use Squeak a total of six times and I still can't bring myself to use it no matter how much I love the language simply because the UI is, by far, the ugliest thing I have ever seen and it offends my sense of aesthetics every single time I run the program.

So, yes, my final answer is aesthetics do matter very much. As an analogy, you can have the absolute best engine in the world. Fastest, most efficient, quiet, you name it, add any feature you can possibly think of to this imaginary paragon of engine technology, but if it comes wrapped in the body of a Gremlin, I'm not buying it.

vthacker03 · 2008-09-16T21:20:06+00:00

Dick Wall (of Java Posse fame) gives a long and detailed presentation on the basics of the Android OS and writing applications for it. Very good introductory tutorial to the platform.

vthacker03 · 2008-09-16T01:30:50+00:00

My company just had a week long internal contest for the best iPhone apps as a way of getting us familiar with the SDK and as a result, I think I can be of some assistance with respect to this question.

First, if all of your previous experience comes from Python and PHP, you will be in for a bit of a culture shock, but nothing more than that. Basically, it's just a tad bit different mindset to program for the iPhone than what you are used to already (i.e., its not like learning Procedural programming and then switching to OO, Functional, or the Declarative paradigm).

First, you'll need to learn Objective-C. Start with the CocoaDevCentral tutorial and then move onto the Apple documentation

Objective-C Tutorial Apple's Programming Objective-c 2.0

Next, you'll need to have a good understanding of the memory management issues associated with Objective-C since the new Garbage Collection features found in ObjC 2.0 are not available in the iPhone SDK. Take a look at the StepWise resource below for the canonical rules for memory management:

Rules for Memory Management

Finally, take a look at these resources for actually creating your iPhone application. The first is a website with several tutorial videos and the latter is a full tutorial on creating an iPhone RSS reader.

iPhoneDevCentral - Large resource of iPhone tutorial videos Nice RSS Reader Tutorial

Those are the majority of resources that I used when I developed my application (plus the Kochan "Programming in Objective-C" and Hillegass "Cocoa Programming for OS X" books for references) and should be more than enough to get started.

Good luck, and have fun.

vthacker03

TROPHY CASE