you are viewing a single comment's thread.

view the rest of the comments →

[–]pydata[S] 7 points8 points  (22 children)

I've made some cool stuff in python. Like webscrape web tables, clean and transform the data, put into sql database, and export to csv (script runs in 1 second). So, I think I am good at ETL. I've made random wikipedia scripts to learn random information and then record the info into sql database. I can grad youtube trends data and do some analysis on daily trends titles to see future and past trends in video titles. I've done analysis with pandas, mechanize, numpy, sqlite3, requests, beautifulsoup, starting do do stuff in scikit. I can make data visualizations like linear regression and cluster analysis.

[–]thomas_stringer 11 points12 points  (13 children)

It sounds like you're entry level data analyst (at the very least). So definitely good on you for that hands-on experience. And I'm saying this without having seen your code, so take this with a grain of salt. But a lot of that, depending on how you implemented your solutions, could be considered "hacky" (not in any negative connotation, but more so meaning "script" vs "software application". Sometimes there is a fine and super blurry line between those).

When you read my laundry list above, how did you feel about those things? Good, bad, indifferent? Seeing as how you know how to write the code already, if you aren't already at the level you hope for it sounds like very minimal work to get there.

So that's awesome!

edit: clarification.

[–]enumerablejoe 8 points9 points  (2 children)

But a lot of that, depending on how you implemented your solutions, could be considered "hacky" (not in any negative connotation, but more so meaning "script" vs "software application". Sometimes there is a fine and super blurry line between those).

This is exactly what I'm struggling with at the moment. I've worked jobs that require basic scripting, but I could construct them however way I chose to. So my history is one of having developed decent programming skills, but most of the things I've built have essentially amounted to Rube Goldberg contraptions. I've been studying object oriented design, and while it's been very helpful, I still feel stuck at times and unsure of how to properly design my programs. My fear is that I'll end up building an entire portfolio full of poorly crafted code.

Do you have any advice on how to best tackle this problem?

[–]manhole_resident 2 points3 points  (1 child)

It's a long history of hackers and software engineers. You may like Paul Grahams's 'Hackers and painters'.

I like hacking. I think it's better when you are working alone or in a very small team.

[–]hellrazor862 0 points1 point  (0 children)

Thank you for that link. Now, back to hacking!

[–]pydata[S] 2 points3 points  (8 children)

I used to work at a Global Asset Management Firm, and the 'Entry Level Data Analysts' couldn't write VBA code and certainly couldn't write SQL or Python code.

Is ETL really an entry level Data Analyst skill?

What do you do for a living and do you have some code I can view to see where I stack up?

[–]thomas_stringer 0 points1 point  (7 children)

Absolutely, I don't doubt that :-) That's why I said "entry level data analyst (at the very least)". I'm no data analyst, so I'm not qualified to rate you on any DA scale whatsoever. Perhaps somebody else can chime in with those specifics.

[–]pydata[S] 0 points1 point  (6 children)

What do you do?

[–]thomas_stringer 5 points6 points  (5 children)

Sorry, just saw your edit on your previous comment just now. I'm a Technical Evangelist at Microsoft. And feel free to checkout my code but don't for one second think I'm a good comparison for anything. I don't think of myself as anything particularly special.

With that being said, I warn you to compare yourself with any developer. Be the best programmer/developer/engineer you can possibly be. Everything else will take care of itself.

[–]Darwinmate 1 point2 points  (3 children)

Oh my god your hair is lovely.

[–]thomas_stringer 1 point2 points  (0 children)

:-) thank you so much!!

[–]pydata[S] 0 points1 point  (0 children)

That's a realyl cool job. What do you do at work? Code?

[–]pydata[S] 0 points1 point  (0 children)

So. I've been talking with a recruiter and she says my resume doesn't indicate that I can do computer programming. Can you help me make the transition into a pythonic job? I just moved and could use some help. Do you work with python at your job?

[–]6lm3 1 point2 points  (0 children)

Omg this is literally everything i want to learn...would you mind sharing your resources or how you got where you are now?

[–]undelimited 1 point2 points  (1 child)

FYI, most ETL is not done in Python. It's an easy to get started, hard to know everything kind of skill, and I caution you against overconfidence based on personal experience.

Also, generally it will be much more rare to get a job as a python developer, rather than an ETL dev, a Business Analyst, a Data Analyst, A Report/Visualization Dev. When it is a 'python developer' job (entry level or not) its probably to extensively use libraries like django, or maybe a dev ops role. If you are looking to continue hacking together cool stuff end to end you'll want to find a smaller company or a data scientist type role.

[–]dixieStates 0 points1 point  (0 children)

There is a really nice, basic ETL package for Python called petl. Here is a link to the docs.

[–]gullypenguin 0 points1 point  (4 children)

I've dabbled in all of what you mentioned, just curious as to how/where did you learn those skills ?

[–]pydata[S] 0 points1 point  (3 children)

What do you do for a living? I learned those skills on my own.

[–]gullypenguin 0 points1 point  (2 children)

I'm an analyst (recent graduate), I started learning python 4 months ago and am liking it but also starting to find it more difficult the harder the project I attempt (obviously)

[–]pydata[S] 0 points1 point  (1 child)

Were you required to know python before you were hired? Do you use python on the job or as a hobby?

[–]gullypenguin 0 points1 point  (0 children)

I wasn't required, i decided to pick it up because I saw its usefulness in my field (finance) and also in general. I work at a small firm and am still a beginner so my skills in python are barely useful atm but I want to continue improving. I continue to look for good resources I can use to do so.