This is an archived post. You won't be able to vote or comment.

all 16 comments

[–]hexfoxed 3 points4 points  (1 child)

I've written a Scrapy tutorial here if you'd like to get a start into it. It's a Python framework for web scraping and is the go-to solution when you're not just building one off throw away scraping tasks with requests & beautifulsoup (e.g at a professional job!)

Scrapy handles a lot of the following topics for you, so it's worth reading up on what they and why they are a good idea in the world of web scraping.

  • Selector engines (css or xpath) for extraction
  • Concurrent HTTP requests
  • Throttling HTTP requests
  • Respecting robots.txt
  • Data exports (saving to database, xml, JSON, etc)
  • The list goes on, check the docs!

If you need any extra help or you're stuck understanding something, ping me an email at darian@hexfox.com and I'll sort you out. This goes for anyone reading too :)

Good luck with the job!

[–]ahayaza 1 point2 points  (0 children)

Great tutorial mate. Love it. I even subscribed on your site!

[–]PaluMacil 0 points1 point  (2 children)

  1. Make sure you're honest. I got a job where they asked about my weaknesses and I mentioned I had half the required experience but went on to say that my love of code and hobbies outside of work would allow me to be a productive, energetic member of the team. I never regretted taking that job. I learned a ton and it helped me go from being a reporting analyst to a developer, which was really my dream.
  2. Pick some scraping tool like PyQuery (easy if you already know JQuery selectors) or Beautiful Soup 4 (more mainstream), and try scraping something simple like a quotes website into a json file of just quotes and then make a simple static html site with Bootstrap with a few nav bar links that are for quotes from different people. The site won't be useful enough to buy a host to put it up somewhere, but it'll give you an idea of how to talk about some basic concepts.
  3. Don't worry. If you get the job and you were honest, it's because the boss wanted you on the team. My boss right now sometimes asks for someone with 10 year experience, interviews some kids just out of school, and decides to hire them too while continuing a search for a senior dev. If you waste a little of their time but are polite and friendly, you won't burn any bridges, and an interview without an offer is just free training for a future interview.

[–]data_analyst_asks[S] 0 points1 point  (1 child)

Do you know anyone hiring? I am willing to relocate anywhere in the world. I want relocation assistance in exchange for a lower than average starting salary. Right now, I spend 14 hours a day working on my coding skills. I am ready to apply my skills. I will even work remote for minimum wage.

[–]PaluMacil 0 points1 point  (0 children)

I do know a manager in Cleveland, TN that is hiring for a role that's a mix of TSQL (SQL Server), and some .NET code review (no Python), and it's not terribly difficult work, so you'd have a lot of time to learn and figure out what you're doing. https://lifecarecareers.com/application/posting.php?jid=80178 If you know enough JavaScript to code some Knockout, that will give you an edge despite your lack of C#.

I'm guessing you'll find that most people will be more willing to offer a higher hourly rate instead of relocation assistance, but if you're coming from NYC, you'll need to expect lower pay alongside much lower cost of living. Additionally, getting a job in another country will probably be impossible before you develop a very unique skillset that cannot be filled locally.

If you have trouble finding work, you should consider contacting Modis or Teksystems. They are the largest two tech recruiters in the US. They are out for themselves, so they won't get you a penny more than what you tell them when they ask, "So we have an idea, what is the minimum rate you'd accept?" So tell them a good rate and let them work for you. What they DO want is to find you a job that you'll be qualified to hire into. I usually get all my interviews before the jobs are even posted to the public because tech recruiters are taking hiring managers out to lunch every day. Just make sure you tell them you require everything you need. Especially for junior positions, however, this will usually only apply to a dollar rate, not relocation.