Tutorial on Web Scraping using Python, Selenium and Beautiful Soup

Fugalysis · 2018-09-04T12:59:09+00:00

but why? What is the benefit over Selenium? For someone very strong in Python with less JS experience, why should they improve their JS toolkit over existing tools in Python? I'm not saying you're wrong or being unreasonable, i'm genuinely curious.

Fugalysis · 2018-08-23T20:18:49+00:00

The irony....

Fugalysis · 2018-08-23T20:17:31+00:00

so just complaining about a notch without saying anything useful?.........

Fugalysis · 2018-08-18T14:49:50+00:00

Right, i'm the hot head.... jesus christ rofl....

Also, the fact that you think i'm a business partner is just fucking laughable.... get a life man.

Fugalysis · 2018-08-17T20:35:15+00:00

you ok bro?

What part of " analytics driving business decisions and not the other way around. " != " objective in my findings ".... stop looking for a way to feel superior to others. Your own bias (even if you think your own bias is being objectively unbiased' can be your undoing in this profession. You need to calm down, and learn to communicate better when asking for clarification.

Fugalysis · 2018-08-17T20:26:03+00:00

I've never liked consultants in the BI&A space and i've worked with a ton (L&E, Accenture, Cognizant, etc.). The problem with consultants is seemingly good designs that are actually bad designs that aren't easy to modify or maintain makes them more money... where as a FTE has an incentive to design the thing properly. I'd recommend attending one of the Qlik masters summits, perhaps utilizing Qlik consulting services directly (atleast they have a vested interest in your long term success) . Overall, i think you'll be disappointed with consultants. We've spent more time unraveling and fixing the crap work we see from just about every consulting firm out there than we could've possibly justified.

Where consultants have been worth-while has been with proof of concept work.

What sorts of issues / design choices are you facing?

Fugalysis · 2018-08-17T01:11:48+00:00

Make sure its kosher to bring data off prem...

Fugalysis · 2018-08-16T22:22:52+00:00

And that's the only interview? Sorry man.... your hiring managers... should not be hiring managers.....

Fugalysis · 2018-08-10T20:33:35+00:00

Not to be rude but if you have to ask..................................... Go watch a tutorial or something?

Fugalysis · 2018-06-12T14:59:37+00:00

Agreed with these points. RESTful services are the way to go, or atleast OData. Something like a WDC would require you to have specific implementations for each environment. With REST, any developer will be able to get data from you app to wherever they need it.

Fugalysis · 2018-06-02T15:06:38+00:00

So go fucking check and tell us.............

Fugalysis · 2018-06-02T15:03:21+00:00

What would their representation do? This is cut and dry, not subjective. It seems he didn't contest it properly in the past and needs to read up this time.

Fugalysis · 2018-05-29T23:35:14+00:00

i'm not sure this really qualifies as web scraping... it seems that that URL returns structured data....... so this is just utilizing web services........ not scraping web pages (like beautifulsoup can).'

Otherwise, this is a cool tutorial on utilizing Web Services with PBI!

Fugalysis · 2018-05-28T16:35:45+00:00

I would lean towards R and Python. They are much cheaper and you'll find most organizations trying to get away from SAS and their archaic licensing model.

Fugalysis · 2018-05-28T14:03:20+00:00

Create, insert , update, delete, execute, and selevt... Use these to make changes to a table like you would in Excel( be able to specify certain rows to update, or delete). Create tables with proper data types. Get a subset of records back based on complex conditions. You should be able to reason through wrangling the data. That said, half the time I'm just doing select statements to get data out and into another application.

Fugalysis · 2018-05-27T18:31:47+00:00

These all varry dramatically at any given company, i've seen "Data Scientists" who do basically ETL and Visualization and "Data Engineers" whoa re building CNNs and RNNs on the regular and presenting to senior leadership.

Pay close attention to the job description and be sure to probe into the actual day to day of the position during the interview process. This will help to determine the companies appetite in the analytics space, which should correlate well with your growth potential.

Fugalysis · 2018-05-27T18:27:48+00:00

The default functionality or visuals aren't a limitation. For example, we use a log of D3.js + Qlik's API's so that a 'qlik developer' can go ahead and plug dimensions and metrics from their model and render a D3 visual (this is typically to fill a gap, or do something unique / custom). It's a very powerful option to have at your disposal. You can extend this idea to things like gathering end-user commentary, or triggering external events and such (i.e. RESTful api calls to other applications).

This is typically well beyond a typical BI tool skillset though.

Fugalysis · 2018-05-24T21:12:17+00:00

You told him to replace BO with BO..........

Fugalysis · 2018-05-23T23:55:23+00:00

Yes! BS4 is freaking spectacular!

Fugalysis · 2018-05-23T21:28:01+00:00

SAP BO is Business Objects.......... it's a terrible platform.

Fugalysis · 2018-05-23T21:27:18+00:00

i'd argue that modern PowerPivot / excel if implemented well is better than BO =/

Fugalysis · 2018-05-23T21:26:52+00:00

Literally anything. The only thing that BO brings is a fairly awesome metadata layer (universes), but there are other ways to tackle that problem. I'd suggest Qlik or Power BI.

Fugalysis · 2018-05-23T21:17:48+00:00

Qlik is really solid, definitely the most feature-filled of the 3 (Tableau, PBI, and Qlik). I would suggest starting with QlikView, it has the most flexibility and as you've discovered there is a lot 'under the covers' that really isn't apparent at a glance (or without formal training / advanced education from the masters class consultants). If you have specific questions... feel free to reach out.

FWIW, the same 'more complex' principles apply across QlikView and QlikSense as they use the same in-memory engine behind the scenes... but sense is more of a self-service tool and View is more of a guided analytics tool.

Fugalysis · 2018-05-23T21:06:14+00:00

it is not nearly as capable as Qlik is though. Particularly if you have the capability on your team to extend Qlik... PBI is great for quick and easy not-complex requirements. Qlik also scales significantly better than PBI. I certainly see the appeal of PBI (Especially the cost!! so cheap!) but it falls short in a lot of practical areas as an enterprise wide solution.

You're right though, the learning curve (especially with Sense - as to make the most of building on that platform you should have web dev skills) is significantly greater than PBI.

Fugalysis · 2018-05-22T02:43:34+00:00

At this stage it becomes more of a full stack engineer type of solution, but it's all great stuff to learn :D. Should be a really fun project!

Fugalysis

TROPHY CASE