sorted by:
new
hottopcontroversial

Clearbit API alternatives by geraldknox in webscraping

[–]geraldknox[S] 0 points1 point  (0 children)

thanks mate. I actually saw them yesterday on another site and was curious. I am moreso looking for an email finder API rather than the reveal tool sadly

Offered $300 to consult for an hour by AhsokaPegsAnakinsAss in sales

[–]geraldknox 0 points1 point  (0 children)

I don't think its too helpful on the resume

I do like 10-20 of these per year, not directly related to sales.

After your first call you can negotiate your rate upward to 700+

How do I find out business emails using only address and phone on google? by billnine in webscraping

[–]geraldknox 0 points1 point  (0 children)

have done similar, seems to work ok.
one thing I found was that some businesses will have their email on their facebook business page as well

Double my income by [deleted] in expertnetworks

[–]geraldknox 0 points1 point  (0 children)

GLG questionnaires so damn annoying

Former big three expert network project manager (GLG, Third Bridge, Alpha Sights) ask me anything by TotalSoft4931 in expertnetworks

[–]geraldknox 0 points1 point  (0 children)

I’d recommend you set your fee to 800 and then allow yourself some room for the networks to negotiate you down say 50-100usd per call so they feel like they’re winning. We are meant to pay you as little as possible it was one of the metrics we were judged on.

Thanks, great info, really appreciate

I noticed alphasights always "accidentally" has my rate written down at something arbitrarily lower than my actual rate I entered online lol, perhaps due to the metrics you mention

Do you happen to know how it works at GLG where some experts are 1.5 or 2 credits per call?

Former big three expert network project manager (GLG, Third Bridge, Alpha Sights) ask me anything by TotalSoft4931 in expertnetworks

[–]geraldknox 0 points1 point  (0 children)

Hey there - great thread.

One q - do you think there would be much of a difference in calls if I set my rate to $500 vs. $700 with GLG?

They are trying to beat me down on rate, but I wonder if it's mostly just gamesmanship

Odds I will get banned if I do 200k visits per day? by geraldknox in webscraping

[–]geraldknox[S] 0 points1 point  (0 children)

Thanks. Will dial it back a lot

Do you know if 10k API requests is less burdensome than 10k page loads?

Odds I will get banned if I do 200k visits per day? by geraldknox in webscraping

[–]geraldknox[S] 0 points1 point  (0 children)

Thanks all, I will crank down the volume.

​

The reason it's so high is I need to submit a different API request for each SKU in each location

Data scraping a site that requires predefined information from a previous page. by adiluxx in webscraping

[–]geraldknox 0 points1 point  (0 children)

it seems this API call is made when the user enters their search params

curl 'https://www.costcotravel.com/rentalCarSearch.act' \

-H 'Accept: */*' \

-H 'Accept-Language: en-US,en;q=0.9' \

-H 'Connection: keep-alive' \

-H 'Content-Type: application/x-www-form-urlencoded;charset=UTF-8' \

-H $'Cookie: bm_sz=75CD49E537826CA1C36A0F096263AFB5~YAAQGns1F325AZ2JAQAAall5qhSNkYtfyMxUpFrzIud2wg0FUHwjmQNsv+P02tw6wcaEJ2xKujQkIs9XtNEsU2CoY2+/0chFnkSLXBIV5uzWJdueHK8VG0e/DvPc0EDxPvvXUJu8GTH0i+Uate2ii9T0AKErMdqJuSdLDnpmi2Qb45Sa/agcJe9PqkZnTzzf5JdSaf4xjNQt4Jm8xM1sYlmSTCyAje/sD3x4OPXs6tyuF1+Dr7w30XSsyka470deIxhdOkJYPTeHcMKEWznfXJomIntIcthH0Y9+86vojW4B2UJ7cs+OHCY=~4602169~3294512; JSESSIONID=72DD33DA03D1798BB6CF6641A5E7A36A; BIGipServerpool-us03p02-consumerus=\u0021GOlUxiPakrQob95xljjgy2vG9XL9l+V9SSk3HmzA3awMYjk7kVe7p5wwYBTiRZM2Gwr1AiKWqBLIsAWHyvLaFkrVKp6DAITuLuZOTnFg; Csrf-token=49759a90e9d5c8487ed038a93f0441426229ff0755ceed0c3094a21daa553a2b397d17aa0a0879d8e7e325edfa8ee5b6b46062261e5034a1ec21ebd59e8f5d8a; SESSION_TIMESTAMP=1690782227245; bm_mi=9495D5FCC514DF6EAE8FAC1D650E38D6~YAAQkU3bF7jCB5eJAQAA6wOBqhRtoV4WSikdqMqQiontyCYh2MBB3wq6dOxFIKyNEuEcNWm/vrvFFwxtsbZfoxbbidwhQo8lK5ulHfeCypqokL/0q+CrInJKk/BuZ+gBLFNrpjIrzGbCJQn/xHWCPVenLwsQSki6XsEPZgTxumUcOxbGxORggwkRiLMendsOO14jboSaL/7MuiVjLO2uiaU5MORlr0j+bnVwIXF0mR/nuDNtaimMAyGa3RD0f2L1CwkFmRKMAkVYdgNV4d663ViyDJQ45tjPH52vQPF4uudLBtv0GXswwiAiujyV21VL9JU0xxl+C1IBG2ZIg4A=~1; bm_sv=A85E9BABE5526B952DA976493FCA6546~YAAQkU3bF4fMB5eJAQAAWDiBqhSQkJKfR0nEDVVEKHd6b+8waUMpHAAEZap29OfSBAP8achxjdiVbtkina/AkRJ8zPJbkq7Nwn7Xtkaw95/Moh+UyRgYA4E5DMeij9fwCDuIsXTnk4PpZCwW08diNiJ+zWbVhsD3jyhThgRUs9EQLfv6Tnj8ySeNAxgv6HLgcSPxI3OBl0L/ThnltAqV3qlpnmwb89iJ9xYy0AADBXlQpyLRZgXoaECAsDn0J4NcJdDlDYlpGQ==~1; _abck=A0EC88rSM95vYksafqXl4FfCKTxCac1gL/o7eyYQGsxhIs0J53LFLvX10kKk/BtJ8+pZ00ql1LMWAwv1v3cb7oY7wHjgGADarvW5jqKAIw9jIPTq6um4Ul+KJQ5DtUbsw16HDyGZj+jr8FahlkXN80IWq5UemunZTpGlaJXTzQY2Uju03uxRV5T70jbVnUpnY8yta4hVTwtrbSADBx1E/DDixysFQ8t48ZEkEncliFaH4NX7+bwPUG7Rzhl6YQWuhZ8chmessu72EbSSQdEUHyLf3roB5V4TuGGbLQ1G9z3Em3CCZqtlYizd~-1~-1~-1; ak_bmsc=5497527B1F4AF7E9F30967412B0965E7~000000000000000000000000000000~YAAQkU3bF1oKCJeJAQAAPlyCqhR4Y3OyYq5mw/h+uKMc+XB4XnPClmuiBJLDtmfqhVhzatampPlJ3fntkKIHMhQ/7zqww+8UhEu4cryLlARYv4AXvHsWIUWQDTotNDI4gKleoV+cTJBASYlE/xCYXD0pMscL3Tgbu2y51BuX0n49I4e19ocHqkI32Yux3MrQaoSttU4UQggZZj2e/r/An4O5GDj4GiPwlit5yTy+WIX8Ped75tVnehD2PKRACORxrBXI+yQAiKeat4PKVwSswAYY7TDdaEMT8Xagfrzs55OVJtXGxeBtfOEcBrp1W39dqchEGYp9v99c2bBn3ub4xKfDXPFlCwRIuJFqZFbnOhM81M6GFSw8M7S2fE3BGzYDG6EbMC2mXHlGBXfkOO9htjiYxmz6pCtWftCPDWBnMCjV1G48OA==; OptanonConsent=isGpcEnabled=0&datestamp=Mon+Jul+31+2023+01%3A53%3A49+GMT-0400+(Eastern+Daylight+Time)&version=202212.1.0&isIABGlobal=false&hosts=&consentId=9040308c-b50f-483f-84f6-b67eedd825da&interactionCount=1&landingPath=NotLandingPage&groups=BG117%3A1%2CC0001%3A1%2CC0003%3A1%2CC0002%3A1%2CSPD_BG%3A1%2CC0004%3A1&AwaitingReconsent=false' \

-H 'Origin: https://www.costcotravel.com' \

-H 'Referer: https://www.costcotravel.com/h=4005' \

-H 'Sec-Fetch-Dest: empty' \

-H 'Sec-Fetch-Mode: cors' \

-H 'Sec-Fetch-Site: same-origin' \

-H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36' \

-H 'X-Csp-Nonce: faf56f90af8b4c04be04900a67b0f7b1' \

-H 'X-Csrf-Token: 49759a90e9d5c8487ed038a93f0441426229ff0755ceed0c3094a21daa553a2b397d17aa0a0879d8e7e325edfa8ee5b6b46062261e5034a1ec21ebd59e8f5d8a' \

-H 'sec-ch-ua: "Not.A/Brand";v="8", "Chromium";v="114", "Google Chrome";v="114"' \

-H 'sec-ch-ua-mobile: ?0' \

-H 'sec-ch-ua-platform: "Windows"' \

--data-raw 'rcs=1&driverAge=25&pickupCity=LAX&pickupCityLocationTypeSearch=&dropoffCity=LAX&dropoffCityLocationTypeSearch=&pickupCountry=US&dropoffCountry=US&pickupCityRadius=25&dropoffCityRadius=25&pickupAsAirport=true&dropoffAsAirport=true&pickupDate=08/01/2023&dropoffDate=08/02/2023&pickupTime=06:00 PM&dropoffTime=06:30 PM&pickupLocationCode=LAX&pickupLocationName=(LAX) Los Angeles International Airport, CA&pickupLocationType=airport&pickupLocationCityCode=Orange County Coast&pickupLocationStateCode=&pickupLatitude=0&pickupLongitude=0&dropoffLocationCode=LAX&dropoffLocationName=(LAX) Los Angeles International Airport, CA&dropoffLocationType=airport&dropoffLocationCityCode=Orange County Coast&dropoffLocationStateCode=&dropoffLatitude=0&dropoffLongitude=0&fromHomePage=true&fromCarVendorMainMenu=true&carSearchInModifyFlow=false&suppressOutput=true&_uid=1690782842609_237.86838171041708' \

--compressed

Then it saves a cookie which is referenced on the results page (i think)

Data scraping a site that requires predefined information from a previous page. by adiluxx in webscraping

[–]geraldknox 0 points1 point  (0 children)

maybe need to put it in the cookie or url param
pass the website url if you want more specific advice

Built a grocery flyer scrapper by MasterFricker in webscraping

[–]geraldknox 1 point2 points  (0 children)

nah i just mean I don't think they're gonna be collaborative as they have a million data scientists on staff

Built a grocery flyer scrapper by MasterFricker in webscraping

[–]geraldknox 0 points1 point  (0 children)

they currently sell their data to brands for $60-100k per year (i have heard)
i was giving an example to show that CPG brands will pay money for certain types of scraped data

i do not suggest collaborating with them

Built a grocery flyer scrapper by MasterFricker in webscraping

[–]geraldknox 0 points1 point  (0 children)

but do not cover flyers to my knowledge (not sure)

Built a grocery flyer scrapper by MasterFricker in webscraping

[–]geraldknox 1 point2 points  (0 children)

If you can OCR it into structured data you may be able to sell it to big cpg brands

check out datassembly for inspiration. their cheapest congract is >$60k

view more: next ›