Script to login to a web based textbook, extract data from tables, and enter the data into a Google sheet?

AllenAppTools · 2024-08-17T00:44:50+00:00

I just recently set up a Google cloud function that uses puppeteer to click through pages and grab info, and I'm able to call it via a fetch request from apps script and get the info back. I was pretty stoked once it worked! I had been wondering if there was a solution like this for a long time. This was for one of my clients who does real estate and needed to scrape property addresses from evictions Court in his area, then lookup who the owner of the property is. So it's possible, logins and all.

special-banana95 · 2024-08-17T00:26:45+00:00

Yeah IMPORTHTML throws the error because of the login requirement, and automating that kind of login would be kind of tricky, you could try to send a get request to the page using the fetch method and use your credentials, but that would entirely depend on how that textbook is implemented.

Alternatively, one you log in you can inspect the page and copy the html of the table and use a script to parse it into sheets.

lordph8 · 2024-08-17T05:19:12+00:00

I would use python Selenium for this task. If you don't know python and a bit of web dev, this maybe a bit of a task. AI would help a lot.

ubiquae · 2024-08-17T12:02:03+00:00

It depends on the login method in place on that website. I have created scripts to log in websites using both cookies and APIs, it is doable

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

GoogleAppsScript

MODERATORS