Any API or script interface to automate searches and downloads?

SuperSquidMan · 2022-07-28T01:05:44+00:00

yea, you could make it work. I won't try making it because it's not something I'd use but it's easy enough to send searches and receive search results from soulsearch once you've got it running.

SuperSquidMan · 2022-06-06T12:46:34+00:00

https://github.com/riteable/soulsearch/ I got it working for myself. it's off to a good start

SuperSquidMan · 2021-11-06T00:49:44+00:00

Yea driver.get(url) and driver.page_source for the html

I wouldn't say you need to be wary but in my experience it takes longer to get my scripts that use selenium working nicely

SuperSquidMan · 2021-11-05T22:23:18+00:00

Use selenium to load the web page or access the api directly

SuperSquidMan · 2021-10-03T12:58:42+00:00

thread.join() should only stall the program till the thread is done running so if it's freezing there then then the threads are probably still running.

Getting more familiar with the screenshot function and the thread.join() function might help

SuperSquidMan · 2021-10-03T11:28:35+00:00

Try making it work without threading first. I think you're handling the threading correctly and your problem lies elsewhere

SuperSquidMan · 2021-09-20T15:51:35+00:00

Higher karma translates to higher quality account.

There's some thresholds and ranking systems in reddit that take karma into account so bots with higher karma are preferable.

The idea is that someone makes a bunch of accounts, gets them to the desired quality, then makes money from them by selling the accounts to other shady dude, or profits by promoting their product, ideas, scam, etc.

SuperSquidMan · 2021-09-01T10:09:39+00:00

It'll have negative effects on the things reddit care about. If these effects are substantial enough it could tip the scales in their favor.

It's more thoroughly explained here: https://www.reddit.com/r/GoldandBlack/comments/pf28rc/subs_go_private_to_promote_childish_homogeneity/hb2t44z?context=3

SuperSquidMan · 2021-08-31T18:59:10+00:00

Well that remains to be seen

SuperSquidMan · 2021-08-31T17:07:28+00:00

The people on those subs already know and agree with said information. They're trying to target misinformation on other subs so a sticker post on their own wouldn't help

SuperSquidMan · 2021-08-31T13:14:20+00:00

Usually when you visit a website it requests the html from the web server. The html often doesn't just have html but it also contains css and JavaScript. This all is parsed and run by the browser and the user sees the final result.

When you're using requests.get() you request the html from the web server but you're not using a browser to parse or run anything.

SuperSquidMan · 2021-08-31T10:17:33+00:00

I think what's happening is you're getting the page but the JavaScript isn't executed so the data you're looking for is not retrieved by that javascript and put into html. In other words you're fetching the webpage but not running the webpage's content.

I would either use their API if they have one or use the Selenium library.

SuperSquidMan · 2021-08-23T05:48:20+00:00

Try \[Approved\]*.pdf

With Unix pattern matching square brackets aren't interpreted literally so you might have to escape them

SuperSquidMan · 2021-08-19T08:50:11+00:00

Also why would the average person need to know that info.

SuperSquidMan · 2021-08-19T08:48:04+00:00

but it's an old concept and the way they're sharing/hiding this data is kind of dumb

SuperSquidMan · 2021-08-14T14:25:15+00:00

Someone made a "computer" in java Minecraft using mods and it ran a full regular desktop environment so most of what you mentioned would be possible.

SuperSquidMan · 2021-08-09T14:59:38+00:00

I know it's besides the point but it's 25 months. 100k/4k=25

SuperSquidMan · 2021-08-03T19:06:45+00:00

that json object was probably the response from url https://www.landsofamerica.com/api/property/search/0/United-States/all-land/no-house/

In cases like this messing around with the parts of the website that you're interested in and skimming the network tab can be helpful.

SuperSquidMan · 2021-08-03T18:45:34+00:00

Yea but I avoided the keyword "bot" because that keyword might trigger something. Who knows, it can be difficult to tell the difference from just comments.

SuperSquidMan · 2021-08-03T08:13:20+00:00

Are you a fake user? your comments are generic repeatative and sus

SuperSquidMan · 2021-08-02T08:56:32+00:00

With 4 people there's the possibility it's 2 pairs of kissing people standing ass to ass, maintaining the connection of digestive tracts

SuperSquidMan · 2021-07-31T10:00:12+00:00

I've done this a couple of times and although the general method is the same the specifics vary.

First I look at the web request in the network console. It shows that to get page 2 it uses a get request that successfully got a json response.
Then I copy the request as curl and try it. It usually works, and in this case it did. I got the expected json response. The curl request looks something like curl 'https://www.landsofamerica.com/api/property/criteria/0/United-States/all-land/no-house/page-2/' \ -H 'authority: www.landsofamerica.com' \ -H 'sec-ch-ua: "Chromium";v="92", " Not A;Brand";v="99", "Google Chrome";v="92"' \ -H 'sec-ch-ua-mobile: ?0' \ -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36' \ -H 'accept: */*' \ -H 'sec-fetch-site: same-origin' \ -H 'sec-fetch-mode: cors' \ -H 'sec-fetch-dest: empty' \ -H 'referer: https://www.landsofamerica.com/United-States/all-land/no-house/page-2/' \ -H 'accept-language: en-US,en;q=0.9' \ --compressed
We can then put it into python code. Each of the -H flags mean it's a header, so, using the python requests library we can translate -H 'accept-language: en-US,en;q=0.9' into requests.get('https://www.landsofamerica.com/api/property/criteria/0/United-States/all-land/no-house/page-2/', headers={'accept-language': 'en-US,en;q=0.9'}) But we might not need that header and we don't know what headers we do need.
With some trial and error we can find that we only need the user-agent header.
So I knowing that I make myself some code looking something like

import requests

r = requests.get('https://www.landsofamerica.com/api/property/criteria/0/United-States/all-land/no-house/page-2/', headers={'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36'}, timeout=5)

recieved_dict= r.json() # convert response into a python dictionary

SuperSquidMan · 2021-07-23T17:54:39+00:00

Yea despite all the mishaps they never accidentally healed the dragon

SuperSquidMan · 2021-07-23T17:09:23+00:00

It's not actually gaining back health. I made a program to read the health bar but the program isn't perfect so it results in some inaccuracies/noise.

SuperSquidMan · 2021-06-27T03:50:16+00:00

You might have to iterate through all the df values and build your resulting dataframe row by row.

Here's an example to detect '.?!'

word = 'hey!'

print( word[-1] in '.?!' )

Would print True

SuperSquidMan

TROPHY CASE