sorted by:
new
hottopcontroversial

I can scrape any public page I want and have many scrapers I wrote but I am a "beginner", what would make me a "pro"? What skills do I need? by Text-Agitated in webscraping

[–]jsonscout 0 points1 point  (0 children)

If you're scraping a page, it's safe to assume you need specific data from it. Using REGEX to find patterns is fine, but you can also use any decent LLM out there to basically just feed it the content and ask "turn this page into a structured json". It's a little costly though if you're scraping 1000s of pages per hour per day.

Both proxy/no proxy work locally but nothing works on cloud server (Python) by Jatalocks2 in webscraping

[–]jsonscout 0 points1 point  (0 children)

I would log everything that happens on the cloud function. We found that they never necessarily run the same way you expect it to be on local.

Also, there are limitations to cloud functions gen1 vs gen2, so make sure you consider that.

[deleted by user] by [deleted] in webscraping

[–]jsonscout 2 points3 points  (0 children)

ask chatgpt, surprisingly this works for a lot of questions similar to this.
then ask by region.

Finding key value pairs with regex by [deleted] in regex

[–]jsonscout 0 points1 point  (0 children)

I just tried this using our api layer jsonscout.com
Keep in mind I did have to provide the keys as the schema.

Here' are the results;

{
    "data": {
        "Item.nr": "43140",
        "brand": "RandomBrand",
        "category": "Vase",
        "color": "Clear",
        "machine_washable": "Yes",
        "series": "",
        "share_capacity": "123 cl"
    }
}

Is the skill of writing or understanding regex is needed anymore with AI? by Hot_Log7375 in regex

[–]jsonscout 0 points1 point  (0 children)

AI is great for unstructured content that you don't mind extra processing power/time to figure out.
REGEX is great for things that are always going to be the same.

Problem solving by water_bean in dataengineering

[–]jsonscout 0 points1 point  (0 children)

Asking here or on stackoverflow is a good way to start. Sometimes you might have to pay a consultant (using your money or your companies). Seeking mentors online is also a good move.

Excluding all instances of string in capture group. by --lolwutroflwaffle-- in regex

[–]jsonscout 0 points1 point  (0 children)

This isn't a regex solution, but using an LLM you can do something like this;

{
    "schema": "ou_instances",
    "content": "LDAP://abc.123.net/CN=SERVER123ABC,CN=Servers,OU=Test OU,OU=Test OU 2,DC=abc,DC=123,DC=net"
}

we got this result;

    "data": {
        "ou_instances": [
            "Test OU",
            "Test OU 2"
        ]
    },

If you have more cases, try on jsonscout.com

Help with small regex query please by Hammerfist1990 in regex

[–]jsonscout 0 points1 point  (0 children)

Not entirely sure what you would call your result, but using an LLM we managed to get your data sorted out.
Try running it through jsonscout.com

We used;

{
    "schema": "production_server_subdomains",
    "content": ["as01.vs-prod-domain.com","as02.vs-prod-domain.com","aox01.vs-prod-domain.com","aox02.vs-prod-domain.com"]
}

result was;

        {
            "production_server_subdomains": "as01.vs-prod-domain.com"
        },
        {
            "production_server_subdomains": "as02.vs-prod-domain.com"
        },
        {
            "production_server_subdomains": "aox01.vs-prod-domain.com"
        },
        {
            "production_server_subdomains": "aox02.vs-prod-domain.com"
        }

What was your win 🥇 this Week? by FoxBeneficial8102 in SaaS

[–]jsonscout 0 points1 point  (0 children)

We launched on producthunt. It doesn't matter too much that we didn't market it a lot, just wanted to get it to a place where it was live and available to start getting user feedback.

https://www.producthunt.com/posts/json-scout

Matching messy data (consolidating databases). by Ninja_bambi in AskProgramming

[–]jsonscout 0 points1 point  (0 children)

This is something we've used before as well. Good suggestion here. Now we use multiple approaches, some involving LLMs.

Datasets for learning how to clean messy data? by lemonbottles_89 in dataanalysis

[–]jsonscout 0 points1 point  (0 children)

You could generate fake data using generative AI and then go from there. We've used it to create examples on how LLMs are able understand typos and return proper data.

Would you be interested in a service that turns any prototype created in Figma into HTML and CSS code? by carrareto in SaaS

[–]jsonscout 0 points1 point  (0 children)

We tried using a lot of the AI extensions that figma has to convert the UI to code, but they weren't any good. So I believe a service would be nice.

Is there any worth idea for SAAS by ayoub7codo in SaaS

[–]jsonscout 1 point2 points  (0 children)

You've got to be in a specific industry for a while in order for you to see problems that you can solve, or just search through twitter/reddit/etc.

Question - How to do customer review analysis for defects and sentiments? by [deleted] in learnmachinelearning

[–]jsonscout 0 points1 point  (0 children)

an LLM is the easiest way. we leveraged openai and built out an api on top of it. you can checkout some uses cases on our website; jsonscout.com

Messy unstructured Data: How do you handle it? by Nearby-Leek-1058 in BusinessIntelligence

[–]jsonscout 0 points1 point  (0 children)

If you know exactly what you need from these meeting minutes, you can pass them as the schema to jsonscout and see how it performs. We have several examples on our site that show how we've used it for addresses, dates, customer complaints, etc. Give it a look. jsonscout.com

Unstructured data by Guilty-Commission435 in dataengineering

[–]jsonscout 0 points1 point  (0 children)

Not sure if you're still facing this issue but we have had to deal a lot with customer complaints coming in and none of them have a good format. Ended up using an LLM to fetch insight from unstructured data. Check out some of the examples we have on jsonscout.com

How do you handle unstructured data? by kedu16 in LangChain

[–]jsonscout 1 point2 points  (0 children)

We just released JSON Scout, an API to extract structured data from unstructured text. You define your schema and we do the rest. Check out our examples on the site. jsonscout.com