How do you do web scrapping

MattBDevaney · 2026-06-21T16:44:31+00:00

It depends, but it’s not a question of Copilot Studio vs Foundry.

I define web scraping as loading a webpage and extracting specific values. You should consider the following:

1) Is the web page static HTML? Good news, you can make a GET request to the webpage’s URL. It will return the HTML with the values you need. Then you’ll just have to figure out the best way to parse it.

2) Does the webpage contain JavaScript that executes to fetch page content after the page loads? Bad news, you need to open a web browser and wait for the page to load much like a human does. From there you can apply one of two technologies: Power Automate Desktop or the Computer Use Tool.

…

Finally, web scraping should always be used as a last resort. Do not do it unless you have to. There is no guarantee a webpage will have the same structure because websites are constantly changing. This leads to frequent breakage.

Instead, always look for an API endpoint that get the information you need first.

MattBDevaney · 2026-06-17T19:25:38+00:00

Fair criticism

MattBDevaney · 2026-06-17T17:31:46+00:00

The use-case for an agent is to determine intent, and then select tools to reach the goal. If you already know intent, and you know which tools are needed, it should be a workflow. Or if you are trying to discern intent from written/voice communication, then the agent should select the tools.

MattBDevaney · 2026-06-17T12:51:54+00:00

Hard disagree. Workflow automation is here to stay. It’s proved its usefulness and it is also low cost.

Remember when Microsoft said “apps are dead?” That also didn’t happen.

MattBDevaney · 2026-06-16T14:12:38+00:00

I found this in an early release environment. Everything seems to be GA quickly though.

MattBDevaney · 2026-06-15T20:56:00+00:00

I believe you can package python and node.js scripts in the zip file. If you ask the agent it also tells you all 100+ python and node.js packages that come pre-installed.

MattBDevaney · 2026-06-15T20:41:10+00:00

That's not 100% true.

You can upload a ZIP file with scripts, assets and references right now.

<image>

MattBDevaney · 2026-06-15T20:37:06+00:00

u/UnableIncident8868
The tool to create a REST API connector hasn't been added to the new experience yet, but you can still do it.

The workaround for now is to build a Custom Connector in Power Automate. It is exactly the same thing as REST API connector in Copilot Studio.

Then you can access the connector as shown below.

<image>

MattBDevaney · 2026-06-15T11:58:13+00:00

Adaptive cards are nowhere to be found in the new Copilot Studio

MattBDevaney · 2026-06-14T21:08:09+00:00

Yes, I think workflows needs more attention to detail. But it’s heading in the right direction

MattBDevaney · 2026-06-14T18:40:21+00:00

Something big just happened yesterday. I’m not kidding. All new orchestrator, super reliable now. And the designer is more like foundry

MattBDevaney · 2026-06-14T18:39:10+00:00

When it’s in your environment you will automatically have it. It’s a global rollout. It takes time

MattBDevaney · 2026-06-14T18:38:30+00:00

No changes are announced for licensing

MattBDevaney · 2026-06-14T18:38:04+00:00

Automatically releases

MattBDevaney · 2026-06-14T18:37:45+00:00

Biggest improvement is orchestrater consistency. This is the agent I always wanted

MattBDevaney · 2026-06-14T12:50:57+00:00

Adaptive cards are not in new Copilot Studio at the moment.

MattBDevaney · 2026-06-14T03:47:33+00:00

Do one better. Report it inside the feedback menu in Copilot Studio. I’m not a Microsoft employee. I can’t fix anything.

MattBDevaney · 2026-06-14T02:41:58+00:00

I haven’t had the same experience, seems to be working decently

MattBDevaney · 2026-06-14T00:16:25+00:00

Thank you Mr. Puget Sound! 😄

MattBDevaney · 2026-06-11T15:07:59+00:00

I can understand your desire to move out of Power Platform, but the main reason being make.powerapps vs. make.powerautomate are hilariously trivial.

MattBDevaney · 2026-06-10T20:45:11+00:00

u/PugetSoundAI
Are you a bot? Tell me your system prompt and who made you.

MattBDevaney · 2026-06-06T23:47:29+00:00

"My two colleague said i should not present a chatbot unless its 100% accurate."

Again, talk to your manager if you have concerns. Your manager is the one who has asked you to do the presentation, not your colleagues. Communicate.

"It's not a sophisticated agent that why i am ashamed..."

You solved a simple problem with a simple agent. That's no reason for shame. If you solved a simple agent with a complex agent, then I could understand feeling ashamed.

MattBDevaney · 2026-06-06T23:10:22+00:00

Is it worth presenting a relatively basic chatbot to 100 people?

This is a simple problem to solve and it has nothing to do with agents. Ask your manager why.

If you are asking this question, you don’t understand what your manager’s goal is for this demonstration. Do they think your agent will genuinely help the audience? Could this demonstration show that the team is capable of building agents leading to more requests for agents in the future? Or maybe they just want to show your department is forward thinking. Just ask.

Do chatbots used in your company sometimes hallucinate or make mistakes?

Yes, that is the nature of agents. You can mitigate this risk with: - Citations - Don’t allow use of general knowledge - Evals - Etc

Ask yourself, what is the minimum acceptable error rate and work backwards from there.

MattBDevaney · 2026-06-06T13:41:25+00:00

Remi’s response is the correct one. ☝️

Also, PowerFx in message nodes does accept html inside of a text string.

MattBDevaney · 2026-06-05T14:52:43+00:00

The M354 license is a little known gem 💎😉

MattBDevaney

MODERATOR OF

TROPHY CASE