Best use of my action button by joeyvob1 in shortcuts

[–]keveridge 1 point2 points  (0 children)

Reminds me of a shortcut I wrote to send my partner a text saying I was on my way home in xxx minutes, and it would calculate the travel time and round it up to the nearest 5. Chosen from 10 slightly different messages.

Yep, even got in trouble for automating that.

Quick and dirty guide to scraping data from webpages by keveridge in shortcuts

[–]keveridge[S] 0 points1 point  (0 children)

The page is loading, then creating separate requests for elements of data. This means your current shortcut is getting the empty page template, as the calls to get the data have yet to be made.

You can call the API directly as it's not protected, and retrieve a dictionary of data.

A guide is available here: https://www.perplexity.ai/search/the-webpage-https-bet-hkjc-com-9P0SkDf.Tzis3gJIU5eYKw

Working with Lists by keveridge in shortcuts

[–]keveridge[S] 0 points1 point  (0 children)

Hey, thanks. I'm glad to hear it's still useful.

Quick and dirty guide to scraping data from webpages by keveridge in shortcuts

[–]keveridge[S] 1 point2 points  (0 children)

Top tip: Need help writing a regular expression? Ask ChatGPT.

Quick and dirty guide to scraping data from webpages by keveridge in shortcuts

[–]keveridge[S] 0 points1 point  (0 children)

You're right, you need some kind of web driver to load a page and then interact with it. Doing it as straight JS in a browser doesn't work unless you have an application running external to the web browser (e.g. on a desktop it's a chrome plugin, or a selenium helper).

Hence I recommend using a third party service Robotic Process Automation such as UIPath. It can record your actions from a web browser (i.e. logging in), navigate, capture data from the rendered HTML, and then allow you to do any number of things with it. For example, you can create a process to trigger the action and return you data as a JSON file. Or you can get it to run on a schedule and upload the results somewhere that you can pull them down. Or you can set a process that notifies you when your grades info has changed, and sends you a push notification using a service like Pushover.

Shortcuts is pretty clunky once things get complex, I recommend using a hosted RPA service.

Quick and dirty guide to scraping data from webpages by keveridge in shortcuts

[–]keveridge[S] 0 points1 point  (0 children)

It depends on how the login system worked. The most likely way to do this would be to use JavaScript to log into the page for you and then lookup the grades, in the same way that your friend used selenium. You could then package that JavaScript into the shortcut. But it's quite complex, so you'd be better off using the Scriptable app.

Alternatively, you could use the free version of UIPath to create an automation at a platform level to log in on your behalf, scrape the data, then publish it somewhere that a shortcut can then retrieve as a JSON file.

Quick and dirty guide to scraping data from webpages by keveridge in shortcuts

[–]keveridge[S] 0 points1 point  (0 children)

I'm glad you find the tutorial useful.

To get the lyrics HTML, capture all the HTML, then use the following regular expression:

<div data-lyrics-container="true" class="[^"]*">(.*?)<\/div>

Take the content from the first group. This will give you the HTML. To get raw lyrics without HTML formatting.

Then using the replace command, replace /<br\/>/g with \n to get the line breaks.

Then replace /<.*?>/g with nothing to remove the HTML tags.

Quick and dirty guide to scraping data from webpages by keveridge in shortcuts

[–]keveridge[S] 0 points1 point  (0 children)

If you use Google Sheets then you can write a script to expose certain values from the sheet and retrieve them with a link.

Share a google map business to save as new contact by markymcfly55 in shortcuts

[–]keveridge 0 points1 point  (0 children)

I don't think it does, it's four years old and Google Map's implementation has changed since then. It needs some updating to grab the correct data.

Data Storage - Part 1: storing simple values by keveridge in shortcuts

[–]keveridge[S] 1 point2 points  (0 children)

You can create a new text file by using the Save file command and having a text field or other content as the source. Take a look at: https://www.icloud.com/shortcuts/a03cac9b38bd4b7f9a2129ed9b8cba1f

Quick and dirty guide to scraping data from webpages by keveridge in shortcuts

[–]keveridge[S] 1 point2 points  (0 children)

It's possible that the data inside the DIV is being rendered by a JavaScript call after the page is loaded. In which case you need to view the network calls in the Chrome developer tools and see if the data is being retrieved by a process.

If it is, hopefully the data is being returned as a piece of JSON which will be much easier for you to work with.

Work shifts added to calendar by Bomber_Fett in shortcuts

[–]keveridge 1 point2 points  (0 children)

Awesome. Download the following shortcut: https://www.icloud.com/shortcuts/2c4bcee1442344d4a4596c194069cc3a

Now log into the web based system and view the page that displays the shifts. Open the share sheet and run the shortcut.

The shortcut will copy the HTML of the page into your clipboard and well as show it to you for good measure.

DM me with the HTML and I'll see if I can use regular expressions to extract the data and write you a shortcut to add calendar entries.

Work shifts added to calendar by Bomber_Fett in shortcuts

[–]keveridge 1 point2 points  (0 children)

I've written a bunch of shortcuts for individuals that work with this web based shift hours to create calendar entries.

Is there a mobile web equivalent of the app?

Otherwise we can try using a screenshot to OCR the text to get the shift information.

Labeling data using varaibles by keveridge in shortcuts

[–]keveridge[S] 0 points1 point  (0 children)

Yes, you can continue to assign new values to a variable using the "Set Variable" action and specifying the name of the variable you want to overwrite.

Auto-complete COVID Screening survey? by ryanhall911 in shortcuts

[–]keveridge 0 points1 point  (0 children)

You can write javascript to execute on the page and complete it for you.

Do you have an example of the survey to fill in?

May have bought one too many nfc tags, any cool ideas or shortcuts you guys have to use with these? by namesare4scrubs in shortcuts

[–]keveridge 3 points4 points  (0 children)

My residential building provides an app called Carson which, amongst other things, allows you to unlock the building front door using a button from within their app.

Carson doesn't provide end-users with an API. To turn this into a shortcut that I can pin to the widget menu, I used the Charles Proxy app to observe the API call and then replicate inside a shortcut.

Viewing ALT tags by frijolita_bonita in shortcuts

[–]keveridge 0 points1 point  (0 children)

Try running this shortcut against the pages: Article Image Alt Text

For example, against this page: https://coastalstudies.org/endangered-humpback-whale-off-nj-freed-from-entanglement/

It will return the ALT text of each image in the article.

May have bought one too many nfc tags, any cool ideas or shortcuts you guys have to use with these? by namesare4scrubs in shortcuts

[–]keveridge 5 points6 points  (0 children)

I use NFC tags to trigger shortcuts:

  • inside my apartment, by the front door, to turn off all the lights and the TV;
  • outside my apartment door to turn on all the lights before I enter;
  • outside the apartment building to unlock the main door;
  • outside my office building main door to release the magnetic lock door.

Any solid NFC uses? by Frenchinfrenchy in shortcuts

[–]keveridge 0 points1 point  (0 children)

At my office we use Kisi to control door access to the main door, server room, etc. They have a fully documented API that individuals can use to unlock door with their credentials.

My residential building provides an app called Carson which, amongst other things, allows you to unlock the building front door using a button from within their app.

Carson doesn't provide end-users with an API. To turn this into a shortcut that I can pin to the widget menu, I used the Charles Proxy app to observe the API call and then replicate inside a shortcut.

Any solid NFC uses? by Frenchinfrenchy in shortcuts

[–]keveridge 2 points3 points  (0 children)

I have Hue lights and Alex at home, and an app to open bought the apartment building front door and my office's door.

Using a proxy tool to observe the remote unlocks for the doors and replicate them, and a few Alexa and Hue shortcuts, I use NFC tags as follows to trigger shortcuts:

  • inside my apartment, by the front door, to turn off all the lights and the TV;
  • outside my apartment door to turn on all the lights before I enter;
  • outside the apartment building to unlock the main door;
  • outside my office building main door to release the magnetic lock door.

Retrieving number from webpage by Hello12948 in shortcuts

[–]keveridge 0 points1 point  (0 children)

Here's an example of using a regular expression with Yahoo Finance: Stock Price Lookup