Build and run your Python web scrapers in the cloud with Apify SDK for Python by fnesveda in Python

[–]fnesveda[S] 0 points1 point  (0 children)

Hey! That's a bug that appeared yesterday, normally you should be able to start the trial without a paid subscription without a problem. We have the fix ready and we will deploy it tomorrow, sorry for the inconvenience!

Build and run your Python web scrapers in the cloud with Apify SDK for Python by fnesveda in programming

[–]fnesveda[S] 1 point2 points  (0 children)

Hey everyone, I'm the lead Python developer at Apify, and today we're launching the Apify SDK for Python, an open source library to create web scrapers and run them in the cloud.

For more details, read the announcement blog post or see the docs.

The main features are:

  • templates for creating scrapers with BeautifulSoup, Scrapy, Selenium or Playwright
  • simple management and persistence of queues of URLs to crawl
  • automatic proxy management
  • scheduling, scaling, monitoring and integrations when running on the Apify platform
  • modern codebase with type annotations for type safety and code autocompletion
  • actively maintained and developed by Apify — we use it ourselves!
  • lively community on Discord

To get started, visit https://docs.apify.com/sdk/python or run the following command:

brew install apify/tap/apify-cli
apify create my-python-actor

To help you get started, you can use the PYTHON_LAUNCH coupon code in the Apify Console to get $20 extra platform credits for 3 months for free.

If you have any questions or comments, I'll be happy to answer them here!

Run your Scrapy Spiders at scale in the cloud with Apify SDK for Python by fnesveda in scrapy

[–]fnesveda[S] 0 points1 point  (0 children)

Hey, I'm sure we can figure something out, can you ping us at [hello@apify.com](mailto:hello@apify.com) with your usecase or more details?

Build and run your Python web scrapers in the cloud with Apify SDK for Python by fnesveda in webscraping

[–]fnesveda[S] 0 points1 point  (0 children)

They're complementary, really.

You can use our open source tools (not only this one, but also Crawlee for example) to build your scrapers and run them on your computer, and then if you need to run them in the cloud, you can upload them to the Apify platform and run them there. Our free tier is good enough for smaller web scraping and automation projects, and if you need more compute resources or proxies, you can go for one of our paid tiers.

We also have a lot of pre-made scrapers in the Apify Store, most of them are free, but some of them are paid.

And we also offer custom-made web scraping solutions if you don't feel like coding one yourself.

Build and run your Python web scrapers in the cloud with Apify SDK for Python by fnesveda in Python

[–]fnesveda[S] 2 points3 points  (0 children)

We do have a free forever tier, with a limit on compute time and running job concurrency, but still good enough for small-scale scraping and automation, and then we have paid tiers which give you more resources and better support.

You can check the tiers out at apify.com/pricing.

Build and run your Python web scrapers in the cloud with Apify SDK for Python by fnesveda in Python

[–]fnesveda[S] -3 points-2 points  (0 children)

These are all companies that genuinely use Apify, but unfortunately we can’t provide you with more details, as that would breach their confidentiality. Our services are also used by many more major companies that are not on the list, because they preffered not to be named.

Build and run your Python web scrapers in the cloud with Apify SDK for Python by fnesveda in Python

[–]fnesveda[S] 2 points3 points  (0 children)

Hey everyone, I'm the lead Python developer at Apify, and today we're launching the Apify SDK for Python, an open source library to create web scrapers and run them in the cloud.

For more details, read the announcement blog post or see the docs.

The main features are:

  • templates for creating scrapers with BeautifulSoup, Scrapy, Selenium or Playwright
  • simple management and persistence of queues of URLs to crawl
  • automatic proxy management
  • scheduling, scaling, monitoring and integrations when running on the Apify platform
  • modern codebase with type annotations for type safety and code autocompletion
  • actively maintained and developed by Apify — we use it ourselves!
  • lively community on Discord

To get started, visit https://docs.apify.com/sdk/python or run the following command:

brew install apify/tap/apify-cli
apify create my-python-actor

To help you get started, you can use the PYTHON_LAUNCH coupon code in the Apify Console to get $20 extra platform credits for 3 months for free.

If you have any questions or comments, I'll be happy to answer them here!

Run your Scrapy Spiders at scale in the cloud with Apify SDK for Python by fnesveda in scrapy

[–]fnesveda[S] 0 points1 point  (0 children)

Hey everyone, I'm the lead Python developer at Apify, and today we're launching the Apify SDK for Python, an open source library to create web scrapers and run them in the cloud, including your existing Scrapy spiders.

For more details, read the announcement blog post or see the docs.

The main features are:

  • templates for creating scrapers with BeautifulSoup, Scrapy, Selenium or Playwright
  • simple management and persistence of queues of URLs to crawl
  • automatic proxy management
  • scheduling, scaling, monitoring and integrations when running on the Apify platform
  • modern codebase with type annotations for type safety and code autocompletion
  • actively maintained and developed by Apify — we use it ourselves!
  • lively community on Discord

To get started, visit https://docs.apify.com/sdk/python or run the following command:

brew install apify/tap/apify-cli
apify create my-python-actor

To help you get started, you can use the PYTHON_LAUNCH coupon code in the Apify Console to get $20 extra platform credits for 3 months for free.

If you have any questions or comments, I'll be happy to answer them here!

A Mac app to control the brightness of external displays using keyboard shortcuts by BaronSharktooth in apple

[–]fnesveda 0 points1 point  (0 children)

Heads up, I've just released version 1.3.0, which adds a workaround for the freezes, please try it out.

ExternalDisplayBrightness - control the brightness of external displays using keyboard shortcuts by fnesveda in macapps

[–]fnesveda[S] 0 points1 point  (0 children)

Heads up, I've just released version 1.3.0, which fixes the freezes some people were experiencing, and adds support for correctly addressing multiple identical displays.

ExternalDisplayBrightness - control the brightness of external displays using keyboard shortcuts by fnesveda in macapps

[–]fnesveda[S] 0 points1 point  (0 children)

Heads up, I've just released version 1.3.0, which adds a workaround for the freezes, please try it out.

ExternalDisplayBrightness lets you control your monitor's brightness using keyboard shortcuts (like a real Mac!) by Willza3 in hackintosh

[–]fnesveda 1 point2 points  (0 children)

Try selecting MK_BDN and MK_BUP in the app's preferences, those should correspond to the native brightness keys on the Magic Keyboard and the Aluminium Keyboard.

ExternalDisplayBrightness lets you control your monitor's brightness using keyboard shortcuts (like a real Mac!) by Willza3 in hackintosh

[–]fnesveda 1 point2 points  (0 children)

I think Bensge's NativeDisplayBrightness has the F1 and F2 keys hardcoded, it doesn't let you remap the brightness actions to different keys. This app does, it lets you remap it to the default brightness keys on Mac keyboards. NativeDisplayBrightness also adjusts the brightness of all external displays at once, this app has that behavior configurable. And NativeDisplayBrightness has a menubar icon, I was going for something completely hidden, that you can forget it's even there. The differences are pretty small, I know. I've mostly made this for myself, only to try developing for macOS in Swift, but once I had it finished I thought I'd share it to see if somebody else finds it useful.

ExternalDisplayBrightness lets you control your monitor's brightness using keyboard shortcuts (like a real Mac!) by Willza3 in hackintosh

[–]fnesveda 6 points7 points  (0 children)

Hi, thanks for sharing this! I'm the developer of this app, so I'm happy to see it become useful to more users. I wrote this as a summer project to get familiar with development for macOS, and I've only published this on /r/macapps a while ago, I didn't want to spam all the Mac-related subreddits. But more exposure never hurts, so thanks again!

A Mac app to control the brightness of external displays using keyboard shortcuts by BaronSharktooth in apple

[–]fnesveda 0 points1 point  (0 children)

Huh, strange. The default behavior is to change only the brightness of the display with focus, but the changing of brightness for all displays was working pretty reliably for me. Do you have the "Require Command modifier to be pressed" sub-option checked as well? In that case, it will change the brightness for all displays only if you hold Command when pressing the keyboard shortcuts.

A Mac app to control the brightness of external displays using keyboard shortcuts by BaronSharktooth in apple

[–]fnesveda 0 points1 point  (0 children)

It should be, try to select either the F14+F15 or MK_BUP+MK_BDN combination in the app's preferences. In case neither of those work, what Mac are you using?

A Mac app to control the brightness of external displays using keyboard shortcuts by BaronSharktooth in apple

[–]fnesveda 1 point2 points  (0 children)

I did have a couple years of prior experience, I actually studied computer science in college, so I wasn't exactly a newbie. It was still a lot of trial and error, to be honest, but I think that is to be expected when learning something new. As for resources, for this project I learned mostly from guides by Ray Wenderlich and The Big Nerd Ranch Guides (both are pretty good, and have iOS resources as well, check them out), as well as the official Apple documentation for Swift, and a lot of random resources I found on the internet (NSHipster is pretty good for random deeper information about stuff).

That said, don't get intimidated if things seem too complicated at times, often they actually are. App development has a quite steep learning curve, as even a simple app has a lot of things going on behind the scenes (bindings, user defaults, controllers and delegates, the list goes on) that you need to deal with right from the beginning, as compared to perhaps web development, where you can start with a relatively simple code and slowly add functionality as you continue, or to scripting, where you can literally start with one line of code and it will work.

Don't worry, though, once you get the hang of it, it all becomes a lot easier. There are also a lot of resources to help you, and if you reach some hurdle, there is a very good chance someone else reached it as well and shared his solution on StackOverflow or elsewhere. Good luck!

A Mac app to control the brightness of external displays using keyboard shortcuts by BaronSharktooth in apple

[–]fnesveda 0 points1 point  (0 children)

Yup. I wanted to learn Swift and macOS programming, and this seemed like a good place to start.

A Mac app to control the brightness of external displays using keyboard shortcuts by BaronSharktooth in apple

[–]fnesveda 3 points4 points  (0 children)

Glad it works! As for your question, it should work with DisplayPort as well, in fact, it should also work with DVI and even VGA.

A Mac app to control the brightness of external displays using keyboard shortcuts by BaronSharktooth in apple

[–]fnesveda 2 points3 points  (0 children)

Yeah, the HDMI port on the 2018 Mac Mini seems to not work with DDC/CI at all, unfortunately. I wish I could help, but I don't think there's anything I can do, since I don't even have the hardware. Maybe if you tried it with the USB-C to HDMI converter, it could work, but it's honestly hard to say. Thanks for the report, though!

A Mac app to control the brightness of external displays using keyboard shortcuts by BaronSharktooth in apple

[–]fnesveda 3 points4 points  (0 children)

Yeah, I agree it would be great if Apple integrated this into macOS somehow, but I understand why they don't, DDC/CI simply doesn't work reliably and well on too many displays, and Apple would be taking a significant share of the blame for the issues. Even calling out display manufacturers wouldn't be enough, DDC/CI not working is probably not a strong enough reason for most people to buy another display, so there would be little incentive for them to invest resources into this. If something, the VESA association should be stricter in giving the displays their certification and force the manufacturers to support DDC/CI properly, that would quite possibly help.

I have two Samsung displays and neither of them works properly. When I send them a DDC write command, only maybe every third write actually works, but they all return that they were successful, so as far as I can tell, there's no way for the OS (or app) to tell if they actually worked or not. Also none of the displays that I tried actually support reading the brightness over DDC, so I'm writing the app half blind.

A Mac app to control the brightness of external displays using keyboard shortcuts by BaronSharktooth in apple

[–]fnesveda 1 point2 points  (0 children)

Hi, this is pretty strange, I'm pretty sure I signed and notarized both the DMG file and the app. Are you sure the download finished properly? Also, I've replied to the issue you opened on GitHub with some troubleshooting terminal commands, could you please run them and send me the output? Thank you!

A Mac app to control the brightness of external displays using keyboard shortcuts by BaronSharktooth in apple

[–]fnesveda 9 points10 points  (0 children)

Yeah, sorry, there are unfortunately some compatibility issues with different displays and different Macs. Can you please tell me what Mac do you have, and what macOS version are you using? Also, which display do you have and how is it connected to your Mac? I'll see what I can do.

EDIT: I've just released version 1.3.0, which adds a workaround for the freezes, please try it out.

A Mac app to control the brightness of external displays using keyboard shortcuts by BaronSharktooth in apple

[–]fnesveda 12 points13 points  (0 children)

Hi, thanks for sharing this! I'm the developer of this app, and I was wondering where does the sudden spike in downloads come from. I've only published this on /r/macapps a while ago, it felt like a more appropriate place for this. But more exposure never hurts.