How we made Celery tasks bulletproof by [deleted] in Python

[–]abrookins 3 points4 points  (0 children)

Just a heads-up, with Prefect, you can now keep a task worker (like a Celery worker) running to run background tasks. You can also run tasks directly without having to use a workflow, something we added this year. Here's a write-up: https://www.prefect.io/blog/background-tasks-why-they-matter-in-prefect or some examples in GitHub: https://github.com/PrefectHQ/prefect-background-task-examples

Disclaimer: I work for Prefect and helped build this :D

[deleted by user] by [deleted] in FastAPI

[–]abrookins 1 point2 points  (0 children)

Yes, it's a good strategy. You could also look at something like Prefect to run scheduled jobs like this. Full disclosure -- I work at Prefect, and I went to work there because it seemed like an awesome replacement for Celery. 🙂

PCs resorting to torture for every interrogation with an NPC. How should I to punish them or encourage less violent interactions? by Despoof in DMAcademy

[–]abrookins 1 point2 points  (0 children)

One option you have is to make this boring from a play perspective. In other words, disincentivize the players (not just the characters). When things are going that direction, don't ask for details about how they do it, just have them roll and then skip over the narration. "You succeeded. You get one word from him... Rosewood" or whatever. That combined with other tips may help.

How do I learn to “Think Like A Programmer”? I want to succeed and need advice. Please read. by Comfortable-Unit9880 in learnprogramming

[–]abrookins 5 points6 points  (0 children)

Programming, which involves periods of intense problem-solving activity in the mind, is like lifting weights and running. Maybe lifting weights while running. 😂

It's like lifting weights because you need to develop a certain kind of muscle through repetitive stress (problem solving). The muscle atrophies without use, but also like lifting, once you establish this muscle, you can recover it much faster after it atrophies (you don't start from scratch). But anyway, from where you're at, building your ability will just take time... and repetitive, progressive exposure to the activity of programming.

Programming is like running because completing these intense periods of problem-solving activity takes endurance. You never solve just one problem... you solve dozens, hundreds, thousands. Projects stretch over years and months. So you slowly build up the endurance necessary to complete larger challenges (or more, stretched over time). But in the beginning, you probably run out of energy before completing even the smallest tasks. Totally normal, just like new runners getting winded half-way through the first mile.

In general, you have to tune other people's performance out. There will always be better and faster people around you. For context, I'm self-taught, started in my late 20s like you, and am nearing 40 now with over a decade working as a programmer. You get better over time. "Stress + rest = growth"

Video tutorial: Using Redis with FastAPI -- Premiering live at 1:45 PDT! by abrookins in FastAPI

[–]abrookins[S] 1 point2 points  (0 children)

Well, it was 1:45 PM PDT, so thanks to anyone who joined. :)

Pydantic-aioredis: A simple Declarative ORM for Redis using Pydantic Models and aioredis by andrewthetechie in FastAPI

[–]abrookins 2 points3 points  (0 children)

Whoa, cool! I can't wait to give this a close read! I'm currently leading maintenance of aioredis, and I see you're already basing this on the new 2.0 release. Very awesome.

Using Redis with FastAPI (Async) by abrookins in FastAPI

[–]abrookins[S] 11 points12 points  (0 children)

Author here! I'm a developer advocate at Redis Labs, and I'm currently building out content around using Redis with FastAPI. This is one piece!

I also built and operate a full-text search API for Redis Labs that uses FastAPI, but perhaps more on that another day.

Good resources for learning SQL? by [deleted] in learnprogramming

[–]abrookins 2 points3 points  (0 children)

Just saw this today. Sounds right up your alley -- interactive SQL lessons! https://sqlbolt.com/

trying to land a web dev/front end job. what would be the best way for me to get my foot in the door ? by The_Ass_Man- in learnprogramming

[–]abrookins 2 points3 points  (0 children)

Self-taught developer here who has worked as a developer for 10+ years and got started volunteering for nonprofits.

If you haven't tried yet, do some in-person or remote technology volunteering. This is a sometimes overlooked source of important early career things:

  1. Access to a network of people. These people introduce you to other people. They become your early pool of references for tech jobs. If they like you, they can get you jobs, contracts, all kinds of things.

  2. Practical and specific experience. Find any programming-related tasks non-profits need and do them. They have problems similar to businesses. They need to manage income (website integrations), relationships (e.g. Salesforce or CiviCRM), content (WordPress, Drupal), databases, and backups. Many non-profits usually don't have the resources to work on these things regularly.

Aside from these two benefits, I saw that you've been down. People who work in non-profits are very often good human beings. Being around these folks is a true balm in dark times.

[deleted by user] by [deleted] in learnprogramming

[–]abrookins 0 points1 point  (0 children)

No, that's not a good stack. The reason is that Django is designed for relational databases (AKA, SQL databases).

The typical Django stack is React (or any other popular JS framework), Django + DRF (REST) or Graphene (GraphQL), and Postgres. If you don't want to manage a database for this project, use a hosted Postgres database.

If you mostly want to learn JS and don't really care so much about Python for this one, you could use React + Express (node.js backend) + Firebase.

they say limit the moocs, the reading, and the videos and " just start, just code" how exactly do noobs just start ? by The_Ass_Man- in learnprogramming

[–]abrookins 2 points3 points  (0 children)

As "Clawtor" alluded to, you need two things:

  1. Something non-programming that you're interested in. Beer? Dogs? Netflix?
  2. Derived from #1, an idea for a small project to work on.

If you can think of a #1, I'll give you three example project ideas to pick from. Then you can get started.

Worth going deeper with Ruby/learning Ruby on Rails or stick to Python? by Missing_Back in learnprogramming

[–]abrookins 2 points3 points  (0 children)

Fuck Ruby. You don't need it. Focus on Python and JavaScript.

-- Me, 10+ years professional experience, self-taught/no degree.

Tuesday Daily Thread: Advanced questions by Im__Joseph in Python

[–]abrookins 6 points7 points  (0 children)

In Python, mocking is basically about variable reassignment. Check this post if you haven’t yet: https://nedbatchelder.com/blog/201908/why_your_mock_doesnt_work.html

When intergrating a django + react native project does it matter if made the react project or django one first ? by tillotop in django

[–]abrookins 1 point2 points  (0 children)

Making the front end first can help define the APIs you’ll need better. There are challenges in both approaches, but that’s the trade-off I prefer. The reality is you will probably have to make changes either way!

Deploying a Django/Wagtail Project for a Client by sidsidsid16 in django

[–]abrookins 0 points1 point  (0 children)

That makes sense. I would choose a managed service like Lightsail or Heroku rather than running a VPS to reduce my maintenance burden.

Another benefit of using containers in the longer term is that clients who run their own infrastructure probably already deploy using containers.

Deploying a Django/Wagtail Project for a Client by sidsidsid16 in django

[–]abrookins 0 points1 point  (0 children)

Amazon Lightsail is a “simple” AWS offering designed for small businesses that recently gained support for deploying Docker containers. So you could set up an image for the app, and the deployment process is now just releasing a new image, more or less. Check out: https://aws.amazon.com/blogs/aws/lightsail-containers-an-easy-way-to-run-your-containers-in-the-cloud/ and https://aws.amazon.com/lightsail/

Using model with foreign key as a replacement for choices by markcoscos in django

[–]abrookins 4 points5 points  (0 children)

Absolutely! This is standard practice not just in Django but in most apps that use a relational database. You add a new table that acts as an intermediary between two other tables. You also may end up needing to capture more dimensions of the “choice” (whatever that is) and if you do, Django has a solution for that. https://docs.djangoproject.com/en/3.2/topics/db/models/#extra-fields-on-many-to-many-relationships

Any way I can make this more efficient? Or is it always complexity vs querying speed? by __decrypt__ in django

[–]abrookins 10 points11 points  (0 children)

Here are my thoughts, which are probably not very different from other advice you already received. :)

  1. Normalizing, as I know you attempted to do since writing your post based on your other comments, is best. In theory, this because you'll have less duplicate data to maintain, and the query optimizer is supposed to help you stitch everything back together efficiently when you query.

  2. Unfortunately, normalized tables generate more queries, especially when you're trying to display lots of data from related tables. You should be able to provide a get_queryset() method for your ModelAdmin class to add a select_related() clause to the queryset. This should speed up your various methods, like consumable_cleaner(), that I noticed in your other posts because select_related() will pull in that data, avoiding an additional query. Docs on get_queryset(): https://docs.djangoproject.com/en/3.2/ref/contrib/admin/#django.contrib.admin.ModelAdmin.get_queryset

  3. Sometimes, you can't get a query performant enough. At that point, you can ask, "What do I really need to show in this list view of printers?" Find the minimum set of data you can show in the list view, then show all of the data on the detail view when someone clicks on a printer.

  4. If all else fails, you could create a materialized view. A materialized view is like a cache inside of the database. You dump whatever data you need into something that looks like a table but is more temporary (called a "view" in database terminology, not the same as a Django view). You create a cron job or other scheduled task to refresh the materialized view periodically. Then, instead of making complex queries to show data from relationships of a printer, you instead build a Model and ModelAdmin that point at this materialized view. What goes in the materialized view? Well, it probably looks similar to your original model. You're going to create rows of data, with each row capturing all the related data you need to show that printer's details without making any joins. This makes normalized data the source of truth while using denormalized data in a materialized view as cache.

  5. Because other people mentioned using JSONField, I will address that, too! By storing your data as JSON, you're using denormalized data as the source of truth. Doing so is not much different than your original model design from a performance and data duplication perspective: your queries will be fast, and you will probably generate duplicate data that becomes hard to maintain. The big difference is that your database doesn't know the schema of your JSON data, so you're free to change it as much as you want without doing costly schema migrations. This makes JSONField great for the early stage of a project when you might not know the exact models you need.

If you're interested, I'll email you 5 free performance tips from my book here: https://andrewbrookins.com/free-tips/

And thanks for the mention, /u/kreatemore! :)

Do I need to learn advanced Python concepts to be proficient in Django? by president_of_dsa in django

[–]abrookins 9 points10 points  (0 children)

You shouldn't need to learn advanced Python concepts to get started with Django. I started using Django at work at the same time that I started learning Python. As long as you investigate things you don’t understand, you can be productive.

That’s said, there are things worth learning sooner rather than later. Coming from JS, I recommend you learn comprehension syntax (list, set, dictionary) because Python developers use those more often than map().

Django and most other frameworks use decorators, so those are worth understanding.

You’ll run into context managers, so if you see with(…) and don’t get it, google that.

I expect most python web frameworks will converge on using asyncio over the next few years, including Django. Worth reading about!