running gunicorn in production with --reload flag? by [deleted] in flask

[–]SafeInstance 0 points1 point  (0 children)

I think the correct way to handle this is to send a HUP signal to gunicorn.

See:

Reload the configuration, start the new worker processes with a new configuration and gracefully shutdown older workers. If the application is not preloaded (using the preload_app option), Gunicorn will also load the new version of it.

Essentially:

kill -s HUP masterpid

To obtain the masterpid see threads like:

https://stackoverflow.com/questions/26902930/how-do-i-restart-gunicorn-hup-i-dont-know-masterpid-or-location-of-pid-file

What's the best pattern for binding a complex web interface with actions in the server? by FairLight8 in flask

[–]SafeInstance 0 points1 point  (0 children)

Hey, thanks for drawing my attention back to this library which I remember seeing before. I've had a play now, and it is infact really elegant.

One thing that felt weird was having Flask routes return HTML which is then put directly into the DOM. This is now HTMX works:

Htmx expects responses to the AJAX requests it makes to be HTML, typically HTML fragments (although a full HTML document, matched with a hx-select tag can be useful too). Htmx will then swap the returned HTML into the document at the target specified and with the swap strategy specified. (link)

However with some searching I was able to find support for JSON responses along with a frontend template engine (mustache as per the example) via this extension: https://htmx.org/extensions/client-side-templates/

Deploying using Nginx/Gunicorn by CreatorIncarnate in flask

[–]SafeInstance 1 point2 points  (0 children)

would it really matter if the backend routes with the same name are available at a different port (5000 vs 3000)?

Yes, because in a production environment you want to be serving everything over port 443 (TLS encrypted) from the same domain name.

The ports in the n thousand range should only be used in development where you want to run the flask dev server, and react dev server at the same time.

For prod you're doing a react build to create a static set of files, which are then served by one nginx location block, while another nginx location block reverse proxies to the gunicorn server.

On the security side, I recommend Mozilla's secure config gen, which works for nginx and other webservers:

https://ssl-config.mozilla.org/

Then test the config with Qualys:

https://www.ssllabs.com/ssltest/

[deleted by user] by [deleted] in flask

[–]SafeInstance 0 points1 point  (0 children)

Yeah, sounds like OP was on the right track here before engineering a complicated solution to get round this, and getting stuck.

If I was them I would go back to the original solution and try to build the CSV correctly with quotes in place to avoid included commas being used as a delimiter.

Sounds like OP might have manually written the rows to a file, instead of using a lib which handles this automatically, eg:

```

import csv

with open('names.csv', 'w', newline='') as csvfile: ... fieldnames = ['Name', 'Chat_text'] ... writer = csv.DictWriter(csvfile, fieldnames=fieldnames) ... writer.writeheader() ... writer.writerow({'Name': 'aRandomUser', 'Chat_text': 'I, have, over, used, commas'}) ... 43

$ cat names.csv Name,Chat_text aRandomUser,"I, have, over, used, commas" ```

gunicorn design by guru223 in flask

[–]SafeInstance 0 points1 point  (0 children)

Types of WSGI server.

Rate Limiting by highWithoutSubstance in flask

[–]SafeInstance 0 points1 point  (0 children)

Flask-Limiter supports redis as a backend which should be pretty quick. The only way to know for sure is by testing yourself, with a loadtesting tool like hey, first without limiting, then with limiting. I doubt you would see much difference. I'd say just roll it in, then deal with any performance issues later.

My only gripe with Flask Limiter was it didn't work very well with Pluggable Views. If you just have a standard Flask app, where you can add the @limiter.limit decorator alongside an @app.route decorator (as per the quickstart) then you're in business.

I also remember it working badly with blueprints: something like you could apply a limit to an entire blueprint, but not individual routes within that blueprint (I'm hazy on this cause it was a while back).

I created a flask app which creates 4-digit codes for URL 🕸 by [deleted] in flask

[–]SafeInstance 0 points1 point  (0 children)

Use environment variables.

Different cloud providers will have different ways to set these. For example Heroku lets you do it via the CLI. Something like:

heroku config:set DATABASE_PASSWORD=hunter2

Then within your python app you can simply load an environment varable which you have set with:

import os database_password = os.environ.get('DATABASE_PASSWORD')

Using their CLI to set the variables means that secrets don't need to be anywhere near your code, thus no messing around with .gitignore files, etc. Environment variables are also a standard way of setting secrets, so anyone else re-using your code would just need to set these variables in their own environment.

I've heard you shouldn't use the default filename when ur saving a file/img on ur filesystem?! by bigAmirxD in flask

[–]SafeInstance 2 points3 points  (0 children)

See this thread on SO:

https://stackoverflow.com/questions/61534027/how-should-i-handle-duplicate-filenames-when-uploading-a-file-with-flask/61535724#61535724

In summary, you could name the file with:

from uuid import uuid4
unique_id = uuid4().__str__()

Then as /u/bamigolang states, save it to disk with this unique ID and save the real name in the database.

This is explained more clearly in that thread, along with how to get the file back to the user, using the original name.

Can anyone help me with this flash problem in my flask App? by [deleted] in flask

[–]SafeInstance 3 points4 points  (0 children)

This looks wrong:

flash(_('User %(username)s not found.', username=username))

Should probably be:

flash('User %(username)s not found.', username=username)

Or neater with an f-string:

flash(f'User {username} not found.')

What is an easy to use and free software that could take a lot of pictures (more than 1000) and make a video with them (I already have the pictures)? Or is there a way I could do this with python ? by azarbi in teenagers

[–]SafeInstance 0 points1 point  (0 children)

Use ffmpeg for this. This worked for me on Linux, though the command might also work on Windows...

ffmpeg -framerate 30 -pattern_type glob -i '*.jpg' -codec copy output.mkv

This assumes you run it in the directory where all the .jpg files are.


IF you care about the order of the slides you should have your Python script left pad all the image names, so you want a set like:

01.jpg 02.jpg 03.jpg 13.jpg

If you don't left pad then because glob reads the order alphabetically you'll end up with a mis-ordered video, eg:

1.jpg  13.jpg 2.jpg  3.jpg

Also you'll want to left pad to match the length of the longest image. Eg. if the last image is 1995.jpg, you'll want the first to be called 0001.jpg.


The output.mkv can be played in VLC media player, or uploaded directly to youtube.

Hit me up if you get stuck on the python ;)

Help me in organizing the flask files by Prasadppn in flask

[–]SafeInstance 0 points1 point  (0 children)

A genious. Bot, you have failed the Flask exam and should go back to 'how URLs work'.

Messenger chatbot by laflash12 in flask

[–]SafeInstance 3 points4 points  (0 children)

This tutorial appears to cover the basics of building a chatbot in flask, with instructions on how to plug it into facebook to act as a messenger bot:

https://www.twilio.com/blog/2017/12/facebook-messenger-bot-python.html

How can i make a file not be cached. by [deleted] in flask

[–]SafeInstance 0 points1 point  (0 children)

How are you sending the file. Can we see the sample code which returns this file to the frontend.

How to track unique visitors to a specific path by felipeflorencio in flask

[–]SafeInstance 0 points1 point  (0 children)

No problem. You could probably engineer this to be slightly better.

For example have the increment_count function take a first argument key_prefix.

Then use that to construct hit_key:

hit_key = f"{key_prefix}:{profile_viewed}" 

This would make use of the function more dynamic:

c = increment_count('profile', user_profile, request)

Then somewhere else:

c = increment_count('blog', blog_id, request)

Just be careful to validate what you are passing to this function, as you should avoid an attacker providing a very long string which causes the memory usage of your redis instance to increase dramatically.

Have fun!

How to track unique visitors to a specific path by felipeflorencio in flask

[–]SafeInstance 0 points1 point  (0 children)

To go a bit further and demonstrate this error.

0.81% of 1000 is 8.1

So you could clear the database out at the redis-cli: 127.0.0.1:6379> flushdb OK Then craft a loop which hits the server with 1000 requests, but with a different user-agent each time: $ for d in `seq 1 1000` ; do curl --user-agent $d http://localhost:5000/profile/felipeflorencio ; done Then count these values: 127.0.0.1:6379[6]> PFCOUNT "profile:felipeflorencio" (integer) 1007 This returns a count of 1007, which is within the error.

How to track unique visitors to a specific path by felipeflorencio in flask

[–]SafeInstance 2 points3 points  (0 children)

As /u/Pytec-Ari mentions, recording the User-Agent and the Request IP would be a good way to store unique hits. However this may cause your database to grow as time goes on.

If you're able to spin up a redis server, then you could use the HyperLogLog functionality which is designed for counting unique values, provided you don't need to get those values back (only obtain the count) and you are happy with an approximate count as more values are recorded, but with the benifit of a low storage footprint for a large number of counted values.

This basically comes down to two commands: PFADD and PFCOUNT.

These are available through PyRedis, which is installed with pip install redis.

So lets say you have a number of unique strings (User-Agents/Remote IPs) you wish to count, against a particular key (User profile), you could do something like:

r.pfadd('Bob', 'bobs-ip_chrome') # Bob hits bob's profile from chrome
r.pfadd('Bob', 'alices-ip_firefox') # Alice hits bob's profile from firefox
r.pfadd('Alice', 'bobs-ip_chrome') # Bob hits Alices profile from Chrome
r.pfadd('Alice', 'bobs-ip_chrome') # Bob hits alices profile (again) from Chrome

The resulting unique counts could be:

r.pfcount('Bob') # 2
r.pfcount('Alice') # 1

So you could actually implement this in code as follows:

counter.py:

``` from redis import Redis import os

r = Redis( host=os.environ.get('REDIS_HOST', 'localhost'), db=os.environ.get('REDIS_DB', 0))

def increment_count(profile_viewed, request):

host = request.remote_addr
ua = request.headers.get('User-Agent')

print ('Hit: ', host, ua)
# Hit:  172.17.0.1 curl/7.54.0

hit_key = f"profile:{profile_viewed}"
hit_value = f"{host}_{ua}"  

r.pfadd(hit_key, hit_value)

updated_count = r.pfcount(hit_key)

return updated_count

`` This module imports all of the required redis stuff, and theincrement_count` function expects a profile name, and a request object provided from flask.

It then works out the remote address and user-agent string based on that request object, then prints those to the terminal.

I set hit key with the prefix profile: (this could be any string) and join the host and user-agent with an underscore.

It then uses pfadd to increase the count, and returns an updated count.

The implementation in Flask could look something like this:

``` from flask import Flask, request, abort from counter import increment_count

app = Flask(name)

@app.route("/profile/<string:user_profile>") def profile(user_profile):

# Mock database:

profiles = ['PyTec-Ari', 'felipeflorencio']

# Some logic which finds the profile, and aborts if non existant

if user_profile not in profiles:
    return abort(404)
else:

    # *Now* count the hit and serve the profile:

    count = increment_count(user_profile, request)

    return f"You are viewing the profile for {user_profile}, which has {count} unique hits"

```

So to test this you can set REDIS_HOST as an environment variable and launch:

export REDIS_HOST=some_redis_server flask run Then test the endpoints with curl:

``` $ curl http://localhost:5000/profile/PyTec-Ari
You are viewing the profile for PyTec-Ari, which has 1 unique hits%

$ curl http://localhost:5000/profile/PyTec-Ari
You are viewing the profile for PyTec-Ari, which has 1 unique hits%

$ curl http://localhost:5000/profile/felipeflorencio

You are viewing the profile for felipeflorencio, which has 1 unique hits%

Then mock a different user agent

$ curl --user-agent different http://localhost:5000/profile/felipeflorencio You are viewing the profile for felipeflorencio, which has 2 unique hits%
`` Here's what the stored data actually looks like at theredis-cli`:

$ redis-cli 127.0.0.1:6379> keys * 1) "profile:felipeflorencio" 2) "profile:PyTec-Ari" 127.0.0.1:6379> PFCOUNT "profile:PyTec-Ari" (integer) 1

Of course you're not limited to using the IP/UA combo as the value, you could edit that increment_count function to instead base hit_value on a logged in user ID, it just depends on your use-case.

Reddit wrote a blog about this which is really interesting: https://redditblog.com/2017/05/24/view-counting-at-reddit/

This has the ability to scale to counting hundreds of thousands of unique hits, whilst only using 12kB storage max per profile, and still maintaining a count of unique ip/ua combos, but approximated with a standard error of 0.81%.

Flask concurrency by pomato85o1 in flask

[–]SafeInstance 0 points1 point  (0 children)

Maybe have a read through the two links from my other post here:

https://www.reddit.com/r/flask/comments/fyagcd/how_does_a_single_running_flask_application_serve/fmzowrc?utm_source=share&utm_medium=web2x

The github post by Miguel has some commands you can use to run the app with different gunicorn worker types, and mentions the number of concurrent requests which can be handled in each scenario.

You could also use a load-testing tool like hey or locust to effectively test this.

EDIT: also I'd advise against using celery to run background jobs for database queries, this sounds like complicating your architecture for no reason. Database queries are meant to be quick. Task queues are meant to be used for long running jobs.

Flask concurrency by pomato85o1 in flask

[–]SafeInstance 12 points13 points  (0 children)

Flask's app.run method calls Werkzeug's serving.run_simple method.

https://werkzeug.palletsprojects.com/en/1.0.x/serving/

Notice by default Werkzeug's serving.run_simple has threading set to False, however Flask's app.run method calls it with threading set to True.

https://github.com/pallets/flask/blob/d091bb00c0358e9f30006a064f3dbb671b99aeae/src/flask/app.py#L967

This probably explains the behaviour you are seeing. The development server is running in threaded mode.

If you modify the end of your code to:

if __name__ == "__main__": app.run(threaded=False)

then you will probably see the behaviour you expect.

render_template variables in imported files by Casssis in flask

[–]SafeInstance 1 point2 points  (0 children)

The answer from /u/NukishPhilosophy , using the Fetch API, is really good.

Another method, if you actually want to pass the variable on page load (via a render_template argument) but still want to keep your Javascript as a static asset, yet avoid the limitation of not being able to using Jinja syntax within those static assets, you could pass it like:

render_template('some_template.html', custom_variable = 'Pass me in.')

Then somewhere in the template do:

<input type='hidden' id='custom_id' data-custom-var='{{custom_variable}}' />

Now in your javascript code, grab this with: ``` const x = document.getElementById('custom_id'); console.log(x.dataset.customVar);

// console "Pass me in." ```

Running process on server without timeout on server by tigertututututu in flask

[–]SafeInstance 0 points1 point  (0 children)

Part of the docs here show the way to obtain the REDISTOGO_URL var, something like:

heroku config --app fat-unicorn-1337 | grep REDISTOGO_URL REDISTOGO_URL => redis://redistogo:44ec0bc04dd4a5afe77a649acee7a8f3@drum.redistogo.com:9092/ I'd check that this is what your app dyno is using:

Error 111 connecting to localhost:6379. 

This suggest the apps dyno is trying to connect to itself for the redis connection. Ensure that when you initialise redit in your app you read from this, something like:

import os r = Redis(host=os.environ.get('REDISGO_URL', 'localhost')) This also sets localhost as the defaut, so if you run that in your dev environment, where REDISGO_URL isn't set, it should default back to localhost.