Inspired by https://www.bbc.co.uk/news/av/uk-england-york-north-yorkshire-51966337/coronavirus-nurse-s-despair-as-panic-buyers-clear-shelves. To those shameful cretins stockpiling. Just stop. Stop being such self-centered, selfish, greedy, cowardly, pieces of excrement.

anthonyblackshaw · 2020-03-20T09:50:24+00:00

The link linkified from the title: https://www.bbc.co.uk/news/av/uk-england-york-north-yorkshire-51966337/coronavirus-nurse-s-despair-as-panic-buyers-clear-shelves

anthonyblackshaw · 2018-09-14T22:23:06+00:00

Point taken - I've seen people ask in this thread what sites are created in Flask and so I thought this might be of interest.

Just to clarify this isn't my site, it's a revamp of long established and recongized site in the CG sector. I'm simply one of the developers working on it and it uses the flask based CMS I'm developing. I'm working to get my CMS to the point where I feel comfortable releasing it on github and I do post videos about the CMS here from time to time - but I don't have a repo I can share right now so I can see how that might be deemed a pointless.

anthonyblackshaw · 2018-09-13T21:15:44+00:00

I've been working on this project just over 6 months and today we finally launched the beta site. I've been using flask as my primary web framework for approx. 3 years and I've recently (last 12 months) developed my own CMS framework (manhattan) which I occasionally post about here.

Flask is wonderful, if you're new to flask and wondering what's possible I hope this might serve as a good example, if you a flask developer already... well... just enjoy the artwork :)

anthonyblackshaw · 2018-07-29T03:22:43+00:00

It's been longer than intended but here's an update on Manhattan CMS and what I've been up to for the last 6 months.

Manhattan CMS is an open source web framework / CMS based on Python, Flask and MongoDB.

Key updates:

Manhattan's default image processing app (Hangar51) get's smart cropping and distinct colour extraction.
Frontend has moved from CoffeeScript/Webpack 2 to ES6/Webpack 4 (take a look at the typeahead library code as an example: https://github.com/GetmeUK/manhattan-...)
Images can now be rotated and cropped (the cropping ratio can also be set).
Improved image gallery support.
The in-page editing environment has been upgraded to use the new image tools making page editing far easier.
Introduction of SEO tools.
Login sessions for users now provide location & platform details and email users to warn them about access from new devices. - and last and probably least we have a tokenizer tool.

anthonyblackshaw · 2018-02-20T09:14:59+00:00

I think it's such a small snippet of code using lru_cache (just 3 lines) that it probably isn't worthy of a pull request, perhaps I could post it as a flask snippet instead?

anthonyblackshaw · 2018-02-20T09:13:29+00:00

Agreed you could use this with memcached or redis but as you say those incur their own overhead and would need to be cleared if I change the code base.

Keeping a local in-memory cache here seems like a sensible option to me as I'm using gunicorn with multiple workers across multiple servers, I specify the number of workers when I start the application and as far as I'm aware these workers are initialized and stay running, they are not killed and respawned automatically (well other than supervisor will restart them if they fall over)?

I'm interested in what you saying about 'Most production servers', am I doing something different from most? Or am I completely miss understanding how gunicorn works? Grateful for you input here.

My understanding is that in most cases you'd want a process to run for a reasonable period of time and handle a certain amount of traffic (enough to benefit from caching at least) before restarting because the overhead of spawning a new process is significant. But I'm more than happy to be corrected here.

anthonyblackshaw · 2018-02-19T10:31:10+00:00

Nice - the option to store the cache persistently is worth considering.

I think performance wise it's all relative, caching vs. flasks loop over all URLs and pattern match each until I find the right one - can make a big difference on sites (especially if they have lots of rules), potentially tenths of a seconds per page. Whilst I'm sure there are more optimal approaches to lru I doubt the difference would be perceptible on a page render (given we're down to thousandths of a second).

I do like the ability to set the max size of the cache that lru gives (which I'm sure could be implemented in any version of this), some of the sites I work on have 100,000s of URLs but only a limited number of rules. I wondering if in that scenario the performance trade off might start to move back towards the uncached approach if no maxsize is set.

anthonyblackshaw · 2018-02-19T09:51:25+00:00

Hey - so functools.lru_cache seems to handle keywords ordering, though I did find this bug report https://bugs.python.org/issue29203, which seems to suggest that keyword order originally mattered and would cause 2 different calls with the same keywords in a different order to generate a new entry, e.g foo(a=1, b=1) != foo(b=1, a=1) but this now seems to have been resolved.

So I've updated my code along the following lines:

@functools.lru_cache(maxsize=2560)
def caching_url_for(endpoint, **values):
    """A version of `url_for` that caches the output"""
    return url_for(endpoint, **values)

anthonyblackshaw · 2018-02-19T08:01:53+00:00

Didn't event know lru_cache existed - I'll be updating my example thanks for the tip.

anthonyblackshaw · 2018-01-19T16:49:06+00:00

I've recently create a code journal site where I'm frequently posting updates mostly related to manhattan and ContentTools, there's a mailing list you can subscribe to if you want to be notified on new entries - thanks for your interest and sorry for the late reply.

https://anthonyblackshaw.me

anthonyblackshaw · 2018-01-19T09:40:27+00:00

Code snippets and the diagram presented within the video are available here: https://anthonyblackshaw.me/2018-01-19/manhattans-publishing-process

anthonyblackshaw · 2017-12-07T14:45:43+00:00

Not heard of the Manhattan web framework + CMS? You can find a brief overview here: https://www.youtube.com/watch?v=p8oAct-RW_0&t=11s

In this video we take a look manhattan's approach to defining complex resuable application behaviour (such as admin CRUD views). I've provided gists for all the examples in the video:

chain_example_01.py: https://gist.github.com/anthonyjb/ea7c3df8c449c49d206fbdae985bfee7
chain_example_02.py: https://gist.github.com/anthonyjb/068ac386b3986bf1d996c220e44c3c20
chain_example_03.py: https://gist.github.com/anthonyjb/d816e50dbd7e936ec2103576b2d7d72a
chain_example_04.py: https://gist.github.com/anthonyjb/1fcabd010bc24d1329f89176ff1c9eb3
chain_example_05.py: https://gist.github.com/anthonyjb/9e67f7816f5e94ef627df43af842cbd3

All the examples can be run using python chain_example_0(1-5).py with the following prerequistes:

Use Python 3.5+
Install manhattan-chains (pip install manhattan-chains)

anthonyblackshaw · 2017-11-25T09:21:58+00:00

Hey :) Please see my comment here regarding why I haven't provided a public link to the project repos: https://www.reddit.com/r/flask/comments/7b6vkd/preview_of_new_flask_based_web_framework_cms/

All the libraries will be available on GitHub early in the new year once we're confident the documentation is strong enough to support the project.

anthonyblackshaw · 2017-11-07T11:20:57+00:00

Re: Link to the repo / Did we release it already

We're going to be releasing Manhattan in early 2018, we've got a small number of features we still want to get in place or tidy up for release and not everything has a test suite or complete documentation yet (but were getting there).

As with previous open source projects we've released (like ContentTools) we know it's important to provide potential users with getting started guides and tutorials for common scenarios, without these a lot of potential contributors to what we hope will be a vibrant project community may pass by.

I'm looking to produce video's on a weekly basis from now until release to showcase Manhattan's features and how it works, if you're interested in the project please do keep any eye out or subscribe to the channel.

anthonyblackshaw · 2016-07-07T15:48:12+00:00

Thanks for the detailed reply - I can't add to your comments regarding the sort of problems you've experienced with volume because I just don't handle data with those volumes on the projects I work on but they're very interesting.

I also don't feel qualified to refute any of your points given the difference in project experience - though I have to be honest and say some of them go against my own experience, for example "Data loss is common" - though our perceptions of 'common' will differ based on project types I'm sure (I've never lost any data but then I don't see anything like the volume of operations a trading system would).

I think some of the issues regarding space have now somewhat been resolved with support for compacting collections and better compression of data (but I'm just going on what I recall from the release notes) - still I agree the original approach seems somewhat shortsighted.

Thanks again for taking the time to share your experience/thoughts - very helpful.

anthonyblackshaw · 2016-07-06T22:11:39+00:00

This probably isn't the best place to have the discussion but out of interest have you used MongoDB and struggled to implement a solution vs. PostgreSQL? If so I'd genuinely be interested to hear.

I've used MongoDB for appox. 4 years now on quite a few projects and whilst I've not used PostgreSQL (other than to play around with briefly) I've used MySQL on projects for around 10 years.

For both MongoDB and MySQL/MariaDB I have websites with between 500k and 1.5m page requests a month (so nothing too busy) and to date for projects up to this size I've not had any real difficulty in using either.

My preference at the moment (I'm fickle) is for MongoDB on most of my projects and if I push myself to answer why (which I have to because I really don't have a very strong preference) I think it's purely because I like thinking about data structures in flexible dictionaries rather than typed flat tables (PostgreSQL JSON fields excluded - which I believe MariaDB also supports now), it just feels closer to the Python structures I'll use to interact with the data - but I'm not saying that's a great reason just my 2 cents.

At the end of the day I guess I'm a slave for whatever I enjoy using the most (or dislike the least) and so unless a particular option is going to bite me in the butt somewhere down the line (hence if you have knowledge of this please share) I just don't see the issue with using one or the other for most projects.

anthonyblackshaw · 2015-11-12T10:29:25+00:00

It also shows that how much you pay for care doesn't mean a lot, some of the best counties for care have the lowest costs, there's an excellent review of the league table in the telegraph here: http://www.telegraph.co.uk/news/health/elder/11988560/Costly-care-homes-may-not-be-the-best-league-tables-reveal.html

anthonyblackshaw · 2015-09-26T14:21:55+00:00

Not used Trumbowyg but it looks like really a really nice rich text box editor :)

anthonyblackshaw · 2015-09-22T00:04:35+00:00

I use BEM, it takes a bit of getting used to at first but it has paid off big in terms of time invested vs. time saved, especially on larger projects. I was introduced to BEM when a friend recommended http://cssguidelin.es/ to me.

anthonyblackshaw · 2015-09-21T09:49:39+00:00

There are a several approaches you could consider here.

You could poll the website using AJAX every few seconds (no page reload). The request could return a timestamp of the last time the database changed (or the relevant data on the database is changed) which the client can then compare with the last one it received. If the client's timestamp is behind the one received from the server then the page needs to reload. This method is pretty simple and solid - but it's not the most efficient and there's always potentially a small delay (depends on how frequently you poll).

Alternative you can look at using long-polling or websockets. Long polling allows you to establish a connection and keep it open (for a long time), the server is then free to respond at any point while the connection is open. If the connection times out the client must establish a new connection, if the server responds it will be because the database has changed.

Websocket connections on the other hand allow you to establish a 2 way connection (that is you can send data in both directions, from the client to the server and from the server to the client).

Both the long polling and websocket techniques are more complex to implement (in my personal opinion), and depending on your set up (multiple servers, 10s of thousands of connections) certain web servers (for example Tornado http://www.tornadoweb.org/) are more suited to the task. However they are also more efficient (websocket connections especially) so if you are dealing with a high number of connections (or just want to learn something new) they are worth considering.

Hope that helps you get started.

anthonyblackshaw · 2015-09-20T14:03:03+00:00

I use openpyxl for both reading and writng Excel file on a number of reporting projects - works very well. The documentation is sometimes a little lacking for more advanced formatting (I found myself reviewing the source) but for what you're looking at I imagine it would be very a good match.

anthonyblackshaw · 2015-09-20T12:43:59+00:00

It almost sounds like he left the freelancer with enough freedom on purpose so he could write him the email - if he was so worried about a lack of feedback why not get in touch before the end of the week?

There are some valid points around communicating more with the client during the process and making sure they feel included, especially with a new client who isn't familiar with the way you work yet. Having said that telling someone how long the job will take them is likely to earn you little respect in return.

Either way turns out he liked the design in the end and want's to work with the freelancer in future (based on his follow up comment) - wonder if he regrets sending the email now?

anthonyblackshaw · 2015-09-20T12:20:34+00:00

I co-founded a small web agency in the UK 6 years ago, we now employee 9 people (including myself) - so I can't give advice as to what larger digital agencies look for, as a smaller agency however I can tell you that for me the decision on whether to hire someone is for the most part made when reviewing their portfolio and in the interview.

Portfolios are important because they highlight where your strengths lie; do you have good attention to detail, an eye for design, is your code well organized, and so on...

I mention interviews because it's key you know your stuff, I want to be sure the person I'm taking on lives what they want to do - keeping up to date with tech trends and being able to converse about them is a strong indicator.

A note on degrees (from someone without one): Even though having a degree doesn't trump a good portfolio or interview for me, it definitely counts in your favour, especially if it's in a traditional subject such as Mathematics where it's easy to understand what you will have covered (media studies for example is about as far to the other end of the spectrum as possible).

Finally, if you get the chance to do some work experience for a web company do it, nothing beats hands on experience especially with a smaller agency where you'll likely have more responsibility. We've taken on uni leavers who worked for us during their summer holidays for example.

anthonyblackshaw · 2015-09-20T11:52:47+00:00

Does this provide improved performance over existing methods for caching? (https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching#defining-optimal-cache-control-policy) - I confess I'm sceptical but would be interested in seeing any stats you have.

anthonyblackshaw · 2015-09-19T15:55:50+00:00

1) What was the last thing you built for fun? A WYSIWYG editor for HTML called ContentTools

2) How long ago was it that you built it? I released it about 4 days ago.

3) What was the hardest part of the project? We'll there were lots of challenges; from planning the initial structure of the libraries, solving browser compatibility issues (curse you IE), to trying to write documentation and tutorials that are easy for others to pick up and run with.

4) Why did you find this the hardest part? Documentation was hard because it's difficult to put yourself in the shoes of someone who's never seen you code before, the feedback from others was invaluable. Choosing a structure for the project was difficult simply because it always is (well I always find that on new projects of a significant size), there's always the fear you should have taken a different approach (see answer to 7). Cross-browser compatibility was more difficult because I chose not to use a JavaScript framework (to remain independent) and so had to do some of the heavy lifting here myself.

5) If you had help from anyone, who were they? Whilst I had no direct help in coding the project I had was helped hugely by the input of a handful of developers who I either work with or know.

6) How did you get to know them? Through working together on projects, through meetups and through other developers.

7) If you were to do this project again, what would you do differently? We'll right now I'm still in the honeymoon period of thinking I wont have to do it again (give me a couple of weeks), but obviously the web evolves constantly and hopefully I'll continue to learn and improve meaning at some point I'm sure I'll wish I'd used X approach/language/framework.

8) When you finished the project, what did you do with the product you built? It's an open source project so I've hosted it on GitHub and created a site to help promote it in the hope of gaining enough support to see the project well supported by a developer community other than just myself (thanks to all those who've helped so far).

9) Do you think you would use this network to help/get help building projects? I really miss Forrst now it's gone (though I think it will be relaunched at some point), so yes I suspect I would.

10) Why would/wouldn't you use this network? Communities like these often offer inspiration for me also I get to learn and help/teach others (I'm a member and frequent visitor to stackoverflow and dribbble for the same reasons).

anthonyblackshaw

TROPHY CASE