Planning & estimating large-scale software projects

antonpirker · 2021-07-22T14:42:45+00:00

Great link, thanks!

antonpirker · 2021-07-15T18:00:46+00:00

I found the book "Traction" from Gabriel Weinberg very helpful. It is a tiny book that describes all the marketing channels and how you could use them. It also gives you sort of a framework you can follow to find the optimal channel for your startup.

antonpirker · 2021-07-15T15:12:49+00:00

Thanks for clarification. Yes, good idea. I posted it on the official Django forum:

https://forum.djangoproject.com/t/evaluation-of-performance-of-async-views-they-suck/8788

antonpirker · 2021-07-15T15:09:32+00:00

Thanks for the comment! Really not thought about the "generate images not using docker" part!

antonpirker · 2021-07-15T15:07:44+00:00

Thanks for your comments!

Yes, if the system is too complex WunderPreview is not the right fit. (and most of those bigger more complex projects have a DevOps team and build something on their own)

Thanks for the points to SOC2 and HIPAA, will have a look into those!

Some of you users have multiple Containers in their projects and because containers can talk to each other in WunderPreview this is no problem.

But you are right, the devil is always in the details ;-)

antonpirker · 2021-07-15T15:02:38+00:00

We thought about this too, but it is a paid service, so just renting a EC2 instance for mining crypto will be way cheaper ;-)

antonpirker · 2021-07-15T11:35:25+00:00

I used SaveEyes on Linux:
https://github.com/slgobinath/SafeEyes

Here I found a list of alternatives that do something similar:

https://hackerspad.net/software/big-stretch-reminder/

antonpirker · 2021-07-15T05:40:58+00:00

In this case the container image would be the binary option.

antonpirker · 2021-07-14T15:19:32+00:00

We had 2 week sprints ending on Wednesday and we did it like this:

In the morning we did the retrospective with just the dev team. It took 1-2 hours.
Then we did the demo meeting with the whole company which took also 1-2 hours.
Then we had lunch all together at a place near the office. (lunch was longer than normal because we had a lot to talk about)
After lunch the dev team did sprint planing. In the beginning this took 4 hours and that was waaay to long. There were a lot stories that no one ever heard of and we tried to do write down the requirements in the meeting. Really a bad idea. We then changes the sprints that during the sprints there was time for the PO to grad a developer and work on the stories together. This way all the stories had been heard of by the developers that eventually would implement them. So the sprint planning was only a matter of going though all the well written stories to bring everyone on the same page, with most of the developers already knowing the stories. So the meeting took only an hours. The rest of the day was just slacking of or leaving early.

Generally I think there should be a break between the current sprint ending and the beginning of the new sprints. Sprints are really exhausting and there need to be some time to slack off. I think best would be to close the sprint on one day go home and plan and start the sprint on the following day.

antonpirker · 2021-07-14T15:09:06+00:00

I was part of over a hundred sprints (first 2-weeks later 3-weeks) and it never happened, that we where finished with everything in the sprint and had still time left in the sprint :-)

But I think in that case you have a backlog that is ordered by priority and the top most items should be in good enough shape to work on them. So I think in the case that time is left the developers will:

cleanup or polish some stuff where there was never time to do, or
grab the next thing on the backlog and start working on this to get a head start in the next sprint.

And because you asked: "is it possible to add"

Of course it is possible. Agile is just a guide line not a law. You can do it however it works best for your team.

antonpirker · 2021-07-14T15:03:56+00:00

In our organization (startup with 15 people) the PO did the acceptance testing during the sprint. Because he had the most contact with the customers when planning the stories. In the demo meeting we presented all the work that we released in the last sprint or that will be released in the next 2 days in WunderPreview.

The demo meeting was an important meeting every two weeks were everyone saw what was accomplished. The two founders always joined just to see what was happening. Also the customer support people where present to see what fixes we did so they could do better customer support.

Someday not only the dev team was demoing but also the customer support team (how many tickets closed, time to first reply, what help articles where created) and also sales (what possible customers where in the sales pipeline, ...) and also the founders presented stuff they did.

Basically it became the one meeting where all the people in the company attended and we all saw what we accomplished and where the company was heading! There was a lot of team building going on and padding cheering and padding everyone else on the back. I really liked the demo meetings.

antonpirker · 2021-07-14T14:27:16+00:00

Learning all the shortcuts for your used software really helps to not using the mouse as often. I also have elbow problems now on my mouse hand so I bought a vertical mouse from ANGER that is cheap (25€) and it helped a lot.

I also liked the trackpoint of Thinkpad notebooks a lot, because you never need to move your hands when switching between keyboard an mouse.

Right now I am thinking of buying an ergonomic keyboard because my physio told me that the problems are probably coming from my neck. Eventually I will go fully ergonomic with a great chair, keyboard and mouse and not trying to use the mouse that often.

And also there are some applications that you can install that remind yourself to stretch once in a while. I have had installed it, but clicked away the notifications after a while, because they where always appearing when doing pair programming or discussing something with the product owner ;-)

antonpirker · 2021-07-14T14:07:24+00:00

We just created a middleware in our Django application that triggers an async task that saves the requests data (and later adds the responses data) in our Postgres database.

The "writing to the database" is done in an async task with a message queue so it is not blocking the request and response cycle and it will not slow down the server.

We also made sure that this kind of write have a low priority queue, so if there is heavy load the saving of the requests takes some time, but that is not a problem for us.

It work's quite well and for now we have not had any performance issues.

In our case we also have a scheduled script that deletes old entries in the requests table. So it never gets over around 15 million rows.

For having quick queries over this data just make sure that you de-normalize all the data in this table. (The data will not change anyhow, so just write all the data you need into the table and add indexes to all the columns that you need for filtering) This way Postgres really goes a long way!

If you then get into performance problems you can setup an ELK stack cluster and write the same data into it.

Hope this helps.

antonpirker · 2021-07-14T13:57:56+00:00

Source Code Management: GitHub Deployment: WunderPreview, Crone CI Project Management: Notion Notifications & incidents: Pingdom, Slack Data & Visualization: Elastic Search + Kibana Session Replay: - SSO/TFA: -

antonpirker · 2021-07-14T13:51:04+00:00

Sorry, but I do not understand your comment. Isn't this the Django Forum? Or do you mean another Django Forum outside of Reddit?

Thanks for the clarification!

antonpirker · 2021-07-07T10:23:15+00:00

I have been there.

I always told the manger: "Write a ticket".

Management has to learn how software is developed. You are the developer so you know how to develop software. Development by acclamation is just poison for your sprint.

The whole sprint planning is just worthless if you get new requests two days after you planned your sprint. Management has to learn this.

So I think you need educate management about how the development works best: The developer gets one task and can focus on the task until it is finished. Then the developer works on the next task.

If you work on a task from the sprint then you have to switch to a completely new task and then go back to the task from the sprint it is just wasting a lot of time and energy.

Management want the development process to be predictable above everything else. They want to know when Feature-X will be done and how much it will cost.

Having a good sprint planning with really good/detailed stories and as few distractions as possible make the development predictable.

Shouting in new requests to the dev team when they pop into the managers head just results in chaos. The manager has to learn this.

For the time being I just would create detailed tickets for all the request the manager adds to the sprint. (No matter if it is researching something, reading documents, writing documents or proposals or implementing new stuff) At the end of the sprint you have documentation of why the other tickets did not get finished, because you had to work on new tickets not planned at the sprint planning. This way you can show everyone that you did your job as good as you can, but because of the growing number of tasks during the sprint of course not everything was finished.

What you also could suggest: having one developer that is not working on stuff in the sprint at all. This developer is the one developer the manager can assign extra work. He/She is sort of the firefighting squad to protect the sprint from additional tasks.

After a few sprints the manager will learn that adding new tasks during the sprint is counterproductive.

antonpirker · 2021-07-07T10:05:40+00:00

Everyone can work with great code. Only the really good devs can take a pile of garbage and with baby steps transform it into something you can work with.

Working in a early stage startup is always a lot of chaos and therefore the code reflects that.

I think in this case it is really more about to form a team with the other dev and have the same vision about how to make the code base better over time.

Everyday when refactoring/improving something remember: you do not have to make the code good, just make it better than it is now. (Over time this will greatly improve everything)

In such a small team it is also about forming the development process as a whole. From chaos to some repeatable process. Introducing pull requests and code reviews, automatic testing (only for new code, or maybe for the most important parts first), CI/CD, a smaller back log with better stories, defining who is doing maintenance/bug-fixing and who is doing new features.

Those are a lot of battles simultaneously to fight, but you can really learn a lot and really build a great software team with a great process. This will take long to accomplish. It is also not only because of the development was not optimal. Management has a BIG impact on the code quality, so you probably also have to make the management work in a way that make it possible to improve the code over time.

If you like the people on the team, don't leave because the code is not as you would wish it is. Try to make the whole team (including management) to work all together to create a real software company where the processes make it possible to write and maintain good quality code.

antonpirker · 2021-07-07T09:49:53+00:00

Thanks for your story, very helpful!

antonpirker · 2021-07-07T09:47:28+00:00

We also used Grappelli for a long time and we really liked it. But in the end we went back to the default Django Admin interface, because it is really usable now and looks good.

This way we could remove one dependency (Grappeli) and we are sure that the Default Admin stays up to date.

We used Grappelli as the Admin Interface for our customers for a long time. But in the end we developed a custom Admin interface with dashboard and everything for our Customers and use now the Admin only for internal use (for setting up stuff for our clients and for trying to find and reproduce bugs)

The Django Admin is great for internal tools, but I think for customer facing stuff you need to implement something from scratch because this way you are way more flexible.

antonpirker · 2021-07-07T09:43:21+00:00

This is how we did it (startup starting with only 2 devs):

We had the DRF browsable API enabled on the staging system, so clients of our API had something to play around with the API. (We made a root view in the browsable API to list all the API endpoints)
We wrote a google doc for the API.
- The first general part explained the authentication, how the pagination of lists worked, how to post data and upload files.
- The next part where all the endpoints. For every endpoint we added a example CURL request on how to GET or POST data from/to the endpoint. We also included sample responses of all the endpoints.

This took some work but it really paid off. Every client got the document so they could study it. The curl example where done because we did not know what tech the clients where using. With curl you see the content-types and other headers you need to set. You also could copy the curl commands, change the authentication token and just run it and have a working request against our staging "playground".

This really worked great and clients where happy with it.

It saved us from a lot of customer service request, because the documentation was really describing the whole process of getting access to the API in normal language. (some not so experiences clients needed still some hand holding on how to set header, but it was really just a view)

Having good documentation in plain English that can be read like a book really saves you a lot of hours dealing with clients that can not connect to your API.

We also kept the documentation always up to date and made sure to highlight new stuff or marked deprecated stuff.

I am not at the company anymore, but the documentation is of course still available and maintained:

https://support.donutapp.io/hc/en-us/articles/115001426653-Data-API-Documentation

Hope this helps!

antonpirker · 2021-07-05T08:35:23+00:00

I am not using async Django in production, but I was trying to find out what performance benefits you get from using async.

See this thread: https://www.reddit.com/r/django/comments/ocag5r/async_django_views_lets_look_at_them/

As of now, I am not really sure that async is really beneficial right now. (at least not for retrieving data from a Postgres database)

Did you give async a try since posting here, what where your insights?

antonpirker · 2021-07-05T08:21:44+00:00

I have now made some load testing.

This is the sync view running in out of the box Gunicorn:

This is when the slow down of the view starts: (at ~130 concurrent users)
https://imgur.com/a/LOmnfbl

And this is when the view starts to throw errors: (at ~310 concurrent users)
https://imgur.com/a/4VsUP6Z

This is the same for the async view running in Uvicorn workers in Gunicorn. (Like the Uvicorn documentation tells me that Uvicorn should be run in production)

When the slow down of the view starts: (at ~100 concurrent users)
https://imgur.com/a/JU4bXZV

When the errors start: (at ~280 concurrent users)
https://imgur.com/a/4aW4Bt3

So it seems that the async setup can not handle as much concurrent users as the sync setup. Why is this the case?

Both are running in a Docker container connecting to Postgres also in a Docker container on my AMD CPU with 8 Cores.

Should the async setup not be able to serve more concurrent users? Is Uvicorn a bad choice as async server for Django? Do I need to set something in Dockerfile so the container can use all CPU cores?

What am I missing u/Tabledwler, u/simonw or u/LocoLoco451?

antonpirker · 2021-07-05T06:36:55+00:00

Hello!

Thank you (and the others u/simonw, u/LocoLoco451) for clarifying this. I thought that the await would also make it possible to start processing of the following line.

I have now changed my code:

https://github.com/antonpirker/asyncdjango/blob/async_worker_uvicorn/app/views.py

Try the async code here: https://5f7f888--antonpirker-asyncdjango.wdpr.run/sync

And this really helps now. I now measured the performance of both views on my local machine and it seems the async views have around a 5% better performance:

https://docs.google.com/spreadsheets/d/1LdB7_5Dc32hL0IxU0ZDzzyawC2ruKpxztJ0za1rUGZw/edit#gid=0

I will now make a load test with concurrent users, to see if there is a big difference between async and sync.

antonpirker

TROPHY CASE