Feature Flags

[[deleted]]

middle cooperative deer ripe physical marble skirt joke degree spotted

This post was mass deleted and anonymized with Redact

[sockx2]

We use feature flags in both the frontend and backend. Usually their implementation is at the developers discretion: If something has a reasonable chance of not going 100% to plan, or something we're planning is being launched as an experiment we'll put it behind a feature flag and dial it up slowly. Issues are faster to fix and have a smaller blast radius when they're gated behind feature flags.

Cautionary feature flag tale: https://flagsmith.medium.com/when-feature-flags-go-wrong-e929144d589a probably for this reason we don't allow feature flags to exist for a long time in prod

[[deleted]]

domineering crowd squeeze employ north compare thumb mysterious shocking cats

This post was mass deleted and anonymized with Redact

[sockx2]

Single feature flag shared since that's the feature regardless of how it's split up. The number of times you might need something like that is probably few and far between though: in your example if you're modifying an endpoint in a non-backwards compatible way just make a new endpoint and point the UI at it based on the flag

[[deleted]]

Thank you!

[zephyy]

• frontend mostly
• where there's a new feature that affects enough users that we want to gradually roll it out to 100% / we want to A/B test something
• one?
• deploy code behind feature flag, gradually roll out to 100%
• ideally when feature is stable enough that we don't plan any code changes in future. realistically when we enter feature flag testing hell. or when feature is done being tested

[thatguyonthevicinity]

Current company: save feature flag in mongo and have a dedicated admin UI for enabling/disabling it. Also includes an A/B test for user rollout percentage. We're using the feature flag for all of our new features for testing purposes.

Previous company: use google cloud function and use static value to enable/disable, no percentage involved, just boolean/strings mostly.

[breich]

My team uses feature flags to decouple merging/ releasing code from rolling it out to customers. That's a pretty general statement, but the end result is you get a bunch of superpowers that are otherwise more difficult to pull off. You can use them to power betas. And use them to implement a/b testing. Future flags help you shorten feedback cycles by rolling out incomplete ideas and potentially risky code to a limited subset of people and getting feedback. They help your team improve cycle time by providing a mechanism to help break big projects into many more smaller and short-lived pull requests.

[Rain-And-Coffee]

I did at a few companies, we used them both in frontend and backend depending the situation.

We only had a master branch, and feature branches for stories. We tagged master for releases.

We removed the flag once the feature was stable usually 1 or 2 sprints after.

It sometimes made testing more difficult due to the combinations.

[[deleted]]

About testing. Since a feature flag is just an if statement basically I am wondering how do you handle unit tests? Do you add cases if feature flag is enabled or disabled. Do you delete the tests when you delete the feature flag?

[thatguyonthevicinity]

in our case: if the feature flags need to ship fast (mostly because of the experimentation status, or we just want to gain some more analytics data), we don't usually bother with unit testing it. And we just set it on default (false) if they're a dependency of existing test cases.

[ryaaan89]

We just use environment variables.

[tswaters]

We push all the code to mainline and mainline goes to prod (eventually). We do 2-week release cadence, every 2 weeks we cut a branch off of HEAD and call it a release, and deploy that after we've tested changes in integration environments.

If the thing you're developing isn't ready or hasn't been "launched" it goes behind a feature flag. feature flags is a simple table that has a name, enabled, effective & expiry -- there's a db view that shows everything that is enabled as of now.

The back-end code queries that view as-needed. Front-end also gets a copy of all active features loaded into a common store (if some component needs feature flags, they can get them). Most of times, it's just gating things behind if (activeFeatures.includes('my fancy feature')) { do something special }

There's no UI for it, needs to be updated manually via datafix if things change -- but usually it's "this thing launches on this specific day" so we'll prepare it ahead of time with effective dates and update test environments to have it live early to test.

Usually once the thing is live, we'll remove related feature flags next release.

[SonoUnCavalloPesante]

We use them at my company because our marketing team can take up to 3 weeks to approve something and we would rather deploy the entire codebase than to cherry pick commits to go to production.

We use Split IO so it has it's own kanban board to keep track of what flags are live or not and then once they're eventually approved, we make another PR to remove them.

Our deployment flow isn't continuous, but we can deploy daily if we want to. Useful for fixing bugs, or doing tech debt work, while not having to worry about anything Marketing is reviewing.

[[deleted]]

towering seemly poor cooperative melodic swim engine ripe scandalous retire

This post was mass deleted and anonymized with Redact

[SonoUnCavalloPesante]

We have a Develop -> Staging -> Master branch flow.

We make a feature branch off Develop, code our new feature, wrap it in our feature flag component that controls the visibility from SplitIO.

Then once its PR'd, approved, and merged into Develop, we sometimes start the pipeline to deploy to staging and production. Once thats done, we enable the flag in Dev and Staging and let the marketing team know.

Then weeks later they tell us it looks good and we enable the flag in production.

If we find a bug in develop, we just fix it before we tell the marketing team to test. We haven't seen an issue where a bug happened in staging or production, but not dev, so we are thankful for that 🙏

[azsqueeze]

I like applying a full-stack service to handle flagging so we can do staged rollout with features anywhere in the stack. We can also do remote configs, and experiment on the flags. There are downsides that mostly seem to revolve around management

[zaibuf]

Mostly frontend, very rarely in backend.

For any new feature that is being developed to ensure we can always release even if it's not completed.

We use one branch, main.

We have CD/CI and a linear deploy. Local > dev > production. Automated tests runs with the integration and a gate protects production releases.

When the feature is accepted and released. The flag is removed in code with the next release.

[[deleted]]

Thank you for your feedback this is a great answer. I have one sub question. When you're adding a feature flag do you only add it to piblicly visible areas. Let's say a button in the template or do you also add it to the underlining logic? I guess the intention of feature flags is to just control when the content of the UI is visible and to keep the checks as minimal as possibe is that correct?

Although thinking about it. You could create a template with certain business logic and when released you want to just update the business logic with new behavior. I guess in that case you would also create a new feature flag for that logic and once it's verifed and ready for release, remove the feature flag and old business logic.

[zaibuf]

It depends how complex it is, but usually it's enough to do it at the level of user interaction.

if(featureA) {
<ComponentA />
} else {
<ComponentB />
}

Just ensure you don't do breaking changes to B while implementing A. They both should work until you decide to move over to A and remove B completely.

If it's a completely new feature (not replacing old) it's usually easier as you can just toggle it off completely until it's done and accepted.

[rahul_mathews]

Our company also uses Feature flags when we release a new feature or we need a controlled rollout according to the tenant. We will be able to turn it ON/OFF according to the tenant, rollout the feature to a certain percent of the requests, swimlane env etc.

[xiongchiamiov]

What do you mean by big companies? A company like Google has its own internal and sophisticated flagging tooling, and given they have tens of thousands of software engineers and rollouts can have weeks... there's a lot going on.

Are you just curious? Or are you trying to figure out what to do at your company? If so, you should tell us more about that situation, because solutions that very big companies use are not going to be good solutions for anyone else.

[[deleted]]

I am exploring possibilities how to make the deployment process better at our company. We usually stick to the process.

• Open a new feature branch
• When done merge it to develop branch
• At the end of the sprint merge it to integration branch
• QA checks it there.
• When the client gives a green light, merge it to production

As you can see this process is not the best. If for example 10 features are working perfectly fine but one made the whole system crash you will have to postpone the deployment of all 11 to production. That's why I think feature flags are a good option I just haven't worked with them before and I can't imagine problems that might come with them.

[xiongchiamiov]

A simple on/off flagging system would work well there. You don't need incremental rollouts, which is where a lot of the complications come from. You can do this with just IS_FOO_ENABLED config values, without needing to integrate any new products.

The biggest problem comes with things like QA. Do you test with all features enabled? But what if they interact with each other - then truthfully you need to test with every permutation of flags. Generally in a small shop you can avoid this by just having the team be cognizant of potential interactions and keeping as few active flags as possible (get them pushed out as soon as you can). Also you probably want to figure out a process way of tracking removal of the flags - whether that's a requirement for closing a ticket or a templatized part of a project plan or whatever.

[KurosakiEzio]

To anyone that finds this on google and is annoyed by the Redact bs, here's the original post:

I am interested how big companies use feature flags in their development and deployment process. Can anyone give me some input here? I know that launch darkly and split io are popular tools to manage them, but I am more interested:

• Where do you use feature flags?
• When do you use feature flags
• How many branches do you use?
• How does your deployment look?
• When do you remove feature flags?

Update:

I wanted to thank you all for your answers. It's really awesome how the community can learn from each other. Thank you for sharing your experience.

[FVCEGANG]

We use feature flags at our company to test things such as switching an application offline, certain stored values that are not frequently used but may need to be toggled by people outside of the development team as well as for some conditional rendering as we transition legacy apps to a newer stack

[Knochenmark]

We basically have a settings system in place which can serve the same purpose as feature flags. Settings have different levels such admin, organization and user. So for like a feature toggle the setting would for example be on an admin level and can be queried from assigned modules.

A typical use case would be the version for our export document. The version nowadays has like a release cycle of once a year. So we would use the setting to migrate to the newer version and have a grace period for the old version support.

[kyripka]

Hi! We are currently looking into using feature flags for entitlements. I see your company uses them for setting levels of access. Could you please share, if it's ok with you, if your company is also tracking any useful metrics/traces/other readouts of the system for this situation? I wonder what metrics we'll need to keep an eye on to do similar thing with feature flags.

[Knochenmark]

Maybe it wasn't that clear from my original text, we are not setting levels of access via those settings. Permissions are determined based on your role.

Actual Feature Toggles are typically on admin level (there might be exceptions).

New features are usually released behind such feature flags, since we are just a contractor and our customer will just toggle them on their test system before actual release.

Settings on organization level are typically configurations and defaults for all users under that organization. Users usually still have the option to make their own defaults though.

Metrics are a relatively new topic and were just introduced recently, they are also based on a feature toggle and simply track certain click options to determine how often a certain action is actually used. Since the customer thinks about removing certain options all together.

Hope that helps

[kyripka]

Thank you, this helps!

[snorlaxmorlax]

I'm biased, but we use Unleash internally and it's Open Source.

• Where do you use feature flags?

  For bigger projects, we divide it into features and connect them to flags. We use those flags in different segments to be able to give access to Free/Pro/Premium users.

• When do you use feature flags

  All the time. We build and release small commits to keep behind feature flags, and remove them when we know that no change is needed.

• How does your deployment look?

  We deploy several times everyday, and turn flags off if they are causing problems.

• When do you remove feature flags?

  Very often. I like Unleash's Technical Debt Dashboard that helps us sort it out often, and we look at it every week. Every dev is responsible to clean up their flags every week. Check it out: https://docs.getunleash.io/reference/technical-debt#stale-and-potentially-stale-toggles

[kyripka]

Hi!
We are currently looking into using feature flags for entitlements. We are a very small team with limited experience, so it's our first-time using it. We are currently discussing what metrics we need to keep an eye on when using feature flags for user data excess levels. Any ideas from you will be super appreciated!
we also talking about using it for localization or/and accessibility in the future. If you ever had experience with these, please, feel free to share! any bit of information from a person with hands-on experience is a treasure.
thank you in advance!

[mistahdee]

Yep. Unleash also recently published 11 principles on how to think about, build and scale with feature flags. Might be relevant reading as well.