Feature Flag Architecture

Why architecture is the part people skip

Most teams adopt feature flags by dropping an SDK into one service and wrapping an if around a new feature. That works until you have forty flags, six services, a mobile app, and an edge function, all asking the same questions about the same users. At that point the naive setup starts to leak latency, produce inconsistent answers, and turn your flag provider into a single point of failure. Getting the architecture right early is much cheaper than untangling it later.

This chapter is about the moving parts underneath a flag call. Once you can picture where a flag lives, where it gets evaluated, and how that decision reaches your code, every other decision about performance and reliability becomes obvious.

The four layers of a flag system

Every feature flag platform, whether it is LaunchDarkly, Unleash, or Flagsmith, is built from the same four layers.

The management plane is the dashboard and API where humans and automation define flags, targeting rules, and rollout percentages. This is your source of truth. It rarely sits in the hot path of a request.

The delivery layer is how rules get from that source of truth to your running code. It is usually a streaming connection or a polling loop that keeps a local copy of the rule set fresh. Streaming pushes changes in under a second; polling trades freshness for simplicity.

The evaluation engine is the code that takes a flag key plus a user context and returns a value. The critical design question is where this runs, which we cover below.

The context and events pipeline carries user attributes into evaluation and carries exposure events back out for analytics and experiment analysis. If you plan to run experiments, this pipeline is not optional.

Local evaluation versus remote evaluation

The single most important architectural choice is where evaluation happens.

With remote evaluation your app sends the user context to the provider and gets a decision back over the network. This keeps rule logic and user data on the server side, which is why client side SDKs for web and mobile often work this way. The cost is a network round trip on the critical path, so you cache aggressively and accept that decisions can be a few hundred milliseconds stale.

With local evaluation the SDK downloads the full rule set once, holds it in memory, and evaluates every flag inside your own process. A flag check becomes a hash and a comparison, costing microseconds, with zero network calls per evaluation. Server side SDKs default to this model. The tradeoff is that the rule set, including your targeting logic, now lives in the client, so you would not ship it to an untrusted browser.

A practical rule of thumb: trusted server environments use local evaluation for speed, untrusted clients like browsers and mobile apps use remote or edge evaluation to protect rules and user data. Many teams run both, which is why providers ship separate server and client SDKs.

Designing for failure

Your flag provider will have a bad day. The architecture question is whether that becomes your bad day too. Three defenses matter.

First, always pass a default value at the call site. client.variation("new-checkout", user, false) returns false if anything goes wrong, so the flag degrades to a known safe state instead of throwing.

Second, initialize the SDK in a non blocking way and let it serve defaults until the rule set arrives. An app that hangs on startup waiting for flags has coupled its uptime to the vendor’s, which defeats the point.

Third, persist the last known rules to disk or a local cache. On restart the SDK can boot from that snapshot even if the provider is unreachable. This is how tools keep serving sane decisions through a network partition.

The mental model to hold: the flag provider is an advisor, not a dependency in the request path. When the advisor is silent, your code already knows what to do.

Scaling to many services and clients

As you add services, two problems appear. Every instance opens its own streaming connection, and every client fetch reaches out to the vendor. At scale this means thousands of connections and a lot of duplicated traffic.

The standard fix is a relay proxy (sometimes called a relay, edge proxy, or unleash-proxy). You run one process that maintains a single connection to the provider and fans out the rule set to all your local SDKs. Your services talk to the relay on your own network, which cuts external traffic, lowers latency, and gives you a control point for compliance. For microservice heavy architectures this is close to mandatory; see our guide on feature flags for microservices for how the major tools differ here.

For frontends, an increasingly common pattern is edge evaluation, where a worker at the CDN edge evaluates flags close to the user. It keeps rules off the browser while avoiding a trip back to your origin.

Consistency and the sticky bucketing problem

If the same user hits two different services, both must return the same variation, or your rollout looks broken and your experiment data becomes noise. This is solved with deterministic hashing: the SDK hashes a stable user key together with the flag key, maps the result onto a 0 to 100 bucket, and compares it against the rollout percentage. Because the hash is deterministic, every service computes the same bucket for the same user without coordinating.

This is why choosing a stable, consistent user key across your whole system matters more than any single line of SDK code. Get the key right and consistency is free. Get it wrong, with a per service session id, and users flicker between variations on every request. The blog post on implementing feature flags walks through choosing that key in practice.

Key takeaways

• A flag system has four layers: a management plane, a delivery layer, an evaluation engine, and a context and events pipeline. Know which one you are changing.
• Prefer local evaluation on trusted servers for microsecond decisions, and remote or edge evaluation for untrusted clients to protect rules and user data.
• Design for provider failure with default values, non blocking init, and a persisted rule snapshot. The provider advises, it does not gate your requests.
• Add a relay proxy once you have many instances, and use a stable user key so deterministic hashing keeps every service consistent.

Next, in Testing With Feature Flags, we look at how this architecture changes the way you write, run, and trust your tests.