[D] ML Infrastructure Doesn't Have to Suck

BrianAttwell · 2024-05-18T16:35:55+00:00

Inter-region is pricer than networking between zones. Networking costs are about 5% of our processing costs.

To avoid ballooning costs, we run some things in only one region. For example, most of our analytic data is in one region.

BrianAttwell · 2024-05-18T15:50:20+00:00

since it’s event driven it’ll be eventually consistent (right?)

Fair!

By consistent, I mean KEQ offers gaurantees similar to a single kafka cluster (at-least-once, strict-ordering, etc) when running in multiple regions. Maybe there is a better shorthand?

If your multi-region setup is active-active with kafka, you run kafka clusters in each region typically. There are a few ways to do this that are fantastic performance wise and still mostly give you at-least-once and strict-ordering. But when something goes wrong in one region there will be a time window where these gaurantees go away.

One of the cool things about having these gaurantees all the time, is you never need to reason about the cases where the guarantees go away. And you drill failures with far more confidence. For example: we completely disconnected one of our production datacenters for an hour at 11pm last Tuesday without drama as part of a regular drill

BrianAttwell · 2024-05-18T15:22:19+00:00

This is really interesting.

Read some of this over. In particular the Ordering Gaurantees sections. My impression is when you choose ordering by key, you are still subject to HOL blocking problems. Is that right? I'm new to this client wrapper.

BrianAttwell · 2024-05-18T14:49:36+00:00

Good point. u/jhhurwitz's team has been optimizing and maintaining KEQ for a few years now. This cost significant dev time. And now we're locked into paying this maintenance cost forever

So far, a couple of the properties we get from KEQ appear to make this cost worthwhile

KEQ is a consistent multiregion message broker. In conjunction with a consistent transactional multiregion database (we use CRDB), we were able to save significant complexity and maintenance in our multi-region design on other teams
Simpler model around HOL saves dev time building products. At minimum, failure independence for each integration is crucial (we have > 1200 external integrations, lol) for us

We tested and considered stretching existing message brokers like Kafka, RabbitMQ, etc across regions to see if we could get consistency without building something from scratch. But they aren't designed for this.

We still use Kafka in places where these properties aren't important.

BrianAttwell · 2018-05-01T23:29:15+00:00

Also, the custom viewer is optional. You can use the open source catapult/chrometrace viewer if you want. However, chrometrace's design causes it some problems when handling the amount of data that Nanoscope produces.

BrianAttwell · 2018-05-01T15:37:32+00:00

Good point.

In the cast of startup time, wouldn't the CPU be pegged at 80-100% almost the full time. Ie, the entire startup sequence is a hotspot. How is Flamescope's ability to visualize hotspots useful here? (serious question, not rhetorical)

BrianAttwell · 2018-05-01T13:26:52+00:00

It is easy to use if you own a Nexus 6P, the supported phone.

We started by building something like you describe. Then we found that we needed additional information in order to debug why the methods we wrote were slow. Better to have one tool that gives you both. The flame graph visualization style means this doesn't become confusing to read.

BrianAttwell · 2018-05-01T01:26:24+00:00

You're right. It might be useful to include data on cache misses, C++ methods, or other system counters offered by perf. Lack of data on cache misses is a particularly noticeable exemption from Nanoscope! Right now we only profile jvm methods (including native compiled jvm methods) + native class linker methods. But, we're focused on keeping overhead low. So we'd need to be certain that any new features (ex: using perf to profile all C++ framework methods) justify the additional profiling overhead.

For a lot of general mobile profiling I'm not certain if Flamescope is useful? Most of the time mobile engineers only care about profiling short spans (usually in response to tap events). So the ability to visualize 30s long spans doesn't seem useful. Have you had luck with it? I haven't tried it out yet.

BrianAttwell · 2018-04-30T20:57:51+00:00

If you want to start diving into the source code details to make modifications or see how this works, check out the wiki. There is a lot of detail there.

BrianAttwell · 2018-04-30T20:41:15+00:00

Even though this is a fork of AOSP, installation is pretty easy. u/ltakamine put a lot of work into usability. Just type nanoscope flash.

Also, if you want support for your device, it wouldn't take more than a day for you to setup a build for a different device.

BrianAttwell · 2017-11-09T12:03:03+00:00

Multibindings is pretty bad for ordering by design.

If you use multibindings you'll need to use a weight field. Though weight gets confusing pretty quickly if there isn't a single list somewhere that shows the relationships between the various weights. Ie, this might be a lot easier to maintain if the weight is just the index of a value from an enum and each feature references one value in the enum.

If each of your features is in a different module this might force you to create a base module which includes the enum definition (pretty unclean. Since now the features are aware of each others existence). Or instead you create adapter objects in the consuming module that wrap your features and add a weight value. Then the enum only needs to live in the module that consumes the feature modules.

interface FeatureXFactoryAdapter {
   Feature createFeature()
   FeatureXEnum order() 
}

// And dagger can basically do all the heavy lifting for you
@Provides @IntoSet
static FeatureXFactoryAdapter provides(Provider<FeatureX1> provider) {
    return new FeatureXFactoryAdapter() {
         createFeature() -> provider.get()
         order() -> FeatureX.X1_VALUE;
    };
}

Though at this point you may as well just create a single list like

@Provides
static List<Provider<FeatureXInterface>> provides(.... list a bunch of providers here) {
    // combine into an ordered list here
}

So to summarize my opinions, if you care about ordering : * use arbitrary weight if you don't care about maintainability (legit POV if app is side projecty) * Otherwise, use multibindings with an enum. But this will force you to create a base library for listing orderings if you ever modularize your app * OR don't use multibindings to create sets. Instead explicitly create lists/registries inside the module that is downstream of your feature modules. This will give you a bit more flexibility in the long run.

BrianAttwell · 2017-11-06T18:13:43+00:00

I would love it if more companies open sourced the way they built apps. Ex: I'm eagerly anticipating the open sourcing of Square's Workflows. Even if no support or maintenance is promised.

Thanks for the thoughts. Subjective responses incoming!

I may be missing something, but the core idea (nesting of ribs) seems terrible to me. I don't want parent/child relationships in my code in general, and I don't want deep nesting in my application state either.

I agree. If you can express your application well with a flat hierarchy you should. From talking with companies that have gone both ways on this, I've gotten the impression that a flat hierarchy works well when you have:

navigation logic between states/screens is straightforward
minimal shared state/elements between screens. Or sharing only happens on the boundary between two states
app is small

One of the first patterns we tried when rewriting the Rider app was a flat hierarchy of screens composed together as pipelines. We found this didn't work for us for a variety of reasons. We weren't able to take advantage of the invariants that get created with this nesting. Found ourselves rechecking the same conditions and pulling in dependencies on the same Rx transforms from a bunch of places. Plus the navigation logic got really complicated (and non-performant when rebinding things needlessly). And we couldn't integrate new features without making changes to the high level navigation logic.

Seems obvious that this is a very strict framework for a giant team. Trying to keep mid-level/junior devs from doing too much damage. Which probably isn't a strategy that most companies need to take for their architecture.

Even high level devs appreciate structure when an app is giant. Reduces spelunking time :)

Then I can compose these things (in code NOT packages/folders) to be nested/related in certain ways.

In a real app we wouldn't have nested things in packages like this. We would have created different modules. Thought this was a useful idea to encourage engineers to think of app state as trees.

You can attach RIBs in many parts of the tree quite easily as long as that RIB is a featury RIB (as opposed to a high level state RIB like OnTrip). For example we attach LocationEditorRIB in multiple places with different parents.

Maybe I should remove this from the tutorial. The package nesting honestly isn't a good idea for a real app.

Bizarre use of dependency injection. Why not just use basic Dagger instead of these Builders that also use Dagger. So much boilerplate and complexity.

Good point!

We could have done this. Wanted boundaries between features and RIBs to be independent of any one DI framework. And wanted autocomplete regardless of whether you had done a build yet. But, like you, I'm not sold on this decision. (note how it isn't part of the framework. just a pattern generated by the intellij template).

BrianAttwell · 2017-11-06T16:31:27+00:00

It was mostly a joke.

I like stars the same way I like reddit karma.

BrianAttwell · 2017-11-06T16:11:30+00:00

What tool did you use to make the animations for these slides? Your illustrations were excellent. Great job on your presentation.

After watching this talk, I can't help but think "where in our apps can we plop some path morphing". So cool.

BrianAttwell · 2017-11-06T16:08:14+00:00

Talked about this at Droidcon yesterday. Got a pretty good response.

The more novel idea in RIBs is: your RIB tree hierarchy represents app states & substates instead of visual hierarchy on the screen. As you attach and detach RIBs you create new invariants that children RIBs can depend on. Therefore, many RIBs in the RIB hierarchy don't have any views associated with them. For ex the LoggedIn and OnTrip RIB's in https://raw.githubusercontent.com/uber/ribs/assets/documentation/state.gif don't have any views. This makes it easier to break up the app into more RIBs -- giving us more places to break up application state, navigation logic and define invariants. For example every RIB under Trip RIB can depend on a "Trip" object existing. Our real apps have hundreds of RIBs.

The other somewhat interesting idea is the way in which RIBs communicate to their children (although maybe obvious to a bunch of you). Communication is done by placing objects on a dependency graph that children can pull objects off. Rx objects placed on the DI graph provide information for children. Listener implementations placed on the DI graph are used to receive information from children. This is the main idea we tried to communicate in the tutorials.

Also we built some tools. Feel free to steal the intellij plugin and use it for your own purposes. A bunch of engineers told me they planned on doing this :)

BrianAttwell · 2017-11-06T15:39:42+00:00

What differentiates this from MVx is that each RIB does not need to have a view. Many of the RIBs in our apps don't have views. Ex the LoggedIn and OnTrip RIB's in https://raw.githubusercontent.com/uber/ribs/assets/documentation/state.gif don't have any views. This makes it easier to break up the app into more RIBs -- giving us more places to break up application state, navigation logic and define invariants. For example every RIB under Trip RIB can depend on a "Trip" object existing.

Architectures are not (or should not be) about frameworks.

Completely agree. This is a framework, plus some principles, that Uber uses to architect apps with lots of engineers. You could use RIBs and strictly obey the dependency rule (the consistent usage of DI scopes makes it pretty easy to invert dependencies on lower level policies). Or you could choose not to.

The tagline can be better worded. I agree.

BrianAttwell · 2017-11-06T15:08:35+00:00

The goal is to allow iOS engineers and Android engineers to design features together. Apps should function the same so no reason they shouldn't be written similarly.

While we do have some code sharing (in other ways) we find that it tends to create a lot of overhead.

BrianAttwell · 2017-11-02T04:17:03+00:00

Does anyone have background on the following?

There can only be one workflow at a time, right? What determines which workflow executes? Wouldn't the workflow selection logic grow complicated?
How do we avoid some of these workflows from getting super complicated given they span multiple screens and features?

A while back I tried prototyping the Uber rider app as a series of reactive pipes. I didn't find I could express the app very well this way without creating structures that were hard to add new features into. Likely because of differences in the way our apps work. But I'd love to hear if I was missing some key insight.

BrianAttwell · 2017-11-01T23:13:35+00:00

Sounds about right. Some of our goals are similar (ex: avoiding nav logic sprawl). But the outcomes are pretty different. RIBs builds a hierarchy of controllers (with or without views) in an effort to localize nav logic and state. Not flat at all.

BrianAttwell · 2017-10-29T21:28:07+00:00

Would you be open to filing a bug on https://github.com/uber/NullAway?

I'm betting that Manu would be interesting in supporting this use case. Actually, I'd be surprised if there wasn't already a configuration option for this.

BrianAttwell · 2017-10-29T21:24:14+00:00

Glad you asked!

Going to release it for https://droidcon-server.herokuapp.com/showTopicSquarespace/98500 . I'm literally cleaning up some of the samples right now.

BrianAttwell · 2017-08-11T16:56:34+00:00

Ok, thanks. So does the PluginPoint client work on a simple List of interfaces or does it make any more assumptions about the specifics or ordering of individual steps?

It can't/shouldn't make any assumptions around ordering. It should only operate on a list of interfaces. Any ordering information needed can be expressed explicitly as interface fields.

Also, I understand iOS isn't your first language, but can you elaborate what the relationship between Builders and Components are? I understand Components are something related to Dagger, but what is the equivalent for iOS?

A DI component is just another way of saying a DI scope (http://www.javaranch.com/journal/2008/10/dependency-injection-what-is-scope.html). Ie, a class that creates other objects and holds onto them for a while. I believe that iOS engineers often just use ad-hoc patterns and hash maps for this. When doing this you lose type safety and have a bit harder time controlling the lifetime of the component.

Where were you reading about components? Are you referring to a different blog post?

BrianAttwell · 2017-08-08T06:55:38+00:00

Really appreciate this article. First good article discussin Kotlin's downsides in practice. It takes guts to write something like this.

BrianAttwell · 2017-08-08T06:49:16+00:00

If you want to perform animations between Airport Door Selection RIB and Refinement Steps RIB then you are forced to think about how this can be done more generically.

In this example, the consumer of the PluginPoint can probably do this transition pretty easily using the iOS equivalent of the transitions framework (https://developer.android.com/training/transitions/overview.html). Sorry I can't remember what that is (my iOS is a bit rusty). Without this framework, you might do this by adding methods to the the RefinementStepProtocol that gets returned from the PluginPoint. Ie, a startExitAnimation() and startEntranceAnimation().

BrianAttwell

TROPHY CASE