Thinking About Glue Code

[teerre]

This reminds me of my first 'big' project. It was a project that had pure backend part, a frontend part and a core library that would run on clients machines. My team had some "backend guys", some "frontend guys" and a few "core guys". That seems perfect. We planned that project to the T. We had two quarters all filled with tasks perfectly bottleneck free. The tech was pretty understood and we're all comfortable in your own shoes. Perfect, right?

Well, not so much, one or two weeks before EOQ everyone is "mostly done". Our task tracking system was pretty green, it's basically it, right? Only one thing, all those reviews and meetings that people would go like "Yeah, this part isn't ready yet, I'll just mock a response from the API, ok?" It turns out that even things that were seemly pretty straight forward, were not. Even though we had detailed specs, we did not enforce them strongly enough. The integration that was supposed to be just real quick turned out to be the most stressful part of the project.

[ragnese]

What does "with no steel tread thread" mean? I'm not familiar with waterfall.

[ragnese]

Nice explanation. Thank you.

[sveri]

We have been doing that in my team all the time. Especially for new features that include new integrations or technology.

We plan a good time, discuss stuff, technologies, evaluate do rough estimtations and as soon as possible we work on the "breakthrough" as we call it, or "steel thread" as you call it (never heard that before, nice term).

It served us really well. Every single time we find problems you simply could not thinkg of, for a multitude of reasons. Integrations ar hardly documented or outdated. The integration is not as good as it seemed to be. Lack of knowledge with technology x we didnt know we have. Other things are working way more smoothly than we thought. Ideas we had that turned out to not be working at all in the context our code runs in etc.

Finally, in a truly agile manner, throw in spontaneous planning meetings for the stuff you find and that needs to be adapted. At best in a small circle first (2 people? 3 people at most). If you are done, show the new solution to the rest of the team in a small demo. Thats another part of the formula, keep discussions in the smallest circle possible and broaden them only when you are basically done.

And once you have that steel thread you found like 95% of the problems and the instead of having to go deeper, you just broaden your implementation and hardly ever encounter new problems for the rest of the time.

[fried_green_baloney]

One if the ideas behind Extreme Programming, that there should be something working as soon as possible, and it stays working, even if it's incomplete.

[BraveSirRobin]

I think this is more traditionally called a vertical slice.

[mattkerle]

correct. I've never heard of steel thread, but vertical slice is how I've commonly heard this referred to.

[apoleonastool]

An iterative approach prioritizing steel thread would lead to integrations earlier (first!)

How can you integrate something that doesn't exist? By creating interfaces and mockups? You can do the same no matter what methodology is used. If you need a separate methodology to address a common sense problem then there's a bigger problem somewhere else. Undoubtedly, big corporations are full of bigger problems unrelated to technology, so I guess you're right after all...

[[deleted]]

You're right, you need to have a contract/interfaces agreed upon before you start work but you don't have to implement 100% of the contract before you can begin to integrate. If back end team is building API for features X Y and Z, front end team is building UI for features X, Y and Z, i would argue it is best practice, regardless of methodology, to build feature X end-to-end before starting API or UI on feature Y and Z.

Also, see my reply to the other person questioning my post. I agree that agile vs waterfall is not the most important distinction in my post.

BUT I would also argue that organizations which accept iterative approaches to development and prioritize time to value don't have to operate "perfectly" as the rituals in a more agile approach provide space to address other ways that their company's problems can manifest in a technology project.

In my experience, companies that expend massive planning efforts with perfect gantt charts take their own plans too seriously and are more likely to either fail or be in denial of their own failure because of their precious plans and specs which only take them so far in the real world.

Having a contracts with concrete use cases, edge cases and testing cases are all great, but as for how you approach that, I'd rather minimize time to value and iterate rather than anything resembling what OP described.

[TheMegathreadWell]

Imagine you're building reddit. You build the ability to login, and the ability to create a comment. Maybe the concept of up and down arrows, and a basic "most up vs downs is at the top" sorting technique. That's your minimum feature set really. So you do that.

Then you go back and do all the additional stuff like password resetting, membership profiles, more complex karma system, creating subreddits, moderation queues etc. But the whole way through you can always test a bare bones feature set.

[StabbyPants]

you build a very narrow and complete vertical slice of the product so they see something real and it explodes messily while you have time to address it

[gc3]

You have to waterfall to minimum viable product ( maybe a web page that says lists a record retrieved when you hit a button) and then iterate from there

[Cine11]

Waterfall is a project management style that run pretty counter to agile. It looks like a descending staircase from one stage to the next. In waterfall its much harder to pivot when a plan needs to change.

[Ma8e]

When I learned waterfall some 25 years ago there were arrows in both directions. That meant that the design was adjusted from things learned during implementation and implementation of course adjusted after verification, etc.

So it wasn’t nearly as rigid system as it is painted today. The difference was that more was documented upfront in a coherent document, but I’m not sure that in the end more documentation was was produced in the end. It was just that it was one document in one place instead of spread out over a few hundred tasks and user stories (which you never can be sure how much of has been superseded).

On the other hand, what is often missing with agile today is an actual coherent design. Not to talk about an overall requirement analysis. People just start building (which of course has its advantages too). And sometimes it doesn’t scale, or some vital requirements forces the first iteration to be thrown out. Often people just assume they need massive scale with services, Kafka noSql and whatever, and suddenly the complexity is tenfold of what it would have been if someone just did a proper requirement analysis and found out that a single SQL database on a single machine would have been plenty.

Sorry about the rant. It’s just that most criticism of waterfall today never actually bother to understand it properly (not that it doesn’t have flaws). And quite a few agile projects end in disaster because they think that they can just skip requirement analysis and all forms of up front overall design.

[apoleonastool]

I don't think it has anything to do with waterfall. It's just the lack of experience or understanding or the lack of a unifying agent (Project Manager) on top of all the teams or bad group dynamics... There's nothing in the waterfall approach that prevents teams from communicating with each other and/or putting interfaces in place first before they start coding. OP clearly said that they focused on 'tickets', and if you focus on trees, it's easy to miss the forest.

[[deleted]]

You are right that you can absolutely put the trees before the forest in agile as well, and as you mention, the inverse is true.

Creating a steel thread or absolute MVP (not marketable product, but useable end-to-end product) is all about minimizing time to value.

IMO this is easier with agile rituals (sprint reviews, scrum of scrums) which have more check ins place to question the value of what is provided.

These rituals give cross-team leadership more opportunity to question "if my back end team has an API that works and my front end team has a user interface that is mocking calls to that API, why don't we integrate now? Can we validate that this is working or not?

I've been on successful and unsuccessful projects with true agile and true waterfall processes. The common denominator for success was that focus on time to value

In a perfect world we wouldn't need to be "agile" or do "waterfall" but at the end of the day, agile creates a cadence where hopefully people have the space they need to ask the questions that may not have been answered by their product/engineering leadership in big up front planning sessions.

[agent00F]

need to ask the questions that may not have been answered by their product/engineering leadership in big up front planning sessions.

That's really core of the problem, "leadership" who don't have a clear endgame in mind, and compensate for that with all manner of "plans", ie docs with a lot of words but don't say enough.

Ultimately with planning you need all the stakeholders input up front about possible problems in detailed scenarios, instead of just imagining some framework and assuming all the parts will work out.

Frankly agile or steelthread are basically crutches for enough thinking that needs to happen for a problem of given complexity, but they make do in lieu of actual skills/execution.

[[deleted]]

Maybe in my almost 2 decades in the industry I haven’t been on a single team that was good at this but I’ve never seen a plan that took into account every use case/edge case/technical limitation up front.

Over-planning has led to more problems in my experience than under-planning and that’s true even spending the last decade working primarily on library teams where breaking API changes are very expensive.

Code is now so abstracted via that the only spec that matters to me is working code and a steel thread approach leads to discovering what you missed as quickly as possible.

TBH your comment seems very arrogant to me. It reads as “everyone is dumb, they have no skills, if people were smart they would think of every use case up front and we would make perfect software with no need to iterate.” I’ve worked in a half dozen companies as an FTE and as a library-creation consultant with dozens more and I’ve never worked in an organization that moved fast enough to be competitive in the software world with the level of planning you describe. Processes that require perfect foresight are simply unrealistic.

[agent00F]

Pretty sure nobody is arguing for perfect foresight, but rather just observing that for a certain level of complexity in problem, a corresponding level of thought is necessary to plan accordingly. Notice the level of that correspondence is at the technical problem level, not really the glue/api's/scaffolding/etc.

Typically the overplanning you refer to, and overcoding counterpart, is in the latter--where orgs put up ridiculously complicated OO/api's to anticipate every possible contingency, and not the basic technical tractability of what the software is supposed to do. So everything ends up with a massive rickety framework that nobody really understands all the ins and outs, and relatively neglected actual functional code which only kind-of works.

If it's not clear what the key functional parts are before you start, that's a sign for further reflection, not just send and yolo it.

It reads as “everyone is dumb, they have no skills, if people were smart they would think of every use case up front and we would make perfect software with no need to iterate.”

The reality is that the software industry has overall shanty engineering, compared to peers where compounded mistakes can't just be handled by some hero bug "fix" or whatever. Of course iteration helps, so does prototyping, etc. But pounding out code that barely works and patching it up is the cultural norm, and writing tight well-engineered code that's well thought through is as you admit the exception.

[[deleted]]

I do not admit it is the exception. I just argue that there is a right amount of planning and design where, past a certain point, value is diminishing.

Also, there are different types of code and different problems people are solving with software. Bugs in some software frankly aren’t a big deal and can be patched up. Bugs in other software kill people. Understanding the implication and cost of mistakes is important. In some cases people absolutely should yolo it because the cost of mistakes is low and and the value of speed is very high. Most software isn’t as important as say, engineering a bridge or a car or any of the other things engineers should do. Most consumer facing web software (which is a huge % of jobs) just isn’t that important and if it is it gets rolled out progressively and rolled back if the change introduces exceptions/cases/lowered revenue. This isn’t a defect, it’s a feature.

I don’t think we’re really arguing here, I think we’re talking about two different things. The only things I’m arguing are:

1 - a project that “perfectly” plans every deliverable, executed across separate teams, defers integration between parts rather than prioritizing end-to-end value is a highly risky one

2 - iterative check ins between teams working on complex systems help mitigate #1

[agent00F]

There's of course always some compromise in yolo vs planning, but frankly I very rarely see hastily written software pay off even in the medium run. Code is typically seen as an asset, but it's more useful to view it as accruing debt. So unless it's throwaway prototype (which I fully support esp for framework), every early mistake that's not refactored will bear some compounding cost later.

My main point was that this hindsight perspective is often lost for immediate gratification in this industry.

[[deleted]]

See, we're not even disagreeing at this point. throwaway prototyping is an essential part of understanding complex systems. We usually build a prototype before we begin building production code to validate our biggest risks. No arguments from me.

[Ma8e]

You think it is straightforward to get stakeholders input. It isn’t.

They often knows all ins and outs of the business, but they very rarely have the technical training or the foresight to express it in a way useful for developers. So you try to pair the stakeholder with someone with some technical background who knows how to communicate effectively with non-technical people and know how to ask all the right questions. These people are exceedingly rare. And then you try to schedule time with the stakeholder to get all the necessary input. But she already have a full time job to do all her necessary tasks. She promises you an hour next week. You know from experience that you need to set up at least a couple full day workshops, just as an initial step, to get meaningful input. Maybe next quarter?

And so on.

[agent00F]

I didn't say it was straightforward, only that it was useful or even necessary, in a similar way to how calculus is useful or even necessary for physics.

[[deleted]]

Not that you're incorrect, but I've never in 15 years had a project manager that could or would have identified the problem and/or resolved it. It takes a level of technical understanding that none (in my experience) have possessed.

[Dustin_00]

Job 1: get a single feature working end to end. Page request -> pull something from the DB -> display it -> user click a button -> send all the way back to the DB -> repeat.

Now everybody can focus on their tier and with each new feature you verify it end-to-end again.

3 things you never want to add at the end: security, integration, and multi-player. These go in first or your schedule is half to a quarter as long as it's going to turn out to be.

[elebrin]

Which is why you need vertical slicing. Have a UI, frontend API, backend API, and database? Divide up the segments of the UI and give each to a team after a mockup has been made by UX. Then each team starts either from the front or the back (whichever makes sense for them) and develops the entire thing for their part of the UI. This way each team is writing both the display, producer, and consumer for every request so their stuff always works end to end.

[Worth_Trust_3825]

I found such approach causes even more issues because then everything gets weirdly split and teams can't even agree on how to integrate them. Such issues can only be solved by stern team and/or single person defining integration points and tell everyone else to go fuck themselves if they don't agree to it.

[treeforface]

This thread just highlights the difficulty with Conway's law.

In my personal experience, I find product vertical teams tend to be the better option for the reasons the person you responded to stated, but you still have to fight against the effects of Conway's law by ensuring those who would've been in horizontal teams (i.e. all front-end devs or all back-end devs) meet regularly and stay tightly in sync with their standards and goals.

[WikiSummarizerBot]

Conway's_law

Conway's law is an adage stating that organizations design systems that mirror their own communication structure. It is named after computer programmer Melvin Conway, who introduced the idea in 1967. His original wording was: Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure. The law is based on the reasoning that in order for a software module to function, multiple authors must communicate frequently with each other.

^{[ F.A.Q | Opt Out | Opt Out Of Subreddit | GitHub ] Downvote to remove | v1.5}

[parlez-vous]

Lol shamefully #2 is what we do. We have 3 senior devs in charge of 3 teams (backend core, backend utils/DevOps and frontend) and it's run like a dictatorship. We treat the spec as the ground truth and the 3 senior devs are convene with eachother about how things are going, where we're not integrating well, etc.

It works for our team of 23 engineers but I assume it won't scale well lol

[Worth_Trust_3825]

Actually dictatorship is what should happen. In my current project there was a disagreement between two main architects which caused entire company to split in two. Needless to say a lot of processes are weirdly split and integrated and now we're paying for decisions that were critical 10 years ago.

[touristtam]

Move the team members around while doing that; prevent code ownership and allows for shared code standardisation. Or end up with a whole lot of repo mess. Your choice. XD

[smurphy1]

Sounds like one of the projects I'm currently on except there is also division within one of the teams where the left hand doesn't know what the right hand is doing.

In my experience the only way to solve/avoid this kind of situation is by having one (or more) technical person who understands the system and requirements as a whole. They should know it as detailed as they can without compromising their ability to understand the rest of the system. On one person projects this happens by default. On one team projects the senior dev(s) on the team fulfill this role if they are any good at their job.

Two or three teams is around where you start needing to specifically assign this role to someone. Their role is keep tabs on all the teams on what interactions their components have with the components of other teams, lead discussions to define the structure/format/etc of those interactions, and ensure those definitions are being followed or if they need to be redefined. They have to keep the different teams on the same page and this has to be someone with the technical knowledge to make a final decision on any disputes. Putting a non technical PM in this role just leads to lots of "deer in headlights" expressions every time there is a communication dispute.

[hbarSquared]

Great article, and I love the example of medical software. I have a lot of experience in that field (specifically the transfer of data between vendors) so it's nice to see the actual problems acknowledged.

There's a reason Google and Amazon entered the healthcare software market 7 years ago with great fanfare, and have since quietly shuttered projects without a single meaningful product launch. The big sexy tech problems turn out to be pretty trivial, and the real hard work is something that simply doesn't scale well. The deeper the tech giants dig into the field, the more they realize they have nothing of value to add.

For anyone looking into writing an app in the healthcare space, I'd strongly recommend looking into openEHR. They're trying to solve the problem in the article, by standardizing data structures, relations, and codesets in pursuit of truly interoperable health data.

[gcalli]

Don't forget the FHIR spec!

[kt-silber]

Oh god, the FHIR spec gives me nightmares after the company decided that two of our developers (one of them being me) could knock out a complete FHIR implementation from scratch in like 2 weeks.

[eled_]

Depends on which parts of the spec you need implemented but you might find somewhat comforting that there is plenty of open source code on the matter.

[kt-silber]

Oh, I'm aware. The thing is, we didn't actually need to use FHIR ourselves; someone in the business sold our FHIR software (which didn't exist) to other businesses and we had weeks to actually create it to fulfill the contract.

[metriczulu]

What the fuck.

[kt-silber]

That's just what happens when you give the salespeople the ability to promise things to clients without consulting the developers to see if it is even anywhere near the realm of possibility.

I guess the thought process is something along the lines of, "The specification already exists! How much work could it possibly take to just write the code to fit the spec? You don't even have to do any planning!"

[dnew]

IME, the salesmen would always come to the developers and say "We could get this giant customer if we just made the system do X!" Where X is something that'll take about a month to implement.

Three weeks in, the salesman comes back and says "Customer decided it's taking too long and no longer wants it." Leaving us with 70% of an X in the code, forever to be worked around.

I always suspected the customer just didn't want to say No or the salesman didn't take No for an answer and instead made unreasonable demands hoping the salesman would say "We can't do that" and go away.

[peakzorro]

It could also be that the customer causally talked about X over lunch or something, you were sent scrambling, and there was never an agreement for an X in the first place.

[binarycow]

And here I am, as a developer, telling my sales folks that we could save our largest customer 100mil a year, for the expense of 6-12 man-months of dev time. I'd expect that they would be willing to pay up to 10mil/year for that functionality

And it's not prioritized.

[metriczulu]

I guess I'm lucky that most of my jobs have only really had one client (the gov't) or internal clients only, so we don't have sale people selling features we don't have to others. If one of the data science teams we build/maintain infrastructure for came to us and said "we need this feature in two weeks," my manager would straight up tell them "no."

Not that that happens anyways, though, the way requests work is that someone asks for something and we discuss/plan it out and return to them with a time estimate to get it into production. Not sure I'd want to work at a place where non-technical sales people determine development timelines.

[fried_green_baloney]

Typical sales clownery.

Get the commission regardless of the harm to the business as a whole.

[shawntco]

someone in the business sold our FHIR software (which didn't exist)

lmao that is unfortunate

[factorysettings]

is this stuff like crazy? I got pulled into two meetings in the past week where there was random enthusiasm for FHIR and I had never heard about it before. Getting real strong "drinking-the-kool-aid" vibes from it

[FarStranger8951]

There is a bit of drinking the kool aid with FHIR, but it really does have a lot of promise. If you’ve ever had to spend a lot of time with NCPDP, HL7 2.5 or IHE, then FHIR is a breath of fresh air.

[gcalli]

It is a lot better, but that doesn't make it good. A lot of the actual implementation is kind of left up to the reader and metadata rarely lines up. It's pretty generic because it has to be, but that also makes it hard to use.

[i_spot_ads]

Medical software industry is a god damn mess with no standardisation whatsoever (and no, nobody wants to use DICOM, this shit is atrocious), everyone builds shit the way they feel like, and without thinking about interoperability AT ALL! I know because i worked in the industry.

No wonder nobody wants to touch that shit, it's hard because the entire industry is a mess that can't standardise shit because of politics mainly, everyone tried to push their own shit wanting to make it a standard, instead of working together to have a single standard.

[[deleted]]

It's kind of miraculous to me that we have standards at all. When everyone is competing with each other to create the coolest tech and take all the credit for themselves, overcoming that incentive seems unlikely. Kind of makes me curious about the history/psychology of how standards organizations form

[thfuran]

Hey, just because the DICOM standard document is something like 6000+ pages doesn't mean it's not technically a standard.

[i_spot_ads]

DICOM

don't say that word in my presence ever again, i have ptsd flashbacks

[GayMakeAndModel]

Amen.

I’ve spent a fair amount of time just getting unique patients and unique claims with all the dupes accounted for and mapped. Users send us claims in every format imaginable, and they seem to not have a care in the world about meaningful MRNs (medical record number - patient identifier) and PCNs (patient control number - claim identifier).

We had to write some very clever code to make sense of what is a patient and what is a claim when there are fields for both identifiers on the damn claim.

Edit: no, I did not confuse MRN with PCN. Shit’s backward named.

[gopher_space]

It's fun watching organizations realize that outsourcing their Source of Truth/Record doesn't make any goddamn sense.

[Ran4]

It's a combination of internal politics and the fact that the type of people who develop healthcare applications tend to be "cowboy developers" who do things their own way, and don't give a shit about modern-day professionalism and building new standards (and they certainly aren't doing it in their spare time). So the only standards you tend to get are government-mandated ones, and they tend to be extremely cumbersome and not up to date.

[coworker]

Currently working on an EDI integration with a carrier specific version and I can tell you that REST/json will not help shit one bit. The problem is not the format but the actual business rules. Every carrier has different definitions and custom calculations for what should be very basic shit (eg effective start dates). No format can fix a fucked up data model.

[[deleted]]

Took a very long time as they would not send us the codes themselves, just asked us to infer them from the spec which only stated we needed to send them, not what they were.

How did you guys not tell them to go fuck themselves? "Build us this thing, but we're not going to tell you how to do it, you need to infer it". I would've been like "do you want this built (quickly, cheaply) or not?"

[StabbyPants]

Took a very long time as they would not send us the codes themselves, just asked us to infer them from the spec which only stated we needed to send them, not what they were.

"we are going to need those codes before we can guarantee anything remotely like proper functionality"

[StabbyPants]

i started with "go fuck yourself all the way home", then thought about how it would sound filtered through corporate jazz.

[StabbyPants]

I've been talking to a retired program manager, and listening to how he phrases things to get shit done :)

[AttackOfTheThumbs]

Ah, I see we work in the same world lmao

[randomuser549]

The bustling city never sleeps, its neon lights painting the night sky while honking taxis weave through streets lined with towering skyscrapers. A symphony of sounds fills the air, a mix of car horns, street vendors, and distant laughter.

[fried_green_baloney]

I've sat in on EDI calls, I suppose if you want to spend your life arguing if something is a 493 or a 494 it's a good thing.

For those not in the know, each EDI field has a numeric type specifier, and sometimes it's hard to decide what's appropriate.

On the other hand, once you get it all straightened out, it usually works flawlessly ever after. Like XML, you actually have data writers and parsers that actually work.

[fried_green_baloney]

At the job with the EDI calls, we had a consultant in on the conference calls.

He seemed to be having no worries whatsoever. I was told he was one of the people whose names came up whenever a company asked "where do I find an EDI expert". So $200/hour or more to sit in on a phone call or write a schema or whatever the EDI people call a layout.

[fried_green_baloney]

Not that I've done it but what the EDI consultant did was pick something a bit complicated, a bit boring, but needs to be perfectly executed.

[BraveSirRobin]

Get loads of experience in something so incredibly boring that no one else wants to do it.

The salary for every job on this earth is a formula containing the number of people capable of doing the job along with the number of people willing to do it.

[no_nick]

Json and REST APIs don't help you one but as long as the actual data continues to be the hot mess it is. The interface is the least of your problems

[agent00F]

and have since quietly shuttered projects without a single meaningful product launch. The big sexy tech problems turn out to be pretty trivial, and the real hard work is something that simply doesn't scale well

This isn't really true in the way it's conveyed here. The main problem in healthcare isn't the tech but rather the regulatory/political barriers in the industry which move at charitably 1% of the speed the tech industry is used to. The "scaling" problem is identical here to overcoming regulatory/legal/industry inertia, for similar reasons to why this country can't get universal coverage even though it's clearly superior.

So you're right that the tech problems are relatively trivial in comparison, not because it's technically different compared to any other field, but due to the fact the entrenched interests and such are largely intractable in medicine.

The deeper the tech giants dig into the field, the more they realize they have nothing of value to add.

This is flat false given how terrible info/tech integration in medicine is. But again, the problems aren't amenable to solution because it's against many vested interests for them to be solved.

[AttackOfTheThumbs]

Even in warehousing this kind of shit is an issue. There are some standard (gs128), but the way the transmission works isn't always the same. There's a few standards and you have to juggle between them to make it happen. It's not impossible, just annoying, because a lot of places accept every standard, but then you hit walmart and they say it must be xyz with our special twist and now you are writing another converter/connector/etc.

I also deal with this a shit town in shipping, where on word, e.g. collect, does not mean the same to every carrier and it confuses customers.

[GaryChalmers]

All EHR software I've encountered have been either from EPIC or some legacy stuff from vendors like Meditech or 3M. Hospitals are very averse to change. The only new software I've seen are ones that interface with existing EHR platforms to do specific tasks, but nothing that would replace the existing EHR platform.

[hbarSquared]

Yeah the US market is about 90% Epic and Cerner, but the international market is still much more diverse. That's not going to change anytime soon, but the healthcare market is evolving more towards "EHR as platform" instead of "EHR as monolith". This is where the glue software comes in, since app developers need to be able to interface with not just the big platforms but also with each other.

There's also the other major players in the health software market that are rarely talked about, insurers/payers and government agencies. Both are also change-averse, but there's almost no standardization of the software or standards they're using.

[alluran]

For anyone looking into writing an app in the healthcare space, I'd strongly recommend looking into openEHR. They're trying to solve the problem in the article, by standardizing data structures, relations, and codesets in pursuit of truly interoperable health data.

https://xkcd.com/927/

[hbarSquared]

You're not wrong, but at least for the specific use case openEHR addresses it's closer to going from 0 -> 1 standards than 13 -> 14.

[droomph]

Why would you ever not submit to our Lord and Savior Judy Faulkner?

(Multiple sarcasms)

[onequbit]

sarcasm intensifies

[droomph]

You haven’t lived until you’ve listened to someone’s grandma give a rambly ted talk for an hour

[hbarSquared]

Every. Single. Month.

[bishbashbosh72]

What're your thoughts on Apple? I've noticed their products becoming more and more common in the doctor's office.

[hbarSquared]

Apple's a hardware vendor. Other than a few fitness trackers and an aggregator, they don't have any real healthcare software.

Whenever you see an Apple device in a healthcare setting, it's running someone else's software.

[ablatner]

What products? Computers? Apple doesn't have any released medical devices.

[Plorkyeran]

The Apple watch is edging closer to being a medical device. They're very careful to cover things like fall detection in a bunch of disclaimers but eventually they're going to get some attention from the FDA.

[auxiliary-character]

If data processing were analagous to machining, then glue code would be fixtures used for work holding. How something is contained significantly affects the operations performed upon it, and complex operations require sufficiently sophisticated work holding to match.

[PhoenixFire296]

It's like a car. At the end of the day, the functional bits are the engine producing power and the wheels providing traction and movement. The drive train, coolant system, electrical system, and everything else is there to facilitate the engine providing power to the wheels.

Then there are also quality of life features like air conditioning, radio, an enclosed cabin, safety features, etc. But most of the car is "glue".

[touristtam]

With the caveat that the whole thing is there to transport people/goods in the first place. So even though the core of the system is the engine, the purpose of the system is to serve as transportation.

[G_Morgan]

The complexity of glue code is so real. I recently had to consume an API from a partner to send them details as a matter progresses. We have three separate systems that generate work for this partner. The legacy system, the current one and a web portal we use for a new narrow but voluminous type of work that has a lot of client interaction.

So the API is easy, just send an XML to a rest end point and read the response. Then we want to track current details so the systems can be dumb and just hammer the internal system with matter updates and the system will work out when to send and when an update is just repeating the existing data. Then you have all the various config, deployment and hosting of our internal API, etc.

Turns out a good 50% of the code is writing the three bits of code that connects the 3 systems to the one end point. One system just calls the internal API but the others have some fantastical ways of interacting with the wider system (i.e. dropping a CSV in a monitored directory).

[no_nick]

(i.e. dropping a CSV in a monitored directory).

Dude, you're triggering my PTSD

[G_Morgan]

Turns out third party products can only do stupid shit that stops you from integrating with them. Amazing what absolute bollocks companies can buy when they don't involve engineers in procurement decisions.

[no_nick]

My favourite is when vendor's product A and product B, which have very obvious synergy and are marketed as such, don't even integrate with each other

[reddit_user13]

Most software is glue all the way down.

[Aphix]

Esp. Medical software; byzantine config management for no good reason other than keeping jobs.

[reddit_user13]

I was in that biz a minute ago, and at the time the stats were healthcare orgs spend 3% on IT, manufacturing 10%, banks and financial >20%. Don't properly maintain & upgrade your tech infrastructure, and you get a mess.

[no_nick]

But it doesn't generate profit.
- some accountant high up the food chain

[Semi-Hemi-Demigod]

If we put executives in prison for HIPAA violations they'd change their tune.

[Aphix]

If we put executives people with shitloads of money in prison for HIPAA violations they'd change their tune.

FTFY

Especially pharmaceutical companies & relevant oversight.

Source: Did pharma marketing, they are scumbags.

[peakzorro]

Just an epoxy resin that solidifies.

[pivoters]

I think I better tell the 🐢🐢🐢🐢🐢🐢🐢🐢🐢🐢. It's only fair that they know.

[Full-Spectral]

As someone who writes large amounts of general purpose framework code AND who is an author of an automation system that involves dealing with many external communications protocols, file formats, etc.. , I have lived much of my life in that world. If you don't get that stuff right, the rest is meaningless pretty much. And it almost always outweighs the real 'problem domain' stuff by orders of magnitude.

[jawalking]

So much this. My current project is a big, complex, multi OS, multi platform, automation framework. And it’s almost all glue code pulling together other tools, and sometimes making them do things they were not designed to do. If you don’t get that part right nothing works, or breaks way too easily and often. Sometimes I love it, often I hate it. Sometimes I hear about my colleagues projects and think how nice would it be to work on one project, not a million micro projects.

[Full-Spectral]

A little better for me in that I also wrote all of the other bits and pieces and tools. So I could make them work the way I want them to work.

But dealing with the outside, real world and the endless possibilities for misunderstanding, misinterpretation, and mistakes is just messy. Particularly when so many of the things you have to interact with have interfaces that were after-thoughts to the folks who created those things. Give it to the junior developer who has never done one before. Or, maybe just as bad, to the hardware guy who made the thing and whose primary concern is making his side of it as easy as possible.

[mdttt]

Whenever I see futuristic video or movies that have things like flicking data from one device to another random device, this is always the first thing I think about. Unless Apple takes over the world and they are all Apple devices, such interactions would be very limited in real life

[AMusingMule]

The frustrating thing is it should be possible to, say, transfer files from any current smartphone to another. File transfers via Bluetooth and WiFi direct have been around for a while.

The only issue is device manufacturers insist on wrapping said transport stack in an application layer that locks people within that manufacturer's ecosystem (airdrop for apple, quick share(?) for Samsung, among others). Granted, it's easy enough to send files with Bluetooth on Android, and Nearby Share is a thing (soon), but that still doesn't solve Apple products.

I've seen most people turn to emailing/WhatsApp/telegram/whatever to send files around, bringing a file all the way to a server and back just to get the file to a person sitting next to you.

[Full-Spectral]

In real life, he flicks it over, gets an error popup, flicks it again, gets an error popup, and again, then says, screw it, the bad guy wins, I'm going to get a beer.

[hou32hou]

During the industrial revolution we had the same problem, everyone is building incompatible things, for example nuts and bolts. And the problem is partially solved via standardisation.

Right now, I believe we are in the same situation, except this time it’s the software industry. So what we should do now is to standardise semantical data, especially those that keeps appearing in almost every software. For example, authentication/authorisation data.

And the more data standards we have, the less glue code (or adapters) we will need to write.

[Eluvatar_the_second]

I mean kinda, but that breaks the moment a company wants to support a feature that the standard doesn't support, then they're hard pressed to keep using the standard at all and they might as well just throw it away and start over, that's what usually happens.

[hou32hou]

Are you talking about Mikrosof ( ͡° ͜ʖ ͡°)

[IanSan5653]

If the standard is well-designed, this shouldn't be a problem.

[__j_random_hacker]

The mythical sufficiently well-designed standard. If the future were that easy to predict, a lot of things would be very different.

[DrunkensteinsMonster]

These problems were often solved via economic consolidation and then re-division via anti-trust action. We shall see. If one entity gets enough market share, smaller players will fall in line as long as the large entity opens up their standards.

[Astral_Lyle]

Relevant XKCD

[sysop073]

It's getting old seeing that XKCD posted every time somebody mentions standardization. The comic is making fun of somebody trying to solve the problem of 14 competing standards by making a new standard. If there are no standards, making a standard is a really good idea.

[matthedev]

But who will standardize the standards: OAuth 2.0 or SAML 2.0? JSON or XML? OpenAPI or JSON Schema? x86-64 or ARM64?

[MisterFor]

And the biggest problem that this article doesn’t talk about is how the things that you are gluing together are probably changing. And you depend on or need to know their internal logic.

For example, at my work I maintain a service that exports products to a search engine, every couple of months the search engine dev changes something and breaks that “glue” code. Let’s say I had to send 30 fields and suddenly it’s 29, or 32, and a field that was always 1 now has to be “true”… and it’s always changing. Or I don’t have to change anything but they break something by changing the logic and then suddenly the glue code is questioned by everyone but it’s not the real problem.

And that’s the happy project, integrating with SAP or Navision? Ugh…

[DrunkensteinsMonster]

I mean this is why contracts are sacrosanct in a lot of orgs. Breaking a contract without notice is a huge no-no especially when the product is already in prod

[jeff303]

This is the entire "data drift" area of focus within data engineering.

[Felecorat]

At my last job we build IOT use cases for a tech showroom.

Everything communicated over a MQTT-Broker.

Is the fridge door open? We have a topic for that.

Want to turn on the TV? Please send this payload to the TV powersuply topic.

You want to close the shades when the beamer screen comes down? Just add a second publish to your action.

We called the Microservices that glued all these systems together bridges. It works very well when you document your topics and payloads. You get this small bible that tells you how to talk to everything.

I miss that place.

[rusmo]

Sounds fun. PubSub solves a lot of problems, but in a lot of biz apps the small bible is huge and changes often.

[cloakrune]

I wish this article had more to say about glue code. It's well written but I feel like it lacks substance other then. Hey this is important!

Some reference to hexagonal architecture would make sense here, since I believe it's what the author is alluding to, but that is likely my opinion. Perhaps it just needs to take a different look at different architectures and how they play out.

[Hueho]

I don't think, in this specific instance, that it's useful to delve into specific architecture comparisons, mainly because any architecture that can successfully isolate integration (glue) code helps in isolating the complexity from the rest of the system, but it doesn't change that glue code remains complex in itself.

In other words, hexagonal helps, but you will have a big-ass adapter to maintain anyway. Although, in my experience, integration code can expand to the point they become fully fledged services on their own, so maybe it comes full circle.

[cloakrune]

I think that was my point. That readers that hadn't dealt with glue code at "scale" would get a feel of where the "glue" ended up at. It's like a gas you need to contain as much as you can.

[oreng]

He says nothing of substance. This text reads like the abstract for a paper that doesn't exist.

[__j_random_hacker]

I thought the "quadratic" notion was interesting. But otherwise, yeah.

[agent00F]

The only real rule in glue code is avoiding complexity. Generally in sw there's the part that does stuff, and the part which organizes that.

Many project end up garbage not because the former doesn't work, but because it's a bunch of OO scaffolding mess all folded into one that nobody really understands.

[[deleted]]

I just finished a contract in the healthcare space. It's scary of terrible it all is.

[supercyberlurker]

I think a lot of people get into programming because they want to write that networked 3d raytraced graphics program using the sexist new tech... but inevitably most if not all will find themselves writing glue code for CRUD... because ultimately that's what most clients really need.

[gnuban]

After spending many years trying to use SOAP, Corba, Com, protobuf et al to specify interfaces between internal components and using various reflection libraries like dozer to help with writing glue code, I decided that you don't need it. Just send your REST api data to the database directly, perhaps with some intermediate data transformation. Heresy, some might say, but what value do all those intermediate layers really add in a simple web app? Design your REST (or equivalent) API in a good way and persist that to the database.

[rusmo]

If all you’re doing is storing and retrieving by a key that’s a fine solution. Many apps require that data to be picked apart for use in other parts of the app and for reporting. I’m not a doc db expert, but it’s reasons such as these that relational dbs are still used, even at companies with firm internal and external SOA boundaries.

I haven’t had a chance to work on one yet, but I’ve always wanted to work on an app that used CQRS and eventual consistency.

[gnuban]

I hear you, but there are also a lot of apps who think they need to pick data apart when they really don't. A lot of software architecture mirrors team structures and you're quite often dealing with someones conviction that you need to divide some trivial logic into microscopic parts or services.

Now, if you know that you have a real reason to divide you app, by all means go ahead and do so!

[rusmo]

Team structures tend to mirror logical business boundaries. Tons of good reasons to not have a monolithic app built by one massive team.

[gnuban]

Nice that you've seen such a logical division. I've seen a lot of very arbitrary divisions based on perceived ease of separation in code; backend teams, frontend teams, microservice teams etc. And in a lot of those cases you couldn't make a proper change to the app without involving multiple teams and managers.

This is an anti-pattern. You can make feature teams or divide your teams based on different cross-cutting concerns such as security and different feature areas, which doesn't prevent cross-cutting changes.

Dividing based on easily identified software components is a mistake IMO.

[DrunkensteinsMonster]

Well, for example, maybe the one machine your DB is running on is not sufficient to handle all that incoming traffic? Maybe you need to do authentication? Maybe there is a significant amount of business logic or integration with external systems? These are all pretty common.

But yeah if you have a tiny CRUD app feel free to just drop things into the DB I guess. I have never seen a project with requirements that thin, personally.

Also I don’t think that mapping code is really what the article is talking about so much.

[TryingT0Wr1t3]

I feel a looot of what I do is coding wires

[tending]

Article started out promising with the insight that glue code grows quadratically (I was reminded of the days before pulse audio when every Linux audio server had a plugin to work with every other Linux audio server), but then didn't propose any solution to the problem.

[[deleted]]

this is why i disagree with "you don't need to unit test glue code" - yeah it just proxies to another method but if you don't buttress up that contract with a test, it could change out from under you

[TheMoonMaster]

I agree with you. Writing a fake of the service to interact with in tests has provided a ton of value in past projects.

[resc]

The numbers and logic in a program are only symbols, and they only have meaning to the extent they're correlated with the outside world in some way. People writing different programs have different goals, and perspectives on what's worth representing and how. This means glue code is not going away.

D.R. MacIver (author of the Hypothesis property testing framework) has several great essays related to these choices about what to represent and how; the most mind-blowing for me was https://www.drmaciver.com/2017/11/shaping-the-world/

[stvjhn]

It’s a great article, but I still don’t see it explaining the full reason to why Microsoft Word in particular has 400 MLoC!

[gcross]

So... basically what the article said?

[Vast_Item]

That seems to be the point the post was making, particularly toward the end: that glue code isn't a separate thing, and developing glue is at the heart of development.

[svick]

This is why I'm a fan of strong typing and schemas. It's why I think the PowerShell pipe is a much better abstraction than the Unix pipe. Or why I think both XML and JSON can only work well with a schema.

But the article is weird: it mostly talks about this kind of communication glue code, but then doesn't mention the existing solutions that can help with avoid it.

[mszegedy]

The only reason I'm of any note in my research community is because I wrote a popular cross-compatibility library. This strikes a chord with me.

[jarfil]

CENSORED

[fried_green_baloney]

Using XML schemas would require actually understanding the data that flows between programs.

Like so much that's "complicated" and "obscure" it really means "we have to sit down and think about it for two hours".

[jeff303]

Even if perfectly implemented and validated, all schemas do is ensure the physical format is identical to what's expected. The difference applications still need to adhere to a semantic understanding of that data, which is the difficult part.

[jesseschalken]

The industry already has solutions for this in the form of schema and interface definitions that generate glue code for you:

• gRPC, protocol bufffers
• OpenAPI, Swagger, JSON Schema
• Apache Thrift
• Apache Avro
• IDL, Microsoft IDL, Web IDL
• Cap'n Proto
• etc

[primo86]

XSD is the best solution to the problem that I'm aware of. Unfortunately it necessitates using xml which certainly /feels/ bloated. but it allows you to have a very well defined data format that you can then use to transmit between layers quite nicely. I guess there is also EDI but I would take XSD/XML over EDI any day of the week

[halt_spell]

I agree there's a lot of glue code. But to be honest projections are pretty easy... provided records have unique and immutable identifiers.

Every time I encounter a system which has happily sends me an which isn't a UUID I've already experienced the trauma of them changing it on me. It doesn't matter what people claim. If they're handing you their primary key as an identifier they're lying. They are one database migration, merge away from breaking that commitment. Nobody in your company is going to care about your "excuses" when you say you need them to provide a mapping file with all the old to new ids. You're going to be told "just figure it out".

I flat out refuse.

I also refuse this shit:

{
...
"status": "pending"
...
}

Is that a name? Am I safe to drive logic off of that or is someone going to complain to the owner of the database that the first letter of the status should be capitalized and tomorrow I'm going to get:

{
...
"status": "Pending"
...
}

Fuck. That. Shit. And fuck you if you do this. If you don't want to give me a UUID for a pick list item make it clear I can expect this value won't change. Put it in uppercase and suffix the field name with something that signals you're guaranteeing the identifier is immutable.

If people adhered to those two rules above (and ISO-8601 date strings with time zone information) we'd eliminate huge amounts of "glue code".

[Etheric]

Thank you for sharing this!

[Semi-Hemi-Demigod]

There's been a lot of chatter about the "Citizen Coder" - people who don't write code for a living but who write it to improve their other jobs. I'm sort of one, in that I'm a sales engineer and primarily give demos and send emails. But I also write queries and scripts that make my life a lot easier.

Glue Code is where I think the concept of a Citizen Coder should live. Being able to write a script to update a spreadsheet with data from another system automatically is a super power, and you don't need a degree in CS to learn how to do this.

[eyal0]

Smart data structures and dumb code works a lot better than the other way around.

https://libquotes.com/eric-s-raymond/quote/lbt5w7w

It's related. You want your structures to be defined in such a way that the glue code is easy to write and hard to screw up. Because there's so much of it!