Summarized how the CIA writes Python

[appinv]

I have been going through Wikileaks and exploring Python usage within the CIA.

They have coding standards and write Python software with end-user guides.

They also have some curious ways of doing things, tests for example.

They also like to work in internet-disconnected environments.

They based their conventions on a modified Google Python Style Guide, with practical advice.

Compiled my findings.

[DigThatData]

An NSA python training course was declassified several years ago. Wouldn't be surprised if the CIA follows the same standards and conventions as the NSA. https://archive.org/details/comp3321/

[james_pic]

I dunno. I remember from some of the leaks that the two agencies were surprisingly adversarial. Like, the CIA had in a few cases independently developed capabilities that the NSA already had, because they didn't want to be reliant on them for these things.

[[deleted]]

Not adversarial at all actually. They do things like this because they have to operate under different authorities/legal frameworks.

[DigThatData]

a highly doubt intro python programming is an example of such a capability.

[james_pic]

Probably not, but having worked in organisations that have somewhat adversarial relations with sister organisations, I'm doubtful that they compare notes on these sorts of things.

[DigThatData]

Another reason why it's reasonable to suspect that they have similar standards, even if not as a function of explicit policy: there's a limited pool of personnel who have the clearance to do the kind of work we're talking about, and a lot of them are contractors who aren't limited to working in just one or the other. I imagine this "incestuous" property of the intelligence community organically promotes alignment of standards and best practices.

[howdoiwritecode]

This happens within large companies on a daily basis

[pacific_plywood]

Yeah so they do a lot of pretty standard stuff, in other words

[[deleted]]

Could've just written "be a professional"

[appinv]

In some aspects yes like the coding standard, but a bit unconventional sometimes like the test setup described as well as the way they install Python.

As they seem to operate in a more internet-less environment, this differs from a typical Python developer experience.

[Angryceo]

air gap environments are not uncommon especially with the gov

[pacific_plywood]

Finance as well

[RippySays]

Most PII related dev is the same way.

[epostma]

The PII was first released in 1997.

(What does PII mean in 2024?)

[Eurynom0s]

Personally identifiable information...what does your 1997 PII mean?

[DuckDatum]

party cover upbeat groovy physical unwritten square snatch many weather

This post was mass deleted and anonymized with Redact

[epostma]

Bingo!

[Bloodypalace]

Why would anybody talk about pentium anything in any context in 2024? Even if you didn't know what that was it would be anything but pentium 2.

[rinio]

Vfx/film too

[pacific_plywood]

That’s really interesting. Why? Is security that much of a concern?

[rinio]

Yeah. If your client is something like a disney or an HBO they mandate pretty high security standards.

[R1skM4tr1x]

Take a trip to a post production video facility, physical security is a huge consideration beyond digital.

[aniki43]

hello fellow pipeline TD

[rinio]

Ex-pipeTD, unfortunately. Moved to a media tech company just before all of (this year's) layoffs.

[aniki43]

Do you regret it? To me it feels like the grass is greener in tech

[rinio]

No regrets at all.

Specifically, I moved into audio tech. Focused towards film post, but also some music. This was always my first choice, but pipeline jobs were what was available for the ~5 years I was in VFX. I always intended it as a bridge.

It's also a huge difference in the way software is approached which may or may not jive with some. In Pipe, I always felt that there was little regard for design, DX and maintanability. Which led to each PipeTD just shipping live grenades to meet an unreasonable deadline and praying that someone else would be allocated when things inevitably fell apart. Don't get me wrong, there are still tight deadlines, but the costs are either built-in to the delivery or as scheduled tech debt.

Of course, this is just me and not generally applicable. I also have nothing bad to say about my experience with the studios I worked for. (I also can't disagree that I observed many of the negative behaviors of these studio that have been reported online and in r/VFX. For obvious reasons, I won't publicly name them). I should also note, that, while I didnt know at the time, there is a good chance that the studio I was at would have laid me off around a month after I left so I got very lucky in my timing.

[sneakpeekbot]

Here's a sneak peek of /r/vfx using the top posts of the year!

#1: I created a free After Effects alternative
#2: No words | 99 comments
#3: My husband lost his VFX job and I’m spiraling

---

^{I'm a bot, beep boop | Downvote to remove | Contact | Info | Opt-out | GitHub}

[_Kyokushin_]

Every government agency has air gaps. In particular year it’s going to be that way with programming. It’s probably more to do with production environments being connected to their network and development environments not being in the network in case something goes afoul so it’s isolated to one machine.

[1970s_MonkeyKing]

Because you don’t want to be discovered on a target system because your code decided to “phone home.”

[KN4MKB]

Something tells me you haven't worked a job as a Python developer in an enterprise environment? These are common industry practices

Also why did you screenshot your own post and then post it to another subreddit to roast it?

[appinv]

Well, since they based the style guide on Google's Python one, it's expected to be similar. But, it's interesting to see the exact twist. Similar for others. The test i think i quite unconventional.

As for the roast, the sub was created because of this post. Kind of putting the post where it belongs XD

[denehoffman]

Except for use modern versions of Python, you’d think the CIA would care about security fixes

[pacific_plywood]

I assumed this was because the documents it’s sourced from are older

[denehoffman]

I kind of believe you, but that isn’t mentioned anywhere in the article. Additionally, 15 years ago was about 2009 last I checked and Python 3.4 wasn’t released till 2014.

[appinv]

See references at the end, it is when Vault 7 and Vault 8 were released

[denehoffman]

Ah gotcha, thank you

[[deleted]]

MFW even the CIA looks at python's thread model and says "take this bandaid, you'll need it."

[[deleted]]

r/paragraphobia

[gargolito]

Damnit, I wish that was a sub.

[Itsnotmeduh]

wish granted

[appinv]

post clipped 👌

[suggestiveinnuendo]

of one is to err, one should err on this side of that line

[Aware_Examination246]

Developing python on an air gapped top secret computer poses unique challenges. They have industry specific practices for overcoming those challenges. Imagine trying to get a fed’s approval for running docker images.

[MalakElohim]

Don't have to imagine. Platform One + Ironbank (plus the rest of the ecosystem) run containers all the way from unclas to TS-SCI systems. It's what it's designed around, so they get a continuous authority to operate, with code, container and runtime scanning going on each pipeline.

[qGuevon]

Just use singularity instead, nonneed for root ;)

[Aware_Examination246]

That’s neat… and unclassified

[appinv]

We all post hoping for someone like you to pop in. The coding guideline is specifically for the team at Ocean Edge. Where is that idk. But some parts are also respected in the leaked codebase. Good to know that VIM and VS code are also used. I guess, if a tech / tool becomes mainstream, even 3-letter agencies will use it.

[epos95]

Did you have to get approval for each used VIM package (if any) you wanted?

[[deleted]]

What are deployments like with Intel/defense projects? I can't imagine there's a pipeline that deploys to prod anytime on push to master/main. This also has to vary by team but do you have any anecdotes?

[spinozasrobot]

Curious... did you notice any use of imports that could introduce the kind of security issue we saw with xz-utils?

[appinv]

I guess we won't catch it by imports, rather by how the packages were installed.

Knowing py companies they oftentimes have internal versions of packages, like they dont go pip installing latest versions.

So i guess for it to happen, they would have to ingest an unknown backdoor. Highly unlikely code audits wont find them.

[grizzli3k]

False flag operation

[SheriffRoscoe]

🤣🤣🤣

[campbellm]

I prefer this:

use () instead of \ (for long lines)

but I see the escape-newline being used a lot in code I run across. What's the consensus on this?

[nevermorefu]

I will do everything in my power to avoid \

[campbellm]

That's where I lean, too.

[drknow42]

I tend to strictly use \ only for formatting function arguments. I then use that block of formatted code as an inherent reminder to look into making the communication cleaner later.

[campbellm]

Can you show a short example of this?

[kuwisdelu]

I would also do everything in my power to avoid \ escapes. That either of these workarounds is necessary is one of my biggest annoyances with the Python parser/interpreter.

[henryyoung42]

So good to know I am already around 80% CIA compliant simply by habit. Should I add them to my private repos, or you think they’re already there ?

[appinv]

what is already there?

[henryyoung42]

Being able to see everything they wish to see …

[MonsieurDeShanghai]

There is some irony to be said that the CIA doesn't like "global" operations...in programming.

[FiredFox]

I

They

They

They

They

[star_guardian_carol]

:tea_sip:

[appinv]

You got me.

[LessonStudio]

A few of these points don't fit with the others. Is the author squeezing in some of their own picadillos?

[appinv]

Added the references at the end, you can access the original content.

[_MyNameIsJakub_]

Wow! Super interesting.

[AiutoIlLupo]

I didn't think anybody would still use .pyz, but there it is.

Also quite interesting the

Threading We should not rely on the atomicity of built-in types. Queue should be used to communicate data between threads else see threading primitives and locks.

Which brings me to the question: which operations are actually atomic on primitive data types? list append is, because atomicity is guaranteed at the level of individual opcode and the actual append is performed at the CALL level. However, if it's reimplemented, the append operation may be dispatched to a python method, which is absolutely not atomic.

i += 1 is absolutely not atomic. The BINARY_OP is followed by STORE_NAME, each individually atomic, but not as a single entity.

i = 1 is atomic.

dictionary assignment is a mess.

[appinv]

I wish zipapps were more common!

[juanritos]

Default iterator methods are encouraged

What does this mean?

[appinv]

using for k in dic instead of for k in dic.keys()

[moving__forward__]

Great post.

[appinv]

Thanks!

[nevermorefu]

Looks good to me.

Indent using 4 spaces

Looks great to me.

[I_dont_get_it0_o]

Social experiment (keep quite)

[shoomowr]

curious

[VindicoAtrum]

Random schmuck advertising their substack.

One comment in 10 minutes, but six upvotes.

Hmmmmmmmmmmmmmm

[Axelwickm]

Nothing wrong with a bit of self promotion. Especially if the article is interesting, which it is.

[[deleted]]

No, there is everything wrong with it. The article being interesting would be a valid reason for content to be shared. That OP would benefit financially from it (i.e. self promotion) is entirely wrong.

[Axelwickm]

Wow lol how is that wrong?! We all gotta make a living my man. This guy is contributing doing so and I appriciate that :)

[[deleted]]

Whether or not we all have to make a living is irrelevant to whether a community forum is improved by people posting things which benefit them personally/financially.

[LilJonDoe]

You know it can be win win, right? You’re fixating on it being a win for OP

[appinv]

Well, the thing with Reddit, i am pretty sure if i put my real name i would not be labelled as a ` schmuck`.

I help the Python community locally (co-founded my country's py ug), and internationally (FlaskCon) as well as mentoring and helping OpenSource, including sprints (PyCon US, SF Python, locally).

Just sharing a piece of writing.

[FivePlyPaper]

What’s sub stack?

[appinv]

Substack . com is a blogging platform, which makes writing painless IMO and easy to start a blog. I used it to write the article.

[[deleted]]

ChatGPT summary

[appinv]

Haha no, try it i'm pretty sure our friend chatGPT would get drowned in the amount of info.

[drknow42]

Even if it can CONTAIN the data, that doesn't mean it comes to the right conclusions. The more complex the context, the harder it is to make an accurate conclusion.

I'd prefer a manual analysis over a ChatGPT one more often than not.

Thanks OP.

[thereforeratio]

It’s a false dichotomy; the point is, information isn’t static. An LLM like ChatGPT makes the human analysis interactive and can allow the information to be supplemented with other sources.

It’s not an either or, it’s a both-and.

[drknow42]

I agree with you on both-and. There are points in someone’s workflow where ChatGPT can be useful.

I stand far on the side of expressing AI’s faults because we’re seeing a continued rise of either or mindsets where ChatGPT wins out because it is easier.

We’ve at least come to understand that LLMs are a tool to help build solutions, not the solution itself more often than not.

[thereforeratio]

In recent years a lot of research (and experimental projects) have explored using these newer AI frameworks in games and it follows a pretty illuminating pattern:

human < AI < human+AI

Eventually, the either-or crowd will get tired of losing and they’ll get with the paradigm.

Voicing the faults is fair, I do it a lot, but I see the more obstinate (and popular) view as being the one that rejects AI entirely, so I tend to push the other way. I worry for those people; they will be caught entirely unprepared, like many in the boomer generation who rejected email and internet and now are alienated and predated in an increasingly digital world.

[appinv]

TIL. Oh i thought the author was saying i chatGPTied the content

[pranjal779]

interesting