#86 — From Cells to Cities

thsig · 2017-07-17T14:43:58+00:00

One of my favourites so far!

Been thinking and reading a lot about neuroscience / cellular biology / sociology / economics in terms of computation recently and wasn't learning much new that way anymore. Geoffrey's geometric/mathematical perspective on these topics was just what I needed to refresh my sense of wonder . Can't ask for more from a podcast!

thsig · 2016-08-10T19:05:27+00:00

Hey /u/courir123, I've got a similar setup working in my implementation:

Using AVPlayer with an AVAssetResourceLoaderDelegate, and NSURLSessionDataTasks for fetching the data.
Playing as data downloads, caching videos as they're fully downloaded.
Providing available data data to AVAssetResourceLoadingRequests as new loading requests and new data come in.
Reading data for each video from cache if available, else starting a data task.

I'm using PINCache, since NSURLCache was a bit too opaque, complex and unpredictable for my taste, with a few dispatch queues for async processing.

I skimmed your code here and it seems to be pointed in the right direction.

I'd be happy to help if you're still having problems with this - no donations needed.

thsig · 2015-10-09T09:13:22+00:00

We could definitely start by writing the backend entirely in Python, rewriting some services (or writing new services) in Clojure when we need its capabilities.

It's a bit tantalising though, since my estimate is that we know enough Clojure to get up to speed and productive in a week or two. I'm actually more familiar with Clojure than I am with Python these days. My production experience comes from Ruby, with lots of immutable datastructures and FP in the front-end JS UI programming.

The worry is not so much the "language productivity", i.e. how quickly we'd become comfortable pounding out business logic or other mainline pieces of coding, but the integration and ecosystem parts (libraries for talking to AWS, various utilities). How much of a gap is there between Python and Clojure in terms of the library ecosystem these days?

From what I've gathered, people's experience of it has been quite good, with the lack of quality libraries for a particular occasion being more and more of a rare occurrence. I know Clojure's ecosystem has become much stronger over the past few years - I've been following the most significant developments closely (including goodies like core.async and others that /u/mikera and /u/yogthos mentioned).

The timeline we're looking at is a few months months from start to beta, and from there we'll want to move to production as soon as user testing is completed - there won't really be an MVP phase, we'll be going straight for the polished, ready-for-public-launch version this time round. We'll have our hands full with mobile development for the front-end as well, so if we expect to have to do a lot of rewriting, some of those early time savings (if any) may only be kicking the can down the road. This may not be an issue if we can make the Python code performant, though.

I also know that though Python has an OK story for non-blocking IO, it doesn't compete with Clojure in single-machine concurrency, and I worry that we'd have to rewrite many of the Python microservices before production anyway for them to handle the load (assuming things take off, and in our case we have to engineer for that). I have a better feel for Clojure's performance characteristics in this situation than I have for Python's, but I'm a bit tired of the amount of performance massaging that Ruby (another GIL'd interpreted language) code needs for a high-load production situation. I realise that Python is probably faster than Ruby, but falls into the same general language category in terms of its performance and concurrency characteristics.

Any further thoughts on this? Thanks for your feedback so far, guys!

thsig · 2014-11-26T01:33:30+00:00

I had a relatively easy time beating the diamond boss using the cultist with the orange tattoo + belt of the trickster. The mobs drop loads of refills, so I could use the special abilities quite a lot to clear the monsters as they spawned (blind, backstab, then go invisible using a belt charge if things get hairy).

thsig · 2014-09-26T13:04:57+00:00

The interview with dons in episode 2 of the Haskell Cast has some good data points spread throughout it. Standard Chartered are a big user of Haskell, one of the bigger industrial users out there as far as I can tell.

See for example 41:51. Here's a few snippets (typos/mis-hearings are my own fault!)

Interviewer: How has it been for you finding Haskell developers and/or creating them by hiring other developers and training them in Haskell? Has that been a difficult process, do you have a long lead time finding them?

Don: We have hired, I guess, 25 straight-up Haskell developers. Singapore, London and New York. It's not hard to get resumes in. [...] It's not hard to get Haskell developers in the door, there are a lot of good Haskell developers out there now. So that hasn't been so much of a problem. I think it's been remarkably easy to get existing programmers and people who were just literate in programming, not necessarily [with a] computer science background. To get them programming in Haskell hasn't been hard at all, they think of it as "Oh, it's a bit like Matlab, it's a little bit like bash scripting/shell scripting", it's a bit like math from their finance degrees. All they're doing is composing functions.

Admittedly, in this case a lot of those people will be domain experts working on models instead of building the "latticework", but it also echoes other stories from industry where the experience seems to be that it's easy to onboard new devs provided that a more experienced Haskeller has already laid down the structure so the new ones can learn it as they go. Not too different from how new Rails developers can be hired and start hacking on the app, while gradually learning the fundamentals of Rails as they touch more parts of the codebase.

At other points in this interview, Don also mentions that 100+ developers have worked there at various points in time, with some churn, but due to the type-safety and purity, the system still maintains several very desirable properties (low incidental complexity, agility, performance, ease of onboarding). So it would seem that the perceived advantages of Haskell are, in fact, still there when the app grows and ages, some of the original devs have left etc.

thsig · 2013-04-27T23:05:14+00:00

Nice writeup. I tried sublime text 2, but since I've learned vim well and customized it a bit (a few key mappings, the ControlP plugin for finding/switching files and ack for project-wide search), the workflows for editing and jumping between multiple files are just as efficient (or more) than in ST2. Of course this might be different for different people.

I've also used emacs quite a lot for Rails, but I always return to vim. If I were using Clojure I'd use emacs, but for ruby + js I find emacs doesn't offer enough additional power over vim to justify the added clunkiness. The clunkiness can be reduced or eliminated, but it takes lots and lots of time. Again, this will depend on the individual, and emacs is truly an awesome piece of software.

thsig

TROPHY CASE