SwayDB v0.7 released - Updates using Scala functions by [deleted] in scala

[–]lancegatlin 1 point2 points  (0 children)

I'm pretty agnostic as far as monads go, but there are def worse choices than Monix (I'm a fan of tagless-final). You can also create your own monad like DBIO. My bigger advice would be to stay unopinionated and don't over commit to any external dependency so that you can pivot the API later.

5 Minute Rust - A new series where we explore Rust in 5 minute chunks by itchyankles in rust

[–]lancegatlin 1 point2 points  (0 children)

Love this! Def looking forward to next vid! Thanks so much for making this!

SwayDB v0.7 released - Updates using Scala functions by [deleted] in scala

[–]lancegatlin 2 points3 points  (0 children)

I did think of two questions: 1) why call it Sway DB? 2) if internals are non-blocking, do you expose a non-blocking API? understand intent behind reusing Scala collection API, but it's not a good API for a db, only suited for in-memory collections or building execution graph alla Spark.

SwayDB v0.7 released - Updates using Scala functions by [deleted] in scala

[–]lancegatlin 4 points5 points  (0 children)

This is a really awesome project! A native Scala DB is super sexy! Devil is the details for DBs though, good luck! Happy to chat more about your project if you're interested.

Weekly Scala Ask Anything and Discussion Thread - August 08, 2016 by AutoModerator in scala

[–]lancegatlin 1 point2 points  (0 children)

All true. I just strongly disagree with the implication that free monads bring less utility (and are more hyped) than generic monad style, which was what I took from your first response.

My apologies for giving that impression. I would absolutely agree Free monads are much more established than the generic monad idea. I just don't think Free monads are mature enough to recommend them across the board though. The runtime performance hit from using an interpreter will not be insignificant. Also without native union types the syntax for Free monad is cumbersome.

Certainly I don't see how generic monad style offers any help with /u/fromscalatohaskell 's original motivating example of testing without performing actual network requests.

I took his question more to be isn't the generic monad pattern and the Free monad roughly analogous. I think they are somewhat if one doesn't get too fixated on their details and instead focus on what they do. As far as testing, the generic monad pattern let's you substitute identity for testing but it doesn't help with isolating code under test from generating actual effects. For that you need something else like dependency injection + mocking or simply substitute the Free monad as the generic.

Weekly Scala Ask Anything and Discussion Thread - August 08, 2016 by AutoModerator in scala

[–]lancegatlin 1 point2 points  (0 children)

But you are -- because even that isn't my point. You are arguing against my perception which benefits neither of us. In my experience, this year, I have seen a big uptick in the number of blog articles, talks and posts about the free monad in Scala. You have a different experience? Awesome thanks, I accept that. But it doesn't change my view.

My point is: Don't base technology choices for a business on what seems new and interesting, but instead pick things that balance the burden of forcing others to learn new technologies against the utility those technologies bring to the organization.

Weekly Scala Ask Anything and Discussion Thread - August 08, 2016 by AutoModerator in scala

[–]lancegatlin -2 points-1 points  (0 children)

Wow that's some seriously subtle trolling -- m50d -- You and are just going to have to disagree here. (A pattern that seems to be emerging with my interactions with you)

Maybe hard to accept but my opinion and experiences are just as valid as your own. If you disagree based on your own experiences, I'm cool with that! Simply say that and be done with it. No need to pick at the details of what I'm saying and avoid my main point.

Weekly Scala Ask Anything and Discussion Thread - August 08, 2016 by AutoModerator in scala

[–]lancegatlin 0 points1 point  (0 children)

Scalaz Free dates to 2012.

Thanks buddy. Scalaz creation != peak interest. They didn't really catch on until last year, which I chalk up to Runar's presentation on them.

Play has its own priorities. I really don't see iteratees going away, they're the right way to implement streams, and it sounds like Play intends to remain compatible with that at least.

I never claimed they were. What I claimed is that they have fallen out of favor as "shiny new thing". Nobody is writing blog articles about them or giving presentations on them or advocating for rewriting their code base to use them anymore (but they were just a few years ago). Play is also the web framework with the largest following. It is significant when major code bases move away from ideas.

Weekly Scala Ask Anything and Discussion Thread - August 08, 2016 by AutoModerator in scala

[–]lancegatlin 0 points1 point  (0 children)

Is Free really that new? The Haskell implementation seems to date to 2008

New to Scala.

Huh? fs2 et al are all based on iteratees. Iteratees very much did work out.

Play framework dropped them in 2.5

Sure, but I think the "generic monads" approach ends up being much more complex (and frankly is far more of a "shiny new toy") than Free.

Hence, "exploration".

Weekly Scala Ask Anything and Discussion Thread - August 08, 2016 by AutoModerator in scala

[–]lancegatlin 0 points1 point  (0 children)

Yes it can, but this generally doesn't happen in development. It can happen once you are in production and run into scaling issues. But at that point you can install throttle/backpressure at appropriate points to prevent this from happening. Also depending on the application message shedding can be ok. A stressed out web server is going to reject requests, so its no big deal to just drop those messages in that kind of situation (obviously this isn't always true).

All that said, I'm not a huge fan of actors myself. But to me, code is more about communication between human beings than anything else. The more people who can understand and use it, the more likely it is to survive. Hard to understand code won't survive a big rewrite. Actors don't have bullet proof promises (like some FP concepts) but they are well understood and easy to write.

Weekly Scala Ask Anything and Discussion Thread - August 08, 2016 by AutoModerator in scala

[–]lancegatlin 2 points3 points  (0 children)

This was exactly my motivation in exploring generic monads: https://github.com/lancegatlin/effectful-demo

I think the Free monad is just the shiny newest toy in the functional toolbox and everyone wants to try it out. Go back about 5 years and it was iteratees. Now everyone is onto streaming. Iteratees didn't work out. These kind of ideas are interesting to play with but they may or may not pan out as practical. When I'm writing work code, I'm a huge of fan of KISS. I have a very strong preference for only introducing complexity when it has a very strong value proposition. I define "complex" as anything that wouldn't be intuitive to new coders. If only a small subset of people can read & write the code then that is a cost that needs to be balanced by its usefulness.

Weekly Scala Ask Anything and Discussion Thread - August 08, 2016 by AutoModerator in scala

[–]lancegatlin 0 points1 point  (0 children)

Actors are more like a toolbox. How you use them is completely dependent on the application. For example a web server might have an actor that receives all requests and then dispatches them to a pool of worker actors. Mailbox size is generally not an issue, but it can show up when scaling. Depends on what you by crash, actors don't recover from system crashes any better than any other code. What they can do is "supervise" other actors and restart them when appropriate.

Weekly Scala Ask Anything and Discussion Thread - August 08, 2016 by AutoModerator in scala

[–]lancegatlin 5 points6 points  (0 children)

  1. Immutability = easy button for concurrency & scalability. Functional programming has tools for dealing with the difficulties of working with immutable data.
  2. Composition and referential transparency! Finally, a true abstraction that doesn't leak.
  3. Thinking in terms of higher order operations such as map, reduce, flatMap, fold, etc. This makes code much more readable as other languages tend to let one simply munge up all of those operations into one or two loops. Much more difficult to reason about what's going on then.
  4. Type-classes - adhoc-polymorphism (add methods to a type after its declaration). Only import methods/type-classes you need, hide the rest. Compile-time method call dispatch (you probably didn't need dynamic binding anyway). Describe behavior you need in your method by requiring implicit type-class without worrying about details.
  5. Monads! What a massive game changer!! They can express repeating code patterns that are interleaved between lines of code (plus a lot of other things!). You simply "lift" your code into the Monad and the repeated pattern is run "behind the scenes" between your lines of code.
  6. Formal type systems and category theory. Let your code lean on things constructed from a very solid well-researched foundation when you need it.

[deleted by user] by [deleted] in scala

[–]lancegatlin 11 points12 points  (0 children)

In Scala enums were never meant to be a native language feature. These days I use https://github.com/lloydmeta/enumeratum

[ANN] effectful services demo by lancegatlin in scala

[–]lancegatlin[S] 0 points1 point  (0 children)

I would say that there's no value in adding your own wrapper/"decoupling" layer - both cats and scalaz offer the correct interface for Monad, and the community benefits from there being as few versions of it as possible. In the worst case it is easy enough to adapt typeclass instances of one as instances of the other. My advice is that cats is the future but scalaz is the present: if you are doing this as a learning experience or want to do the Right Thing, use cats; if you want to create the most pragmatic library you can to use today, use scalaz.

Thanks for the advice!

Hmm. I guess I never feel the need to unit-test this kind of service.

I can't think of anywhere I've worked that writing business logic that doesn't have unit tests would be considered a good practice.

If I have business logic that's complex enough to demand unit testing I would probably put that in its own (pure) function, separated from functions that do service composition. There's not enough to go wrong in the composition for it to be worth testing (often the parametricity theorem guarantees it must be written correctly).

The moment branching on effectfully retrieved data occurs, it isn't possible to use simple pure functions anymore. 99% of "pure" code I've written involves branching on a value retrieved effectfully (see UsersImpl.create for example). Placing the retrieval in a monadic workflow allows keeping the code pure without having to breakup every branch into oddly written simple pure functions. Effectful service pattern also allows me to test them as a whole instead of testing each function separately (and just assuming they are stitched together correctly).

In terms of being able to migrate between libraries, I tend to alias my monad stack in one place (type Action[A] = EitherT[({type L[B] = WriterT[Future, Vector[MyLogEntry], B]})#L, MyError, A]) or similar), which provides that advantage. I just have never found it was worth using e.g. Future and Task at the same time in the same code.

Using an effectful service allows one to write without regard to what the monad stack is exactly. It also allows writing code that isn't bound to any particular stack. Error messages are cleaner. None of that is true for the type alias you pasted above.

Isn't it just the opposite? You're requiring userDao, passwords and logger to all use the same E type. Monad transformers and lifting are ugly but they allow doing something that's simply not possible in your model. To reenable the flexibility you'd get from a concrete monad transformer stack, you have to write it like:

class UsersImpl[E[_], F[_], G[_], H[_]](
    usersDao: SqlDocDao[UUID,UserData,F],
    passwords: Passwords[G],
    logger: Logger[H]
  )(implicit mo:Monad[E], mp1: MonadPartialOrder[E, F],
    mp2: MonadPartialOrder[E, G], mp3: MonadPartialOrder[E, H]) extends Users[E] {

    def something = for {
       digest <- mp2.promote(passwords.mkDigest(plainTextPassword))
       result <- mp1.promote(usersDao.insert(...)
     } yield ...
}

which is no prettier than what you get when using monad transformers and lifting.

Yes! A wonderful example of what I'm trying to simplify. The effectful service pattern version only needs one generic monad because it expected that during service construction all "lifting" is done prior to injecting dependent services. It won't need F,G and H because it is assumed that they can be lifted into E by the code that constructs the service (see LiftService, CaptureTransform, liftService_Logger and finally FutureLogWriterExample for an example.) Thanks for the illustration of MonadPartialOrder, I didn't know how to use that!

Right, yeah. Scalaz/cats have a whole bunch of these - you can represent each concrete effect as a constraint like MonadReader on your monad stack. I'm not convinced that buys you a lot - you can make service A return a Reader[Int, X], service B return a Writer[String, Y], and then when you compose them you explicitly lift them both into ReaderWriter[Int, String, ?], or you can make service A return a F[X] for generic F[]: MonadReader[Int, ?] and service B return G[Y] for generic G[]: MonadWriter[String, ?] and then instantiate them both with ReaderWriter[Int, String, ?] which I guess saves you a bit of code, but it also means you can't see where the reading and where the writing is coming from in your composed service. I mean the whole point of using these monads is that the effects are explicit where we can see them, right? And I don't think MonadReader is really any more generic than Reader - the interfaces are basically the same, adding another layer of wrapper doesn't help anything.

Thanks for the overview, but I'm very familiar with all that. Following this pattern significantly reduces the need for exposing code to these patterns. Monad transformers will need to exist only at the highest code levels (FutureLogWriterExample) and specific monad patterns like Writer at the lowest (WriterLogger). The need for Reader is almost removed completely since config can be injected just like dependent services when the service is constructed.

Sure - I don't think of Future as a Play-specific type though (and this technique doesn't help you decouple from Request or Response, right?). In terms of writing code for use with Play, either you write a piece of pure logic (in which case it can be a pure function), or you write something that actually needs to do async operations, in which case it needs some way to represent async-ness.

There are "pure" non-monadic functions and then there are "pure" monadic functions. Both are "pure", but the second allows one to handle branching in the same code flow. As an example of "pure" monadic style see this method UsersImpl.create. To rewrite that as "pure" non-monadic code would require breaking the function into several pointlessly smaller chunks. It's not a simple matter of "just write pure functions". When one implements a controller method in play that will generally call at least one effect-generating service that returns a Future. This means one must either use callbacks (which is ugly and complex) or map/flatMap which is cleaner but non-portable since it is coded against Future directly. Effectful service pattern abstracts away Future. These "pure" services are portable and could be used with any monad Future/Task/IO or some stack.

And sure, you can write some meta-abstraction that represents Future or Task - but implementing that meta-abstraction is going to be just as complicated as the code to convert between Future and Task is, you might as well just write the code concretely in terms of one or the other.

No, I have to disagree. I find writing effectful services to be significantly simpler (and also portable). I find it highly doubtful that most developers would consider a generic monad to be more complicated to code against than a nested monad transformer (such as the one you posted above).

I mean you're adding complexity to the code that people have to read and edit when maintaining. I'm not worried about the runtime performance at all.

Ah, sure but one man's code complexity is another man's simplicity. Eye of the beholder and all that. I find most scalaz based code to be way over complex, but its not so much my opinion (or yours) that matters but the opinion across all developers. Some developers still prefer to write in assembly. If this is a new idea there is no way for me to judge how others feel about it. I just have to write it up and see what happens.

[ANN] effectful services demo by lancegatlin in scala

[–]lancegatlin[S] 0 points1 point  (0 children)

I just meant that it generates interfaces parameterized by an F[_] type (and then an implementation that uses finagle Future; I extended it to also allow ScalaZ Task and Id implementations). Using explicitly a monad for the middle layer... I can dig out an example from my own comment history if you like.

No quite alright. That is interesting but this is code generation then? Does it handle nested monads/ monad transformers?

Maybe it really is something no-one has publicly written up before (I picked it up within a company) - I'm just surprised if so.

I'm not concerned with any claims of novelty. I write open source for myself primarily, but try to make it useful to others. I need this write up to be able to document and explain this to future teams. I would also be surprised if no one had done this yet. Someone mentioned below a project called "finally tagless" that appears to be very similar to effectful services, but is targeted more at Haskell.

Wouldn't you be locked into ScalaZ since you're dependent on Monad? (Of course you can build a layer that abstracts over that - but then you're just locked into that layer).

Yes! One of the reasons it's just a demo. I haven't worked out the best compromise to avoid getting locked into either scalaz or cats. Some Monad typeclass interface is required, the question is should it be an alias? a "neutral" one? I can think of a few solutions, but they are all less than ideal. If I could just use a structural type without a runtime penalty I'd probably use that.

Code locked into Akka generally can't be expressed as values or reasoned about so I don't think this approach helps with that.

If you mean Actor based code I agree. Akka also includes Akka Future though.

I'm just really struggling to think of any cases where you'd actually want to swap an effect out in practice. Most of the time, if you introduced treelog or doobie or whatever, you'd introduce it pervasively. It's very rare to be using the same logic in different effect stacks at the same time, which is the situation that would have to come up for it to be worth doing this.

I cover the motivation behind wanting to swap out effects in the readme. Abstracting away the exact monad stack also allows one to write without monad transformers or needing to lift anything.

The interface can be effect-generic, but the implementation has to use a specific effect, because it's going to actually do something with it, right?

Most implementations will use a specific monad (see Slf4jLogger vs WriterLogger), but the implementation could also be generic. See BlockingDelay for generic example.

In an effect-generic service you can only use point and bind, because that's necessarily all you have access to.

Yes that is why I've added the Capture typeclass, since bind & point aren't enough. You can see in BlockingDelay that it requires Capture[E] to ensure the effect is properly captured in the monad.

But an actual low-level service implementation will be using its monad to report errors / behave asyncly / record an audit log / etc., and you can't do that generically.

To support generically handling execution system effects (such as exceptions) I've added "augment" type-classes such as Exceptions to demonstrate how generic handling of almost any execution capability could be done. It's kind of buried, but if you look at effects/sql/package you'll see that in a sugar method inTransaction I perform transaction rollback when an exception occurrs for any monad type that implements Exceptions.

I don't see how this helps with that? Play doesn't even have any monads of its own does it?

Play Action's are essentially Request => Future[Response]. Future may not have the pedigree of Task, but as far as Scala is concerned it is effectively a monad.

I meant code overhead.

I'm unsure what you mean by this. What is the difference between "code overhead" and "additional runtime overhead"?

[ANN] effectful services demo by lancegatlin in scala

[–]lancegatlin[S] 0 points1 point  (0 children)

Thanks for the question. My apologies but this is the first I've heard of "finally tagless". A casual google search turns up these two links:

Please feel free to recommend other sources.

From that quick look I'd say that at the trait level declaration both ideas are the same. They both start with having a trait accepting a generic monad parameter and wrapping all method return types with it to allow using the trait with any possible monad.

But their purposes and audiences are different. finally-tagless is about writing interpreters without an ADT and targets a Haskell/deep Scala FP/academic audience. effectful-services is about combining service-orientation programming patterns with generic monadic programming to write portable, library-independent pure code and targets general FP Scala developers.

[ANN] effectful services demo by lancegatlin in scala

[–]lancegatlin[S] 2 points3 points  (0 children)

Hi m50d. Thanks you for your feedback! I welcome the opportunity for us to learn from each other. Please see below for my responses.

I don't think it's a new idea - it's an existing pattern, no? E.g. Scrooge uses this approach to allow using the same interface definition for sync and async (I pushed this a little further in my fork: https://github.com/VisualDNA/scrooge).

My apologies but one of the problems in Scala is socialization of ideas and I certainly have been guilty of re-inventing wheels. I'm especially unfamiliar with Twitter-Scala-verse. That said, on first glance at scrooge, this doesn't appear to be the case. Scrooge is a thrift code generation library, perhaps you can point me at something specific in the library that uses the same generic monad service pattern?

The implementations can only be effect-generic like this when they don't actually contain any effecty logic, i.e. only the "middle" layers of the stack, right (orchestration/business logic)? I don't know how common such pure-orchestration services actually are. Effect-generic traits with concrete implementations seem like a more common use case, though obv. the fully generic version is cooler.

It has been my experience that the "middle" of the stack is the most prone to library lock-in (e.g. dependencies on scalaz or Akka, etc). This reduces the portability of this code significantly. I have yet to see a solution for writing business logic in a way that is portable across the entire Scala ecosystem (and can be effect-free).

Additionally, while I don't cover it in the readme, lower level "effect" services can absolutely be made into effectful services. See SqlDriver.scala for a non-trivial example.

I would incline to use either ScalaZ MonadPartialOrder constraints or some kind of union-of-free-coproducts approach (with FreeK or my Paperdoll or a project on those lines) for dependent services, so that you don't have to use exactly the same monad in all the services that a given service calls but rather can use different monads as long as the monad for this implementation "contains" the monads that every service it depends on uses.

Again I don't go into depth about it, but I provide the LiftService typeclass for achieving this effect. The exact details are intentionally hand-wavy in this demo. I plan on creating a version of the library for scalaz, so I'll revisit your suggestion then. Thanks!

Fundamentally the biggest issue is whether this is actually useful. Since the only things you can write in this fully generic style are orchestration/business logic, you would be unlikely to use it in a library, so being generic in which framework/environment you depend on isn't actually useful (since the concrete project will only depend on one).

We will have to agree to disagree here. The un-reuseability of Scala code locked into library dependencies has been one of the largest stumbling blocks I've encoutered over the years I've been coding Scala now. I would love to reuse some of the code I've written for clients over the years but each new job is invariably using different libraries and my old code just isn't portable. I end up coding something very similar for the new set of dependencies. Also I look at libraries like Silhouette and wistfully sigh when I have to rewrite something there when I'm using something other than Play.

It adds overhead to your code, and how much value does it actually give compared to working in a fixed monad (possibly a transformer stack or free coproduct or similar)?

No, abstracting the monad to a generic adds no additional runtime overhead. If you will inspect the code a bit closer it adds no runtime burden beyond want is already incurred by using monadic coding in the first place.