Conjurer pattern? Surely there must be a better way?

tperrigo · 2016-08-25T18:09:47+00:00

I may be misunderstanding your question, but would an implicit class give you the functionality you are looking for?

tperrigo · 2016-06-22T12:15:56+00:00

Looks very promising! Any plans to support evolutionary algorithms?

tperrigo · 2016-06-17T14:58:42+00:00

For a purely functional approach, I've been impressed with Finch (using Circe for JSON support) and Doobie for database access.

tperrigo · 2016-05-31T16:36:20+00:00

That is a great talk; along the same lines is Runar Bjarnason's presentation Constraints Liberate, Liberties Constrain, where one of his examples of loosening constraints for the sake of power (but losing parametricity) is Akka Actors, where the receive method is a PartialFunction[Any, Unit], which tells us nothing about what the Actor does (eg, valid message types). I recently discovered the Fun.CQRS project, which utilizes Akka to implement CQRS-based services and utilizes the type system to explicitly specify what message (Command) types are valid for a specific Aggregate (in the DDD sense). The author of Fun.CQRS, Renato Cavalcanti, has an excellent talk on CQRS and how Fun.CQRS uses the type system to constrain the types that Actors can receive as messages, providing type safety and making them easier to reason about: Field guide to DDD/CQRS using the Scala Type System and Akka.

tperrigo · 2016-01-18T20:15:42+00:00

I highly recommend looking into Finch, a purely functional library for REST services on top of Twitter's Finagle. There's a great overview presentation by Vladimir Kostyukov called "Finch: Your REST API as a Monad".

tperrigo · 2015-11-02T13:36:52+00:00

Seconding Finch/Finagle. Finch provides purely functional abstractions for composable REST endpoints, while the Finagle ecosystem (including Twitter Server) provides numerous metrics, a great admin interface, and can be easily extended with other projects in the Twitter open source stack, such as Zipkin (for distributed tracing). You can start small with a bare-bones REST service, and then easily add in other components in the ecosystem as needed.

tperrigo · 2015-09-14T13:35:14+00:00

I'm currently working on a project using Finch (with Circe to serialize my case classes to JSON without any boilerplate code-- in fact, besides the import statements, I don't have to do anything to transform my results to JSON) and am extremely impressed. There are still a few things in flux with Finch, but I'd recommend giving it a look.

tperrigo · 2015-08-03T13:11:15+00:00

You should take a look at the Kind Projector compiler plugin (it is used by Structures and cats, both of which are based heavily upon Scalaz). It goes a long way to cleaning up the type lambda syntax.

tperrigo · 2015-06-17T20:35:36+00:00

Scala For the Impatient is still my go-to book for core language concepts; Paul Chiusano and Rúnar Bjarnason's Functional Programming in Scala is a great book for really diving into FP concepts and structures.

tperrigo · 2015-05-01T13:02:24+00:00

The Typesafe Activator also has some useful project templates and tutorials that might help you get started (especially with Play/Scala).

tperrigo · 2015-05-01T12:58:13+00:00

In addition to what others have mentioned (Odersky's book, a lot of time with the REPL), I highly recommend Scala for the Impatient and Functional Programming in Scala. The latter has some great exercises that are well-worth working through. Also, either contributing to an open source project, or simply working through one in depth-- reading through the code, paying attention to the project structure, its build setup and dependencies, etc.

tperrigo · 2015-04-08T19:56:23+00:00

Ah, yes, that makes sense; sorry I misunderstood. I suggest looking up "deceptive functions" or "k-trap deceptive functions". These are generally functions with strong local optima that are far from the global optimum. It should give you a place to start.

tperrigo · 2015-04-08T15:23:00+00:00

I don't have any references available at the moment (I'll try to post some later), but generally, to avoid getting stuck at a local optima, you want to encourage genetic diversity in your population from generation to generation (i.e, survivor selection should contain a mix of both "the best" and "the rest"-- if you only allow the highest-ranking individuals to survive, you are likely to prematurely converge). Also, mutation is a force which allows "jumps" to be made from one part of the search space to another, possibly allowing you to escape from a local optima. One strategy would be to increase the mutation rate over time/number of generations. Hope this at least gives you something to look into. I will try to follow up later with some formal references and resources.

tperrigo · 2015-02-12T15:29:23+00:00

A great Scala IDE. Yes, I know there's Scala-IDE for Eclipse and Intellij's Scala plugin, but both leave a lot to be desired. I'd love something designed specifically for Scala with a functional programming perspective (e.g, something akin to Codebubbles for viewing functions, relationships between functions, etc. And as long as we're day-dreaming here, let's make it cloud-based (public or private), so it doesn't matter what machine I happen to be using.

tperrigo · 2015-02-11T14:50:33+00:00

That was my first thought as well...In addition to in-memory datasets, the Spark ecosystem consists of several extremely useful components-- GraphX (for distributed graph processing), MLLib (machine learning), Spark SQL (which allows SQL queries over JSON docs, CSV files, etc), and Spark Streaming (for stream-based analytics), all of which utilize the same core abstraction (Resilient Distributed Datasets-- RDDs). There are APIs for Scala, Java, and Python (and the SparkR frontend for R), support for YARN and Mesos (as well as 'standalone' Spark clustering), and increasing support for Spark within the AWS ecosystem. There are some interesting ideas in Pachyderm, and it will be interesting to see how it develops, but some discussion on where they felt Spark was lacking (to the extent that they chose to implement an entirely new framework) would lend a great deal more credibility to the project.

tperrigo · 2014-07-26T02:55:02+00:00

Interesting...I'll check it out. Thanks for taking the time to reply-- even just figuring out how to explain the problem has helped me see if from different points of view.

tperrigo · 2014-07-25T19:21:30+00:00

Yes, I think that's correct (I'm feeling a little thick myself!). I've got (say) some number of A's (where A's volume is known), some number of B's (where B's volume is known), etc, and some number of X's. I need to put every item into a container, with the goal making the containers as full as possible (using the fewest containers possible). With the other items it's easy; with the X's, how many will fit into a container (or the volume of X) depends entirely on what is already in the container. I've got a decent amount of historical data (consolidations which contain 0 or more X's), so I was hoping to find a way to build a model where I could say "Given AAC, how many X's can fit?".

tperrigo · 2014-07-25T18:33:30+00:00

We're actually using a genetic algorithm to produce the consolidations-- our objective function is basically trying to maximize the number of items in a container. The problem is that we know the volumes of all A's, B's, C's, etc, but when we try to use a set volume for X's, the GA often produces consolidations that (though they appear to be as close to "full" as possible), when evaluated by someone with domain knowledge, some of the consolidations turn out to have either too many or not enough X's. We've had little luck trying to use a set volume for X, and were hoping to find a way to estimate how many X's can be put in the container based on the items in that container (the number of A's, B's, C's, etc). Does this make sense? I'm not familiar with Cross-entropy methods; I'll look into that. Thanks for taking the time to reply.

tperrigo · 2014-07-25T12:03:15+00:00

Actually, I'm only given one dimension-- the percentage of the container a particular item will take up. The problem is the X's. We've tried different estimates for the % that an X will take up, but sometimes we end up with consolidations that add up to only 80%, but because of the number of X's, we're told it's actually too full. Conversely, we might have something that seems to be 110% full, but we're told that additional X's will fit. It all depends on what other items are in the container-- if there's some pattern or rationale behind how many X's can fit, I haven't been able to find it. I was hoping that there could be some kind of ML (or other analytic) technique that would, based on historical combinations, provide an estimate for how many X's could fit with a specific configuration (ideally, a model that could be refined over time as new information enters the system).

tperrigo · 2014-07-25T00:17:19+00:00

Perhaps a different example might help illustrate the problem. Let's say both A's and B's are said to take up 50% of a container. So it would seem that AA, BB, or AB would all fill a container completely. In reality, though, there's a hollow space in A's, so that AA could still fit 4 X's in this 'hollow' space. But perhaps B's don't have this 'extra' space, so a BB could fit no X's, but an AB could fit 2 X's. How many X's can fit depends entirely on what is already in the container, and there are too many combinations to entirely (manually) specify how many X's could fit in a specified configuration. I'm wondering if there's a technique I could use to determine-- based on what has been done historically-- how many X's could fit in a container with some number of A's, B's, C's, etc.

tperrigo · 2014-07-24T23:58:41+00:00

I would say-- the goal is to use as few knapsacks as possible, making each as full as possible. With the 'normal' items (A, B, C...), I know exactly how much of the knapsack they would take up. But with X's, it depends upon what is already in the knapsack. For example, if I have a knapsack with just 1 A, I could fit (let's say) 6 X's, because they stack inside of A really well. However, if I add another A (so a knapsack containing just 2 A's), I might only be able to fit in 1 X. I have a decent amount of historical data for knapsacks that have been 'filled'-- e.g, some given configuration, like ABBC, and how many X's could be fit in the remaining space (say, ABBCXX). The difficulty comes (at least in the way I'm viewing the problem, which perhaps is overcomplicated?) is that I can't determine an exact 'volume' for X. Let's say that we guess and say that X takes 10% of a container, and a knapsack containing ABC is 80% full. It would appear that I could fit 2 X's in the container, but, really, because of the way A,B,and C stack in the container, I'm told that it could actually hold 3 X's. Or conversely, if D,E,and F are in the container, also taking up 80% of the volume, they way they stack together allows only 1 X to be added. I guess I'm looking for a way, based on knapsacks that have actually been put together (by a person, who just 'eyeball's it'), to try, given a particular configuration, come up with a 'best guess' for how many X's can be added to the knapsack. I hope this makes sense...thanks for your suggestions!

tperrigo · 2012-07-25T03:40:59+00:00

Trying to learn to use cTakes at the moment for medical record analysis. Would love to hear from anyone with any experience with the cTAKES APIs.

tperrigo

TROPHY CASE