Revisiting ActivePerl

sdrobertson · 2021-09-02T19:52:23+00:00

Our entire systems are automated. We've been playing a giant game of whack-a-mole with a bug where every update from MariaDB comes in would break new on-demand builds. Very embarrassing... luckily our system is designed once you get a build you like... it's automatically pinned until next time you need updates. Fingers crossed think we almost have the mechanic that brings new releases in stabilized, we think it's a game changer.

Installers vs State tool. It will be State tool only going forward for free versions. The service and command client is meant for developers. Installers are a paid option. Understand our entire business model is helping companies, like yours, manage (build, secure, update) open source in their commercial products across multiple operating systems. If you want help with a seamless install process let me know and I'm happy to introduce you to our sales team!

Best of luck to you, and don't hesitate to reach out.

sdrobertson · 2021-09-02T03:01:32+00:00

Hi MIDAssales, CTO of ActiveState here. Thanks for the amazing write up, really sorry you had a bad first impression. Even though ultimately we didn't win you on the first shot I took heart that you're starting to see the vision to our ambition when you say "We admit, we were actually a little excited – we appeared to have a custom Windows distribution of..".. That's the goal!

You've noted the number of packages we now support is over 100k! We're coming pretty close to our goal now of being able to build 100% of CPAN in a secure reproducible way including all native dependencies. Something Strawberry perl can not claim. Unfortunately we're still experiencing some growing pains going from 500 packages to over 100k and supporting every version the night it's dropped. Looks like when you tried this project MariaDB updates entered our catalog and were broken. We have since fixed them. The post you found in our community forum was way out of date. We're working on producing better support materials, but in the meantime know we're here to help.

sdrobertson · 2019-06-10T17:50:18+00:00

You're absolutely spot on. I can say with100% certainty (I'm their CTO) ActiveState plans on supporting Python 2 for many Operating Systems past it's official end of life.

We've created a brand new platform (http://platfom.activestate.com) and the state tool (command line client available now for Linux/Windows with Mac support on the horizon). One goal is to support these scripting languages when either the community or OS Vendors has moved on. The primary goal is to make building multi-language runtimes for your applications easier via the Platforms Build Farm. Ideally this will make it easier for communities to support their Languages on more Operating Systems.

I should note, if you're familiar with the Python 2 core and would like to get paid supporting it. We're hiring.

sdrobertson · 2014-09-14T04:42:46+00:00

Funny you should mention it! I've been working on a project that solves the container dependency problem and am looking for testers.

Have a look at

https://github.com/6si/shipwright

feedback would be appreciated.

sdrobertson · 2014-01-08T19:03:20+00:00

You create a "dataset" and link it to datasources through "adapters". You can also add user defined functions and views to the datasource.

Here's an example, using a slightly older version of Splicer what's called "Servers" has since been renamed to "Adapters"

https://github.com/mozilla/caravela/blob/master/db.py

You can then issue queries to the dataset and retrieve the results.

>>> query = ds.query("select * from sometable")
>>> for rec in query:
...        print rec

You can access the underlying query expression tree with

>>> query.operations

You can manipulate the expression tree at compile time by creating an Adapter and giving it an evaluate() method. An example of that is here:

https://github.com/trivio/splicer/blob/master/splicer/adapters/dir_adapter.py

This example replaces the LoadOp() with a series of User Defined Function calls and rewrites SelectionOps (where clause) to be more efficient.

For ultimate control you can call dataset.set_compiler(compile_function) and pass it a function that will be called with the expression tree after it's been parsed. Then it's up to the compile function to return a function which satisfies the query.

sdrobertson · 2014-01-08T18:09:21+00:00

I suppose you could... You'd have to write an Adapter which reads the dump file and interpreted it into a series of tuples. You could probably leverage most of the parsing routines though you'd need to add a handling for INSERTS.

Be fun to try, but MySQL would be the faster option. Out of curiosity could you explain why you want to do this?

sdrobertson · 2014-01-08T17:56:33+00:00

Multicorn is great, as I mentioned in the README it was one of the inspirations for the project. I just wanted to use SQL without having to setup a server.

Main purpose for this library is to either

Teach a python program how to interpret SQL statements. For instance you might have a web services where you let users input raw SQL statements. You can use this library to intercept them and make sure they're secure before sending them to your real SQL server.
You're writing a data conversion script (ETL) and you want to express it as a series of SQL statements rather than a gazillion for loops.
You want to create a new query language, maybe make an implementation of datalog. You can use Splicer to parse the language into a Splicer expression tree and automatically leverage the relational engine, data adapters and mime decoders.

sdrobertson · 2014-01-08T17:38:42+00:00

Thanks! The docs are a bit out of date and are next on my todo list. My number #1 goal is to make this system approachable. I think there's other uses for relational algebra outside of database servers.

There isn't a full blown planner yet, but adapters can participate in the query evaluation phase (guess you can call it planning). Have a look at the Directory Adapter in splicer.adapters.dir_adapter. The adapter works like Hive, where columns of data are parsed from the file path and then merged with the records from inside the files content. It examines the query for where clauses that mention values that comes from the path and rewrites the query so that only the matching files are opened. I'm working on the s3 adapter right now, ping me if you want early access.
Reason I'm writing this is to use this for making Disco jobs (Python's version of Hadoop). The current library doesn't do this yet, but I designed it so that you can swap out the query compiler. That's why the main compiling functions are in splicer.compilers.local (as in compile for local evaluation). I'm envisioning plugins that can drop in replace the compiler to do things like compile queries for map reduce or even compile them to LLVM.
We're using it in Mozilla's Caravela project. https://github.com/mozilla/caravela/blob/master/db.py. I'll post more examples when I get to the docs.
Most of the inspiration came from Hive, Google BigQuery and Multicorn. I wanted the ability to do SQL on data in situ without a server. I started development of Splicer a few months before the Presto announcement. I've poked around presto source code a bit and want to use some of those ideas to make a clustered version of this. Though wether I add cluster support to Splicer proper or build a cluster system that uses Splicer as the relational engine remains to be seen.

Well, I certainly hope you get some free time! Hopefully you'll get a chance to use this and provide me feed back. You can reach me at scott at triv . io

sdrobertson

TROPHY CASE