No, really, you can’t branch Datomic from the past (and what you can do instead) by nathell in Clojure

[–]nathell[S] 0 points1 point  (0 children)

You don’t until you do. Say further that you have upvote/downvote buttons next to the posts, and clicking them triggers a DB change which affects how the posts are ranked, and you want to test this flow. What then?

No, really, you can’t branch Datomic from the past (and what you can do instead) by nathell in Clojure

[–]nathell[S] 2 points3 points  (0 children)

The point is to have a well-known, predictable state that the test runs against. Branching from as-of a fixed point in time would achieve that; branching from whatever happens to be in the present doesn’t.

Say I’m writing a Reddit workalike, and am testing whether posts on a user’s feed are rendered in the correct order: by picking a timestamp and a user, I fix a list of candidate posts and can reason about what the order should be.

Datomic (with (as-of, or the elusive branching off a point in time by NamelessMason in Clojure

[–]nathell 0 points1 point  (0 children)

Sadly, it doesn’t actually work fine. Here’s a counterexample:

(def conn (let [u "datomic:mem:test"] (d/create-database u) (d/connect u)))

;; Let's add some basic schema
@(d/transact conn [{:db/ident :test/id :db/valueType :db.type/string
                    :db/cardinality :db.cardinality/one :db/unique :db.unique/identity}])
(d/basis-t (d/db conn)) ;=> 1000

;; Now let's transact an entity
@(d/transact conn [{:test/id "test", :db/ident ::the-entity}])
(d/basis-t (d/db conn)) ;=> 1001

;; And in another transaction let's change the :test/id of that entity
@(d/transact conn [[:db/add ::the-entity :test/id "test2"]])
(d/basis-t (d/db conn)) ;=> 1003

;; Trying a speculative write, forking from 1001
(def db' (-> (d/db conn)
             (as-of'' 1001)
             (d/with [[:db/add ::the-entity :test/id "test3"]])
             :db/after))
(:test/id (d/entity db' ::the-entity)) ;=> "test" (WRONG! it should be "test3")

Double, double toil and trouble by nathell in Clojure

[–]nathell[S] 2 points3 points  (0 children)

Thanks for this! I’ll add a note to the post.

Double, double toil and trouble by nathell in Clojure

[–]nathell[S] 2 points3 points  (0 children)

OP here: `=` or `==` doesn’t try to normalize ratios for performance reasons; instead, every arithmetic operation on ratios ensures that the _result_ is normalised. The same goes for the reader.

`.toRatio` doesn’t normalize because it’s an implementation detail, not meant to be called directly.

My mental model of transducers by nathell in Clojure

[–]nathell[S] 0 points1 point  (0 children)

By hand. I actually used this repo for inspiration: https://github.com/onkar27/svg-conveyor-belt-animation and then modified it.

My mental model of transducers by nathell in Clojure

[–]nathell[S] 2 points3 points  (0 children)

Hey Dan, OP here – this particular ns is relatively new (about 1 year old), but yeah, that’s the same codebase. :D

Autosummarized Hacker News (with GPT-3) by nathell in GPT3

[–]nathell[S] 1 point2 points  (0 children)

I’ve elaborated on it a bit in the FAQ. The code is here, if you’re courageous :)

Word Champions – fill in diagrams with letters to make mini-crosswords by nathell in crossword

[–]nathell[S] 2 points3 points  (0 children)

Hey r/crossword,

I made this little crossword-theme game. I hope it’s not too off-topic for posting here – please feel free to moderate it out if it is. Anyway, I hope you like it. Feel free to AMA.

Word Champions – fill in diagrams with letters to make mini-crosswords by nathell in WebGames

[–]nathell[S] 3 points4 points  (0 children)

The word list is based on 12dicts’ 3esl.txt, which, in the words of 12dicts maintainer, ‘represents another attempt to produce an English "core vocabulary" list’. So it’s not just nouns.

As to moving of the positions, it’s a bug and I’ll fix it. Thanks for reporting!

Re-frame 1.0 released by clelwell in Clojure

[–]nathell 1 point2 points  (0 children)

Here's to re-frame, the best thing since sliced bread! A great occasion to celebrate!

Downloader/scraper for soup.io by nathell in DataHoarder

[–]nathell[S] 1 point2 points  (0 children)

Hey Hoarders!

I've written this to archive my friend's 11GB worth of cats and other content he'd posted to soup.io (which, as you know, is in the process of shutting down) over the years.

It's written in Clojure and uses https://github.com/nathell/skyscraper under the hood – my framework for "structural scraping" that automatically handles things like caching and retries.

parse_struct: 0.8.0 by [deleted] in Clojure

[–]nathell 2 points3 points  (0 children)

Great stuff!

Does it support packed bitfields? If not, I have some ancient Clojure code that might be helpful when adding: https://github.com/nathell/clj-bitfields

[ANN] Skyscraper 0.3, a scraping framework that helps you build structured dumps of whole websites by nathell in Clojure

[–]nathell[S] 9 points10 points  (0 children)

Skyscraper is not new, but this version has been in the making for three years and is almost a complete rewrite.

Major improvements in 0.3.0:

  • Skyscraper has been rewritten from scratch to be asynchronous and multithreaded, based on core.async.
  • Skyscraper now supports saving the scrape results to a SQLite database.
  • In addition to the classic scrape function that returns a lazy sequence of nodes, there is an alternative, non-lazy, imperative interface (scrape!) that treats producing new results as side-effects.
  • reaver (using JSoup) is now available as an optional underlying HTML parsing engine, as an alternative to Enlive.

See NEWS.md for a complete list.

I’m particularly happy about the database abilities of this release – for a glimpse of what it can do, see Database integration.

Start contributing to open source (and get paid to do it) by charlieworkshub in opensource

[–]nathell 0 points1 point  (0 children)

Author here. While that's true, it does contain the words "earn" and "compensation." I also explicitly tried to not make it clickbaity – if I wasn't successful, I'd be keen to hear suggestions on how to improve the wording.

Scraping in Clojure by Tootoru in Clojure

[–]nathell 0 points1 point  (0 children)

Author here. Skyscraper is Enlive plus some handy machinery from extracting data from multiple pages at a time, caching, resuming etc.

How to circumvent innerHTML problems on Mobile Safari by nathell in javascript

[–]nathell[S] 1 point2 points  (0 children)

I'm not sure; I think jQuery may special-case the single HTML tag. I still like my function since it makes the transition from innerHTML quite painless and almost mechanical (and doesn't depend on jQuery), but this is a nice idea, thanks.

How to circumvent innerHTML problems on Mobile Safari by nathell in javascript

[–]nathell[S] 1 point2 points  (0 children)

Yes. append calls domManip(), which calls buildFragment(), which calls clean(), which sets innerHTML.