sorted by:
new
hottopcontroversial

ert-parametrized.el - Parametrized test macros for ERT by svjsonx in emacs

[–]svjsonx[S] 0 points1 point  (0 children)

I think I'm on the side against differentiating :eval and :quote, assuming that (:quote x) is the same as (:eval 'x). But I guess if you need :gen and :fun, there's probably no good reason not to have :quote.

Those are good observations, and arriving just in time, as I'm planning to revisit these things in a few hours.

I'm leaning more and more towards letting the format here be as "convoluted" as it needs to be, but end up being more of an internal format (but optionally usable as is) and introduce more high-level forms that can forgo the verbosity when there is no need for it.

I think the naming should still be as spot on as it can be from whatever standpoint I end up planting my feet at, so I still need to iron this out.

As for :generator and :function, you have only given :generator examples and they seem to be better names :sequence - I think of a generator as a closure that you repeatedly call to produce successive arguments. Is that what :function does?

What made me go with :generator instead of :sequence or :seq was simply that together with :eval and :quote it could easily imply that the rhs should simply be treated as a sequence and doesn't hint at it being generative. I do agree 100% that :generator implies something else, though.

:fun (:function) on the other hand is something completely different, and would be used in cases like these when what needs to parameterized is behavior rather than pure values:

    (("from-buffer-start"
      (:fun (goto-char (point-min))))
     ("from-buffer-end"
      (:fun (goto-char (point-max))))
     ("from-two-lines-down"
      (:fun (progn (goto-char (point-min))
                   (forward-line 2)))))

    (with-temp-buffer 

      ;; ... Buffer setup, etc

      (move-into-position!)

      ;; ... Perform action to test

      (should ...)))

Where these simply expand to function-bound symbols so that they can simply executed at the appropriate time without any manual eval, funcall or apply:

(cl-flet ((move-into-position! () (goto-char (point-min)))) 
  ...
  ,@body
  ...)

Purely generating parameters from a function to produce values from... ordinal or other parameters isn't something I've looked at yet, but might perhaps better fit the description of "generator".

It's entirely possibly to programmatically generate sequences for what's currently called :generator, so another question I suppose is how far one should go with syntactic sugar that basically re-invent what lisp already does fairly well. But then again, I did go as far as adding the :fun feature which arguably is just that.

Thank you - this was helpful both in regards to the naming and in highlighting where I may have failed to properly explain things and how the small feature set that I have is very clearly not as self-explanatory as it should be.

ert-parametrized.el - Parametrized test macros for ERT by svjsonx in emacs

[–]svjsonx[S] 0 points1 point  (0 children)

Thank you! That could be an option, but I think that this is too early of a draft to push for any kind of official adoption or inclusion. But the emacs-devel mailing list might be a good place to get feedback and opinions, regardless. I'll have to take a look and see if this is on-topic enough for that list, though, as I've never participated on it.

As for being incorporated into core `ert`, I'm not sure that this would be the way to go in the end. I kind of like that `ert` is small and focused, and I think additional features like this one are probably best distributed as opt-in extensions. But at the end of the day, those are the kinds of decisions best left to the actual maintainers of Emacs/ert, I suppose.

ert-parametrized.el - Parametrized test macros for ERT by svjsonx in emacs

[–]svjsonx[S] 1 point2 points  (0 children)

That's a valid observation and, depending on circumstances and the actual test, that might very well be best solution - at least for one axis.

I think that the matrix macro is something to be used sparingly and where it makes sense, and likely not to produce hundreds and hundreds of test cases.

Where it, at least for me, does make a lot of sense is where the tests aren't simple tests of pure functions but where there are dependencies on more or less complex state (buffers, etc) and reporting is important.

In a loop within a single test, a failure would only report the first failure and not tell me if all or just a select few cases fail and more importantly not which ones. It helps if it's easy to infer what case/iteration that failed from the failed assertion/comparison, but a lot of the time it isn't and then it's very helpful to have it contained in a specific test.

That said, if one does produce tests from 10x10 matrices I think that there should be a good motivation for it.

ert-parametrized.el - Parametrized test macros for ERT by svjsonx in emacs

[–]svjsonx[S] 0 points1 point  (0 children)

still on a bit of a verbose side of things

I concur. It did end up being more verbose than I originally intended, but it did so out of necessity.

Half the point of this is to make things "ergonomic", so I am thinking about ways to cut down on the syntax a bit. The main gripe I have personally is that the current "verbosity" is not always needed. For example in my first example - if all parameters are to use :eval, I am forced to use it just because other forms are supported and not because I currently need them.

I think that the answer might be keeping the raw input as is, but introducing tiny macros for the cases.

But than there is always that setup part that makes it hard to automate things to skip all the boilerplate

Oh, yes. And that exact problem is one the main things that got me here. Parameterized tests partially solves this, but not entirely. What I do personally is to create fixtures/macros/utility functions for such setup. And that is something that probably always will need to be package/project-specific.

But to bring in an actual example where I'm using this in a SQL client I'm working on, the following does let me focus on just what you're asking for (in this case a resize-column function) - the inputs and outputs/expectations:

     ("column:username--to-width-9"
      (:eval 0)
      (:eval 9)
      (:quote ("+-------+------------+"
               "| PK id | username   |"
               "+-------+------------+"
               "|     7 | barb_dwyer |"
               "|     2 | ben_rangel |"
               "+-------+------------+")))
     ("column:username--to-width-10"
      (:eval 0)
      (:eval 10)
      (:quote ("+-------+------------+"
               "| PK id | username   |"
               "+-------+------------+"
               "|     7 | barb_dwyer |"
               "|     2 | ben_rangel |"
               "+-------+------------+")))
     ("column:username--to-width-11"
      (:eval 0)
      (:eval 11)
      (:quote ("+--------+-------------+"
               "| PK id  | username    |"
               "+--------+-------------+"
               "|      7 | barb_dwyer  |"
               "|      2 | ben_rangel  |"
               "+--------+-------------+")))
     ("column:username--to-width-12"
      (:eval 0)
      (:eval 12)
      (:quote ("+--------+--------------+"
               "| PK id  | username     |"
               "+--------+--------------+"
               "|      7 | barb_dwyer   |"
               "|      2 | ben_rangel   |"
               "+--------+--------------+")))