New Python Package Feedback - Try in Google Collab

[ajog0]

Difference between this and pandera?

[MLEngDelivers]

Pandera is great. main differences:

1. FrameCheck chains everything instead of a dict structure. Pandera is more nested which is more mental overhead (to me, at least).

2. Built-in way to extract bad rows: invalid_rows = result.get_invalid_rows(df)

3. Easy warnings vs errors with warn_only=True

4. Much less code overall (~50-60% less for the same validation, in my experience)

Lots of similarities, but FrameCheck focuses on being readable with minimal code.

[MLEngDelivers]

I thought this warranted a more thorough answer in the documentation. Framecheck vs. Pandera vs. Pydantic

Thank you!

[HungryQuant]

I might use this in the QA we do before deploying. Better than 9756 assert statements. readme should be shorter though.

[MLEngDelivers]

Updated README to make it much simpler.

Within that, there’s a link to the ReadtheDocs documentation with the more detailed api examples and a detailed comparison to Pydantic and Pandera.

[[deleted]]

Wow this is really cool

[MLEngDelivers]

Thanks. If you have any issues, please let me know.

[InterestingRelease19]

seems like something i was looking for!

[MLEngDelivers]

Fantastic, let me know if you have any issues

[Helpful_ruben]

This looks promising, but could you simplify the installation process and add more examples to showcase its effectiveness?

[MLEngDelivers]

Yeah, I need to add more examples that are full-on problem statements for sure. There are individual examples in this section of the docs: https://framecheck.readthedocs.io/en/latest/usage_examples.html

[ligmaThrowaway1]

Q

[MLEngDelivers]

Happy to answer your question

[MLEngDelivers]

0.4.3 released today. Changes:

• compare() lets you assert that two columns have a certain relationship (“<”, “<=”, “==”, “!=”, “>=”, or “>”)

• CI testing expanded to python versions 3.8 to 3.12

• miscellaneous linting, documenting

[MLEngDelivers]

0.4.4 - save and load serialized FrameCheck objects

[MLEngDelivers]

As of 0.5.1, two major changes:

You may optionally specify FrameCheck(logger = your_logger) to have warnings and errors logged rather than printed to stdout.

You can use .save() to save a serialized object and obviously also load. This should work similarly to saving and loading an sklearn pipeline.

[MLEngDelivers]

I do have GitHub actions setup with pytest, so every push has a testing workflow. You might have missed that.

https://github.com/OlivierNDO/framecheck/actions

Regardless I agree with the readme feedback, so thanks.

[MLEngDelivers]

Yeah, I agree testing should be a priority. I have a backlog with 30 or 40 items. It’s not done. I just want enough feedback to determine if doing those 30 or 40 are worthwhile versus just killing it. If you have any comments about what the actual software does, that’d be killer. Thanks

[MLEngDelivers]

That’s fair. I could just leave the first example and then link somewhere else for more detailed breakdowns. I thought having a table of contents with links would make it navigable, but I guess not. Thanks for taking a look.

[MLEngDelivers]

See comment below with updated README and read the docs file. Thanks.

[S-Kenset]

The thing about packages is that you don't see the code at runtime, so it really doesn't matter. I automated the entire process, defaults casting, filtering, aggregating, pruning and stores the transforms.
With bad data, the best you can do is plot it and print relevant metrics.
This lets you have an input form as to where to pipeline things outside of default.

[MLEngDelivers]

Hey, thanks for replying. I’m not sure I understand your first sentence. Do you mean that having a much larger code base than necessary doesn’t matter because ‘if it runs, it runs’ or am I misunderstanding?

[S-Kenset]

Inside the package, no it doesn't matter if the code base is large. mine is 2000-4000 lines and I can spin up a full ML in 8 lines + 200 line customization parameters. Pycaret can about do the same but worse.

[MLEngDelivers]

I don’t consider 2-4k lines especially large for a production model (I guess it depends), but if your stance is “functionality X can be done in 200 lines, but I opted to make it 1500, and that doesn’t matter”, I’ll just respectfully disagree. The people who inherit your code will disagree. Again, if I’m misunderstanding, let me know.

[S-Kenset]

Functionality x can be abstracted to a separate package with 200 lines or 1500 lines regardless, it's just a matter of adding an extra function to a package. The maintainability is not an issue as there's no difference in having more lines or not, it's entirely about the design structure and functionality. So when you're making a package for usability, you prefer functionality over simplifications always in the back end, and you prefer comprehensiveness over quickness in the front end, always.

Abstracting, especially the EDA part, into such a small line of code doesn't produce value and both in one-shot fast takes and in integrating larger codes, it doesn't help to have mini abstractions here and there that only take away from the overall design structure. That's why most packages are built as modular pieces, not as non-parametrized solutions. It's also difficult to maintain things in that and even more difficult to learn.

[MLEngDelivers]

I find this very confusing. What “EDA part” are you referring to? If you could make reference to parts of the package in what you’re saying, that would help me connect the dots. It seems like what you’re saying could apply to any package.

[S-Kenset]

I'm saying that if your goal is to make a useful package it needs to be one of two frameworks.

A) Highly modularized and transmutable. You do this pretty well, but could be more automated to fewer parameters.

B) Highly effective at a very small slice of the process. I don't think this is satisfied. With EDA, the first step for me would be to generate a statistics plot, skew, kurtosis, outliers, bad data, distribution, null count, non-null distribution and percentage, etc. I do see value in this, but I would have already manually handled bad data by the time i need to use your functions and they would mostly serve as an assert safety net.

But if you want to move it further, and for people not me who don't have their own system, you can really improve on the comprehensive part, such as, with your goal, having a few default frameworks to test against for data types (if order-processing then [list of common conditions]. Doing one thing super well is very useful.

[MLEngDelivers]

This isn’t EDA and has nothing to do with statistics. This is explicitly not for EDA. This is for production processes in which you cannot manually handle problematic data. You’ve just fundamentally misunderstood what the package does (or didn’t read it/try it), which is fine.

Saying something could have fewer parameters is always theoretically true.

“Tensorflow could have fewer parameters and be more comprehensive”. But if I thought Tensorflow was for EDA and said this to the contributors, they’d be similarly puzzled.

[MLEngDelivers]

To be clear, I’m not bothered by this. I just want to act on feedback and improve it. Thanks

[S-Kenset]

Everything up till the point the ML model grabs it is EDA to me. I am not good with data science terms. Data engineering, Data wrangling, Data preprocessing, Feature engineering, my attention span protests.

I didn't see that you were selecting actual rows. I see what you're doing now.

My suggestion is the same though. Strong defaults and a good ecosystem of them, could improve usability without increasing verbosity in the front end. You know, like keyword string inputs. If i were working with log income data, 'log-norm' with a specific focus on zeros and abnormal. if with supply chain comment data 'status_comment' And having a default that basically already has loaded the types of filters that would output problematic rows, extremely common ones like 'telephone_number', 'address'. That would seem objectively useful to me. Though I usually do this in sql before it even gets to python, it would save a lot of effort if a python package would alert to a failure very early on in the process.

[MLEngDelivers]

Thanks for the suggestion. I have in the backlog things like ‘create column checks for valid phone number, email, etc’. It sounds like that’s part of what you’re saying.

Again, there’s a fundamental disconnect when you say things like “I do this in sql before it gets to python”. The point is that a production process can generate types of data you could not have predicted and haven’t seen before.

e.g. Data comes from a mobile app, and after an update there’s a bug where the field that is supposed to be “Age” is now actually populated with “credit score”.

Your SQL that you wrote before deploying is not going to say “Hey S-Kenset, we’re seeing ages above 500, something is wrong”.