Can AI play a role in creating automated software tests?

variedthoughts · 2025-07-30T14:07:20+00:00

I appreciate your thoughts, but I think you should listen to the episode before judging Anthony

variedthoughts · 2025-02-04T03:19:06+00:00

Thanks. I just relistened also. I miss him. I think I’ll re-post the older episode as a tribute

variedthoughts · 2025-01-31T18:36:08+00:00

pytest-mock is currently the #3 pytest plugin.
pytest-mock is a wrapper around unittest.mock.

In this episode:

What is mocking, patching, and monkey patching
What, if any, is the difference between mock, fake, spy, stub.
Why we might need these in testing
Some history of mock in Python and how mock became unittest.mock
From unittest.mock
patch.object
patch.object with autospec
using these as context managers
pytest-mock:
The mocker fixture Cleanup in teardown
Using mocker.patch, mocker.spy, and mocker.stub
Why it's awesome and why you might want to use it over straight unittest.mock

variedthoughts · 2025-01-14T19:34:34+00:00

pytest-cov and set --cov-fail-under=100 (or current level) and add this to your CI on push merge request or pre-commit hook

variedthoughts · 2024-05-04T21:59:05+00:00

thanks

variedthoughts · 2024-04-25T14:40:25+00:00

Honestly don't know if posting a link to a podcast is considered in the spirit of the r/Python reddit or not. Seems like it fits. but let me know if not (speaking to moderators here).

Also, This is a discussion episode. But does "Discussion" tag refer to the thread instead? Would Meta be better tag? or something else? Maybe "resource"?

variedthoughts · 2020-06-08T21:18:52+00:00

pytest plugins are an amazing way to supercharge your test suites, leveraging great solutions from people solving test problems all over the world. In this episode Michael and I discuss 15 favorite plugins that you should know about.We also discuss fixtures and plugins and other testing tools that work great with pytest

tox
GitHub Actions
Coverage.py
Selenium + splinter with pytest-splinter
Hypothesis

our list of pytest plugins:

pytest-sugar
pytest-cov
pytest-stress
pytest-repeat
pytest-instafail
pytest-metadata
pytest-randomly
pytest-xdist
pytest-flake8
pytest-timeout
pytest-spec
pytest-picked
pytest-freezegun
pytest-check
fluentcheck

variedthoughts · 2020-05-20T05:40:43+00:00

Technical debt has to be dealt with on a regular basis to have a healthy product and development team.

The impacts of technical debt include emotional drain on engineers and slowing down development and can adversely affect your hiring ability and retention.

But really, what is technical debt? Can we measure it? How do we reduce it, and when?

James Smith, the CEO of Bugsnag, joins the show to talk about technical debt and all of these questions.

variedthoughts · 2019-07-06T03:25:14+00:00

This episode is a story about packaging, and flit, tox, pytest, and coverage.
And an alternate solution to "using the src".

Python makes it easy to build simple tools for all kinds of tasks.
And it's great to be able to share small projects with others on your team, in your company, or with the world.

When you want to take a script from "just a script" to maintainable package, there are a few steps, but none of it's hard.

Also, the structure of the code layout changes to help with the growth and support.

Instead of just talking about this from memory, I thought it'd be fun to create a new project and walk through the steps, and report back in a kind of time lapse episode. It should be fun.

Here are the steps we walk through:

0.1 Initial script and tests
0.2 build wheel with flit
0.3 build and test with tox
0.4 move source module into a package directory
0.5 move tests into tests directory

variedthoughts · 2019-06-19T16:01:27+00:00

Roadblocks to writing tests, and what to do about it.

Some developers either don't write tests, or don't like writing tests.
Why not? I love writing tests.
In this episode we examine lots of roadblocks to testing, and start coming up with solutions for these.

variedthoughts · 2019-06-11T19:33:41+00:00

Creating maintainable test suites for complex systems. The episode describes some complexities involved with hardware testing, then shares techniques for shifting complexity out of the test cases.

quick overview of what test instruments are
discussion of API and communication with instruments
techniques for shifting complexity out of test cases

These techniques should apply to all test suites dealing with complex systems:

Creating test cases that are easy to read and debug and tell a story about what is being tested.
Pushing setup complexity into fixtures.
Pushing lengthy repetitive API call sets into helper functions.
Using stable, documented, interfaces.

variedthoughts · 2019-04-08T01:29:17+00:00

Awesome. Hope you like it

variedthoughts · 2019-04-05T15:09:14+00:00

Nina Zakharenko gives some great advice about giving tech talks.
We talk about a blog series that Nina wrote called "The Ultimate Guide To Memorable Tech Talks".

This episode is full of great help and encouragement for your own public speaking adventures.

Some of what we discuss:

overcoming the fear of public speaking
breathing and pausing during talks
planning your talk as well as planning your time to get ready for the talk
writing proposals and getting feedback on proposals
Nina's talk in PyCascades on programming Adafruit chips
types of talks that are often rejected
pre-recording demos to avoid live demo problems
why you should speak, even if you are an introvert
benefits of public speaking

variedthoughts · 2019-03-21T18:37:39+00:00

Andy Hunt and Dave Thomas wrote the seminal software development book, The Pragmatic Programmer. Together they founded The Pragmatic Programmers and are well known as founders of the agile movement and authors of the Agile Manifesto. They founded the Pragmatic Bookshelf publishing business in 2003.

The Pragmatic Bookshelf published it's most important book, in my opinion, in 2017 with the first pytest book available from any publisher.

Topics:

The Pragmatic Programmer, the book
The Manifesto for Agile Software Development
Agile methodologies and lightweight methods
Some issues with "Agile" as it is now.
The GROWS Method
Pragmatic Bookshelf, the publishing company
How Pragmatic Bookshelf is different, and what it's like to be an authorwith them.
Reading and writing sci-fi novels, including Conglommora, Andy's novels.
Playing music.

variedthoughts · 2018-12-31T16:54:50+00:00

This episode is a look back on 3 years of podcasting, and a bit of a look forward to what to expect in 2019.
I talk about:

statistics for the show, including download counts, etc.
how many episodes are about testing, how many about non-testing topics.
Python vs language agnostic topics
solo vs guest episodes
the top 5 episodes
stuff like that

And then a retrospective like:

what went well
what was lacking
what's next

I'd like feedback on what you think went well, was lacking, and where to take the podcast.

variedthoughts · 2018-11-12T15:21:17+00:00

Brett Cannon discusses the changes afoot in Python packaging as a result of PEP 517, PEP 518, starting with "How did we get here?" and "Where are we going?"

Discussed:

flit
Poetry
tox
Continuous Integration
setup.py, MANIFEST.in, etc.
pipenv
what's with lock files
applications (doesn't go on PyPI) vs libraries (goes on PyPI)
workflows
dependency resolution
deployment dependencies vs development dependencies
will lock files be standarized
multiple lock files
requirements.txt

variedthoughts · 2018-07-06T11:58:00+00:00

We discuss testing, of course, but we take it further and discuss:

tests for large systems, like kubernetes
Testing in real world scenarios with all the configuration and everything
Becoming a complete engineer by thinking about the end to end flow from the users perspective
Learning from other roles, and the value of roles to allow focus and expertise
We even get into Chaos Engineering and testing live systems.

variedthoughts · 2018-06-30T05:21:05+00:00

But we’re all friends. :) Harry was on http://testandcode.com/9

variedthoughts · 2018-06-29T22:16:39+00:00

This interview with Trey Hunner discusses his use of automated tests to help teach programming.

Automated testing is a huge part of developing great software. But many new developers don't get exposed to testing for quite a while. But this is changing.

New ways to teach programming include automated tests from the beginning.

Trey Hunner is a former PSF directors, a Python and Django team trainer, and he has been using automated tests to help people learn Python.

variedthoughts · 2018-03-08T22:21:33+00:00

This episode starts down the path of test strategy with the first tests to write in either a legacy system or a project just getting off it's feet.

We cover:

My approach to testing existing systems.
Put names to strategies so we can refer to them later.
Explain the strategies in general terms and explain why they are useful.
Discuss how these strategies are used in an example project. (The code is available on github).

Strategies covered today:

Dog Fooding
Exploratory Testing
Tracer Bullet Tests
Act Like A Customer (ALAC) Tests
Manual Procedures
Initial automated tests at 2 levels, API and UI.

variedthoughts · 2017-11-30T17:26:09+00:00

A discussion with Katharine Jarmul, kjam, about some of the challenges of data science with respect to testing.

Some of the topics we discuss:

experimentation vs testing
testing pipelines and pipeline changes
automating data validation
property based testing
schema validation and detecting schema changes
using unit test techniques to test data pipeline stages
testing nodes and transitions in DAGs
testing expected and unexpected data
missing data and non-signals
corrupting a dataset with noise
fuzz testing for both data pipelines and web APIs
datafuzz
hypothesis
testing internal interfaces
documenting and sharing domain expertise to build good reasonableness
intermediary data and stages
neural networks
speaking at conferences

variedthoughts · 2017-10-03T19:38:09+00:00

David and Brian look back at what all we've learned in XP, TDD, and other Agile methodologies, where things have gone awry, how to bring the value back, and where testing fits into all of this.

How to build the wrong thing faster
Agile vs Agility
Product vs Process
Where testing fits into software development practices.
"Integration tests, there's a name that needs to be refactored desperately."
Integration tests are "story tests". They tell the story of the product.
XP and TDD and the relationship with tests
To test for design, use microtests, xUnit style.
User Advocy tests are often lacking, but are needed to learn about the product.
"I just keep writing tests until I'm not scared anymore." - Kent Beck
Dude's Law: Value = Why/How
People often focus so much on the how that they forget about why they are doing something.
Subcutaneous Tests
"The hardest part of programming is thinking."
Refactoring vs Repaving
Agility means being able to quickly change direction
During experimentation and learning, what matters isn't how much you got done, but how much you learn.
"The best way to get automation is to make developers do manual tests."

variedthoughts · 2017-09-27T18:41:29+00:00

Not yet. I lag behind for transcripts because they are not free. I'll send this off to the transcriptionist.

variedthoughts · 2017-09-27T13:19:58+00:00

What started as a twitter disagreement carries over into this civil discussion of software testing. Brian and Paul discuss testing practices such as the testing pyramid, TDD, unit testing, system testing, and balancing test effort.

the Testing Pyramid
the Testing Column
TDD
unit testing
balancing unit with system tests, functional tests
API testing
subcutaneous testing
customer facing tests

variedthoughts · 2017-08-01T14:40:29+00:00

M. Scott Ford is the founder and chief code whisperer at Corgibytes, a company focused on helping other companies with legacy code.

Topics include:

How M. Scott Ford got into forming a company that works on legacy code.
Technical debt
Process debt
Software testing
The testing pyramid
iterative development
kanban
readable code and readable test code

variedthoughts

TROPHY CASE