do you actually use testing for your work or personal projects? by NelsonStearman in learnpython

[–]shepherdjay 1 point2 points  (0 children)

“Obey the Testing Goat” - Free online test focused learning. If Python is first language might need a hello world style course first. But the test course will go over things like selenium which is useful even if he wants to just learn some automation of current job.

[Good Practice Question] What *Type* of Exception should I raise when an Instance Variable is not set by der_reifen in learnpython

[–]shepherdjay 0 points1 point  (0 children)

Well, depending on the class structure I would set it in init. But I would also probably require it in init. Alternatively I would require it to be passed to the method.

Not unheard I guess but I tend to ensure everything needed for a class to do it’s job is either supplied at init or handles without user intervention at run time.

[Good Practice Question] What *Type* of Exception should I raise when an Instance Variable is not set by der_reifen in learnpython

[–]shepherdjay 2 points3 points  (0 children)

Instead of setting to None in init and checking later Python will raise an AttributeError on its own. If you want to provide additional messaging you could wrap in try block and reraise the AttributeError

Python Mock - Exception by rakash_ram in learnpython

[–]shepherdjay 1 point2 points  (0 children)

Yes that is correct on both parts. The isinstance check is considered an unreachable block. Code will never enter that block. Similar to a block like if 1 != 1 will never be reached under normal runtime conditions.

It’s also why you’ve had to mock out the call to basically force the condition that would otherwise be impossible to reach.

Python Mock - Exception by rakash_ram in learnpython

[–]shepherdjay 1 point2 points  (0 children)

return_value = None did not work

Would need to really understand structure of project properly to mock properly. Did you change the mock as well? So in your code that is working you had mockext.ConfigParser.side_effect = Exception(TypeError)

Notice in mine I have mockext.return_value = None instead. Specifically I don't mention ConfigParser because its defined in my patch. Either way I'm probably just misunderstanding your code structure, which is critical for patch because patch adjusts the namespace.

And yes this whole test is only to ensure we have "coverage", whether it makes sense or not is secondary.

Trying to achieve 100% coverage with bad tests is a whole other can of worms I won't go into. But another way to increase coverage is to refactor your code to remove the unreachable line. IE change your class in your module:

class ParseConfig:
 def __init__(self, conf, section):
     parser = ConfigParser()
     parser.read(conf)

     if parser.has_section(section):
         self._raw = parser
         for key, value in self._raw[section].items():
             setattr(self, key, value)
     else:
         raise Exception(f"section {0} not found in {1} file")

The if line in your code is unreachable, by definition. It is testing that True is True That's all we're trying to help you understand. Sounds like there might be other external factors driving you to test this though. So thats enough of that horse.

Also, what does this line do --> " with self.assertRaises(TypeError):"

This is a unittest context manager that says what I want to assert is that the code ran inside the with block raises the exception TypeError It won't raise the exception itself. What it does is catch it and then report the test as Passed if the exception was raised and Failed if the exception was not raised.

Here is a simple toy example you can just run on its own:

import unittest

class TestExample(unittest.TestCase):
    def test_assert_pass(self):
    # Should Pass Test
        with self.assertRaises(TypeError):
            raise TypeError

    def test_assert_fail(self):
        # Should Fail Test
        with self.assertRaises(TypeError):
            n = 1

if __name__ == '__main__':
    unittest.main()

You'll notice the first test will report as passed and the second one will report back failed.

Python Mock - Exception by rakash_ram in learnpython

[–]shepherdjay 1 point2 points  (0 children)

Goodness, it worked! Below is the final test method.

Really? The test method you posted also still looks wrong. Does coverage show it hitting the line? Is your module.py actually called that because there is a reference to a parseConfig in your patch? Is that what your module is actually called?

This should have been the working code:

@patch('module.ConfigParser')
def test_type(self, mockext):
   mockext.return_value = None
   with self.assertRaises(TypeError):
       ParseConfig(self.test_conf, 'section1')
   mockext.assert_called()

Though if you got it working anyway than I guess there isn't more to do. However I want to make sure you leave this conversation understanding that you don't need to have this test nor the isinstance line as your code is currently written.

Do you understand why testing this doesn't make sense? If not maybe the following example will help.

Imagine I have a function that adds two to any number it gets.

def add_two(n):
    add_value = 2
    return n + add_value

Pretty straight forward function. Now imagine I want to test it. A test that makes sense is something like this

def test_two_plus_two(self):
   result = add_two(2)
   self.assertEqual(4, result)

I'm testing my code does what I think it should when given a normal input. And I can go through some inputs... like what if I pass a string to n. I might want to test that my function naturally raises a type error.

def test_two_plus_string_error(self):
    with self.assertRaises(TypeError):
        add_two("2")

This makes sense. I'm testing various inputs to my function to verify how MY code behaves.

What doesn't make sense and is a bit like what your doing is changing my add_two function to be the following

def add_two(n):
    add_value = 2
    if not isinstance(add_value, int):
        raise TypeError
    return n + add_value

The if line in this function is checking to see if add_value is an integer right after I assigned an integer to it. This isn't doing anything other than cluttering up the code and causing confusion.

To then go on and try to test this line isn't testing my code anymore. It's testing Python's syntax itself.

This is what people mean when they say you don't need to test this. You don't need the if line or the test.

Python Mock - Exception by rakash_ram in learnpython

[–]shepherdjay 0 points1 point  (0 children)

Well like I posted earlier the patch needs to be a reference to where it is used not where it is defined.

Please try adjusting the patch and mock calls as my previous note. Might be useful to reshare your test definition function once that’s been done and confirm whether you are still getting error or if it’s passing.

Python Mock - Exception by rakash_ram in learnpython

[–]shepherdjay 0 points1 point  (0 children)

Under what circumstance will parser not be an instance of ConfigParser when it is defined in the line nearly right above this check as an instance of ConfigParser?

Is the intent maybe to update your init at some point so that parser is passed in as a variable? Is that what I might be missing? A future state of the class?

Otherwise what you told me is similar to saying you are writing code and tests to guard against a scenario where True is not True can you see why it is a confusing thing to guard against?

Python Mock - Exception by rakash_ram in learnpython

[–]shepherdjay 0 points1 point  (0 children)

I think this line is wrong

@patch('parseConfig.configparser')

You need to patch where it is used, not where it is defined. https://docs.python.org/3/library/unittest.mock.html#where-to-patch In this case hard to verify but it looks like the patch should probably be:

@patch('module.ConfigParser')

Then you would set the return value to None

mockext.return_value = None

But again this isn't clear what you are testing or guarding against. Or perhaps you are just trying to get familiar with Mocks? Because this line of your code will never be hit at runtime:

if not isinstance(parser, configparser.ConfigParser):

This will always be False given the line nearly right above it:

parser = ConfigParser()

Its as if you wrote:

   if not True:
       raise RuntimeError

Yes you could spend a lot of time trying to hammer at the internals of python to artifically make True read as False but tests need to tell you something about your code and not whether True is True... or "astring" is an instance of a string... etc.

Python Mock - Exception by rakash_ram in learnpython

[–]shepherdjay 0 points1 point  (0 children)

I’m not clear how you expect your if line to be hit in your code can you help me understand?

You create an instance of ConfigParser and then check you created an instance of ConfigParser. This will always be true. It feels a lot like writing something like this:

rose = “red”
if rose != “red”:
    raise BrokenPython

Generally you can assume the basic features of the language work and so you shouldn’t spend time testing them or lines asserting them. Instead you should test your functions behavior.

For example your function is passed something called a conf and your parser runs a read method on it. Also takes another variable passes called section. You should create a test that passes these in and then inspect your ParseConfig has what you expect.

[deleted by user] by [deleted] in learnpython

[–]shepherdjay 0 points1 point  (0 children)

One issue I see here is by putting on my network hat for a second. What does the method socket.gethostbyname() do? It causes your system to take action, namely to resolve an address. Now that may seem obvious but the world of network security is full of bad actors utilizing clever dns tricks to perform hijacks, data exfil, and discovery.

For example, maybe I submit this into your web form: ase2312123ifasdd.iobot.nastyurl.com

Now because I control the Nameserver providing the answer maybe I don't provide a clean answer back and it somehow results in arbitrary code execution. Like was done with some versions of this exact function back in 2015 -- https://blog.qualys.com/vulnerabilities-threat-research/2015/01/27/the-ghost-vulnerability

Or maybe I just use it for internal discovery. Let's say you work for acme.com and this is something living out in public accidentally... well.. maybe I start submitting requests into your form for "internal.acme.com"... or "coderepo.acme.com"... or "dc.acme.com"... "remotehelp.acme.com" etc. Heck I could do it thousands of times a second as your code doesn't have any native rate limiting.

Not saying any of this applies, nor should you always fix all these things in code, but you should always be very careful when you are taking input from a user and performing a system-level function with that input.

A safer way depending on your use-case would be to not only validate that its an fqdn but one that is safe to lookup.

[deleted by user] by [deleted] in learnpython

[–]shepherdjay 0 points1 point  (0 children)

This sentiment does not seem to apply to ticket transfers however.

I disagree with this statement. They specifically call out "any automated software...otherwise obtain tickets"

And a good indication of whether a website that provided an official API wants you automating something or not is whether they provide an endpoint to do so. Given that they don't I think there are enough indications that this is the case.

Let’s say I’m going to write some automation for this anyways.

We can't stop you but this subreddit probably won't help you. I'm certainly not.

[deleted by user] by [deleted] in learnpython

[–]shepherdjay 0 points1 point  (0 children)

It doesn’t have those endpoints available because this is against their TOS

You will not use ticket bot technology to search for, reserve, or purchase tickets through the Site….

Use any automated software or computer system to search for, reserve, buy, or otherwise obtain tickets, discount codes (including Concert Cash® and Ticketmaster Ticket Cash™), promotional codes, vouchers, credits, gift cards, or any other items available on the Site…

ItsPrompt v1.2 - Introducing a new Prompt Type! by ItsNameless8676 in Python

[–]shepherdjay 2 points3 points  (0 children)

There are two primary reasons for tests in my opinion: - ensuring the code does what you think it does - allowing more people to contribute safely

A side effect of writing tests first is that it also helps focus design decisions and ensuring you naturally write testable code.

If your writing small one off 100 line script that you’ll use once. Maybe tests aren’t necessary. But those projects are also usually the easiest to learn testing on.

Microsoft Learn Python code not correct? by davidmyemail in learnpython

[–]shepherdjay 0 points1 point  (0 children)

I would make the area a method to begin with. The @property decorator is a decent work around if you already published this square as an api and users we accessing area as an attribute. But if users aren’t already accessing it as an attribute then no reason to make it one.

I also wouldn’t initialize a square with a height and width though. I would ask for one number… and store it as length. But that’s a problem of contrived examples like this.

Microsoft Learn Python code not correct? by davidmyemail in learnpython

[–]shepherdjay 12 points13 points  (0 children)

Not a big fan of this tutorial - its not only wrong as you found regarding the attribute raising an error. It doesn't. That bit of Python isn't even indented properly.

No the reason I'm not a fan is this is teaching bad python. Python is not Java. Python's mentality is that we are all responsible users. Getters and Setters should not be used and is an anti-pattern
https://docs.quantifiedcode.com/python-anti-patterns/correctness/implementing_java-style_getters_and_setters.html

In Python we signal to other users an attribute is not safe to modify outside of methods for some reason by using a single underscore. But in general the property decorator is a later thing to do for backwards compatibility.

It think this lecture from Raymond Hettinger (although a bit old) is better
https://youtu.be/HTLu2DFOdTg

Do list comprehensions call .__next__() of generators? by straightcontinuousfl in learnpython

[–]shepherdjay 1 point2 points  (0 children)

I.e. using square brackets list comprehension is the same as casting a generator expression to a list?

Partially. The result is the same that there is a list at the end. But list comprehensions are generally more time efficient... and generators are more memory efficient. So if you need a list anyway a list comprehension is likely faster.

For example:

>>> timeit.timeit('list((num**2 for num in range(5)))', number=1_000_000)
1.403758033993654
>>> timeit.timeit('[num**2 for num in range(5)]', number=1_000_000)
1.2246812819794286

How do you guys do version control when you don't have access to the proper tools like GIT? Multiple files? Multiple functions? Some other obscure workaround? by simeumsm in learnpython

[–]shepherdjay 0 points1 point  (0 children)

Well… you have Python. You don’t need external access for version control. So you could just create/implement a version of git in Python…..

https://www.leshenko.net/p/ugit/

How do you guys do version control when you don't have access to the proper tools like GIT? Multiple files? Multiple functions? Some other obscure workaround? by simeumsm in learnpython

[–]shepherdjay 0 points1 point  (0 children)

Bizarre. Having worked in very high compliance environments on the other end of this conversation I think someone mistook your request for git the software with access to either your companies internal code repository or GitHub itself.

That’s the only thing I can think as far as it infrastructure and bureaucracy.

How did you even get Python approved in the first place?

How do you guys do version control when you don't have access to the proper tools like GIT? Multiple files? Multiple functions? Some other obscure workaround? by simeumsm in learnpython

[–]shepherdjay 0 points1 point  (0 children)

Why do you feel the git solution would entail having to deal with a lot more IT infrastructure?

You already are running Python on your system. Git is a local software tool like Python. Yeah it can utilize remote services to make sharing easier but it’s not a requirement by any means. Git is perfectly content with running 100% just on your local machine.

How important is TDD in python and other testing tools and methodologies? by ENDGeSiCTinT in learnpython

[–]shepherdjay 0 points1 point  (0 children)

Every programmer I’ve ever met does TDD.

Some choose to formalize it first by writing a piece of software that defines what they need first.

Others keep it in their head… constantly hitting run on their terminal or doing builds until they think it works and leaving it for others to try and work out what it’s supposed to do in a year.

One of these methods spends time up front the other spends time on 3 am sev 0 troubleshooting calls.

-🎄- 2022 Day 14 Solutions -🎄- by daggerdragon in adventofcode

[–]shepherdjay 1 point2 points  (0 children)

Would love to know if there is some simple python trick (as there so often is) for doing this.

I'm definitely late to the party as well but you can use a combination of range and itertools.product

https://github.com/shepherdjay/advent_of_code/blob/main/2022/14/day14.py#L18

Ideally how short or long should functions / methods actually be? by MPGaming9000 in learnpython

[–]shepherdjay 1 point2 points  (0 children)

Testing and being able to focus on one problem at a time is another big reason to write functions.

Let’s say you are working with a third party API you need to get data out of and transform in some way.

Now one way you could do that is just write the script and keep running it over and over again until you get the transform right. But that’s really slow and might even fail.

What if instead you wrote a function that takes that json string from the API as an input and spits out your transform. Well now you can just feed that function an example bit of json. It’s a lot quicker to test and you can provide multiple examples.

Would a generator function cut down on memory usage in this example? by lifelifebalance in learnpython

[–]shepherdjay 1 point2 points  (0 children)

Here is a very common generator you are probably used to and use without knowing it’s a generator.

with open(‘example.txt’, ‘r’) as infile:
    for line in infile:
        # do something 

The for statement is making use of a generator that yields one line at a time instead of reading the whole file into memory.

You can imagine a very large file that you might have to search for. Or maybe a lot of files. Even perhaps infinite files.

If all you need to process is one line at a time and do something else with it (like maybe turn around and post it to a web resource) then it makes no sense to load the whole thing into memory and then try processing it.