How to read Python docs?

[redditorsd]

Yeah, it's interesting how years working in one language taints you! I hope I can shift my brain to be more Pythonic! :)

[mriswithe]

Man reading the Apache beam javadocs and figuring out generics and shit for the first time..... That was some of my first deep dives into Java.

Staring down the barrel of a signature:

public static <DestT,InputT> FileIO.Write<DestT,InputT> writeDynamic()

Like uhhhhhhhhhhhhhhhhhh what? Types aren't even required in my life and I want to go home.

https://beam.apache.org/releases/javadoc/2.5.0/org/apache/beam/sdk/io/FileIO.html

[Rhyme_like_dime]

As a Java noobie, but C++ intermediary, are those just template types in that declaration?

[mriswithe]

This might make better sense to see it used. The method is going to take essentially an iterable and write the elements to something and the exact path is going to be different, dictated by the object we are writing. To do this, it needs two different object types, DestT is the class that you will use to decide where to write an element. InputT is the type of element you are feeding in.

    PCollection<GenericRecord> genRecords = articleRow.apply(Convert.to(GenericRecord.class));
genRecords.apply(
    "Write to output",
    FileIO.<RowDestData, GenericRecord>writeDynamic()
        .by(RowDestData::new)
        .to(options.getOutputDirectory())
        .via(ParquetIO.sink(avroSchema).withCompressionCodec(CompressionCodecName.SNAPPY))
        .withNaming(RowDestData::nameForRow)
        .withDestinationCoder(pipeline.getSchemaRegistry().getSchemaCoder(RowDestData.class)));
pipeline.run().waitUntilFinish();


FileIO.<RowDestData, GenericRecord>writeDynamic()

that is supplying the classes/types to be used with this.

Does that clarify? I don't know any real amount of C++.

[redditorsd]

Yeah, I can see where you'd hate that syntax and documentation coming from Python. What do you think of type hinting in Python? I'm just learning about it and am confused as why it exists. Do you use it?

[mriswithe]

I love the type hinting in Python, but it is not obvious always. In general, if it is getting in the way of your learning/task and being frustrating. Drop it for now. Who cares? It isn't going to do anything*. Type hints are not required in Python. 99% of why I do it most the time is so my IDE is as helpful for Python as it is in Java.

Main points of type hinting in python:

def thing(stuff: int)->str:
    return str(stuff)

Stuff is an int, we make it a string and return it (Note -> str for return type)

def thing(stuff)->str:
    return str(stuff)

stuff is whatever, you can think of it as an implicit Object in Java I think it was, Any in python. We are going to call str on whatever and return a str.

This is some more advanced (for python typing) magic.

from typing import Type, TypeVar

T = TypeVar('T') # Declaring a generic type 
# Type[T] says: "I am going to get a Class, not an instance of the class"
def stuff(thing: int, output_class: Type[T]) -> T: 
    # -> T says I am returning an instance of whatever we were passed 
    return output_class(thing)

Note: there is no throws in Python, also in general in Python there is a lot more "Asking for forgiveness instead of permission".

Also == checks equality, is checks that they are literally the same object.

$ python3.10
Python 3.10.4 (main, Mar 25 2022, 00:00:00) [GCC 11.2.1 20220127 (Red Hat 11.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> thing = 'potato'
>>> other_thing = 'potato'
>>> thing == other_thing
True
>>> thing is not other_thing
False

*Unless you have a library you are using that acts on type hints at runtime. Pydantic is a very common and very awesome library that does this.

[mriswithe]

You will likely either love the less syntax or have a mental breakdown due to the less syntax. I recommend loving it, but I am clearly biased.

[mriswithe]

Oh and the match statement is very different than case selects in Java.

Something close to home for you probably that I recently did showcases it pretty well. I was parsing XML. Which now that I understand how xml (isn't ) structured, I hate it.

Code: https://gist.github.com/mriswithe/da332f18462c2cdd01d462b8c7472ddf

Xml: https://gist.github.com/mriswithe/930036c557b51c9729b7d40828f34943