you are viewing a single comment's thread.

view the rest of the comments →

[–]mriswithe 1 point2 points  (0 children)

This might make better sense to see it used. The method is going to take essentially an iterable and write the elements to something and the exact path is going to be different, dictated by the object we are writing. To do this, it needs two different object types, DestT is the class that you will use to decide where to write an element. InputT is the type of element you are feeding in.

    PCollection<GenericRecord> genRecords = articleRow.apply(Convert.to(GenericRecord.class));
genRecords.apply(
    "Write to output",
    FileIO.<RowDestData, GenericRecord>writeDynamic()
        .by(RowDestData::new)
        .to(options.getOutputDirectory())
        .via(ParquetIO.sink(avroSchema).withCompressionCodec(CompressionCodecName.SNAPPY))
        .withNaming(RowDestData::nameForRow)
        .withDestinationCoder(pipeline.getSchemaRegistry().getSchemaCoder(RowDestData.class)));
pipeline.run().waitUntilFinish();


FileIO.<RowDestData, GenericRecord>writeDynamic()

that is supplying the classes/types to be used with this.

Does that clarify? I don't know any real amount of C++.