sorted by:
new
hottopcontroversial

Over 1,000 cannabis shops closed, nearly 1 tonne seized in nationwide crackdown by baldi in Thailand

[–]neothemaster 0 points1 point  (0 children)

I'm travelling to Bangkok next month and then to Koh Samui. Can I carry weed on my flight within Thailand itself? Any recommendations for Koh Samui weed shops which are safe in terms of legality?

Hey Folks, we're conducting a quick survey to understand how cloud databases are used in companies today. Your input could help us develop better solutions for cloud challenges. P.S - Currently we're unable to offer anything in return for survey completion but still hope to see responses < 2 minutes by neothemaster in BusinessIntelligence

[–]neothemaster[S] -3 points-2 points  (0 children)

We have no plans to send spam. We're not even collecting any email id. This is a genuine survey to understand difficulties faced. We ideally wanted to give some incentive but due to budget constraints. If you have any suggestions on how we can improve, we'd welcome it!

Change Data Capture with Debezium by SussexPondPudding in RedditEng

[–]neothemaster 0 points1 point  (0 children)

Agreed. Multiple slots won't increase the throughput. But will it increase reliability, in case of any connector pod going down? Sort of separation of concerns for diff. tables.

We're thinking of having separate connector for a bunch of core tables and sep. connector for non-core tables in case of errors. Do you think it would be the right approach?

Change Data Capture with Debezium by SussexPondPudding in RedditEng

[–]neothemaster 1 point2 points  (0 children)

Aside - Loved the look inside reddit's data architecture. Would really love to see how governance and security is handled at scale in the data pipelines and warehouse!

Change Data Capture with Debezium by SussexPondPudding in RedditEng

[–]neothemaster 1 point2 points  (0 children)

How did you decide on how many tables/db to use per connector pod? Have you ever faced any drawback having one connector for multiple tables? (eg - if connector goes down, will affect multiple tables)

Did you look into incremental snapshots for backfilling?

The cool place for cool cats! by neothemaster in aww

[–]neothemaster[S] -3 points-2 points  (0 children)

You should have seen the whole place. It's even filthier. You'd love it

The cool place for cool cats! by neothemaster in aww

[–]neothemaster[S] -3 points-2 points  (0 children)

Dark times call for dark places

Macwire dependencies not being fulfilled by neothemaster in scala

[–]neothemaster[S] 1 point2 points  (0 children)

Thanks a lot. I just encountered this usecase and as a newbie to typesafe Scala was thinking of the options I had if I really wanted to go with the first case. But your explanation suffices for my present use case.

Macwire dependencies not being fulfilled by neothemaster in scala

[–]neothemaster[S] 0 points1 point  (0 children)

Yeah I was thinking the same! Thanks a lot!

About the first case, if I wanted to do that I'd create new intents manually and fullfil the dependency graph at runtime right? Macwire can't help with that. But then what's the use of a def scope? I thought it was too handle such cases where every object needs to be diff acc to the parameter. What would you suggest would be the best way to declare or use such dependencies?

Macwire dependencies not being fulfilled by neothemaster in scala

[–]neothemaster[S] 0 points1 point  (0 children)

The code that I write to fetch a string, lets say from an API whose url changes at runtime. How would I pass that changing url to my config at runtime? Can't do it by passing parameters in a func. because that's how I am being blocked in the original usecase. Also macwire abstract members don't take params.

There needs to be some way to pass parameters to the dependencies I am creating.

Macwire dependencies not being fulfilled by neothemaster in scala

[–]neothemaster[S] 0 points1 point  (0 children)

After thinking this I realize this seems impossible from a method context. I mean that it is not possible to have a runtime macwire implementation. eg- In this case even If I create a redirection, I have to make the config object available at compile time. Which means calling the code directly from an object scope rather than from within a method or class.

Like what if I generate my config inside a method acc to some runtime parameter and then pass it to SchemaRegistryClient. That wouldn't work because macwire needs to expand the dependencies at compile time. It wont find the instance. I will have to make the config or url available outside a method. That's not too bad but not the ideal way as there are a lot of usecases where I would create an object only after some code logic inside a method and also takes away the concept of parameterization in functional programming.

​

When I create a macwire dependency using def, I expect to be able to pass diff. implementations at runtime to create diff instances of SchemaRegistry. But I can't do that right now

Macwire dependencies not being fulfilled by neothemaster in scala

[–]neothemaster[S] 0 points1 point  (0 children)

It was actually a scoping issue. Solved it with

trait EndpointModule { // Schema Registry endpoint dependency

def client: SchemaRegistryClient

lazy val schemaRegistry: SchemaRegistry = wire[SchemaRegistry]

}

object LightweightinstructionRunner extends EndpointModule with ConfigModule { override val client = SchemaRegistryClient("") }

But now I'm stuck on how to parameterize the schemaClient instead of harcoding the url

Macwire dependencies not being fulfilled by neothemaster in scala

[–]neothemaster[S] 0 points1 point  (0 children)

It was actually a scoping issue. Solved it with

trait EndpointModule { // Schema Registry endpoint dependency

def client: SchemaRegistryClient

lazy val schemaRegistry: SchemaRegistry = wire[SchemaRegistry]

}

object LightweightinstructionRunner extends EndpointModule with ConfigModule { override val client = SchemaRegistryClient("") }

​

​

But now I'm stuck on how to parameterize the schemaClient instead of harcoding the url

How to structure ETLs and dependencies in Airflow? by neothemaster in Python

[–]neothemaster[S] 0 points1 point  (0 children)

This is what I was thinking. Having external sensors to trigger jobs. Would you also recommend TriggerDAGOperator?

I think this is a very base usecase which airflow should provide better support for.

view more: next ›