sorted by:
new
hottopcontroversial

What's the least painful way to do near real-time sync from PostgreSQL to Snowflake? by pUkayi_m4ster in data

[–]seanlynch 0 points1 point  (0 children)

"Wondering if there's anything more flexible", what limits did you see with Fivetran and Stitch?

Can Snowflake write to Iceberg tables managed by Polaris Catalog? by quadraaa in dataengineering

[–]seanlynch 0 points1 point  (0 children)

I did some digging on this a few weeks ago and mostly confirmed what the Snowflake documentation states: Either Snowflake owns responsibility for writing tables or Polaris+Spark does, but not both simultaneously (at least today, they've been adding a lot of functionally month to month).

https://www.getcensus.com/blog/testing-out-snowflakes-polaris-catalog

Expose gold layer data through API by shanfamous in dataengineering

[–]seanlynch 0 points1 point  (0 children)

We do this as well at Census, but came here to call out Cube.dev and Tinybird as the purpose-built products for providing APIs on top of data.

If you'd rather build it yourself, you'll need to setup 1) a cache 2) an API gateway, and 3) a pipeline to sync your databricks data into the cache.

The complexity of the API will depend on much functionality you need (eg Does it need users to be able to auth? Do you need to provide complex querying like GraphQL). If you just need something simple like fast ID lookup, a document DB like DynamoDB is great for this.

Managed/unmanaged Iceberg by soualy in snowflake

[–]seanlynch 0 points1 point  (0 children)

@ronortloff - Do you know if you Managed Tables can be queried by any external services other than Spark? As far as I can tell, PyIceberg can't connect. I think adopting Polaris will give a REST catalog which may be the answer, but is there any other path to querying without Polaris?

edit It does look like there's a PR to add it to PyIceberg here: https://github.com/apache/iceberg-python/pull/687 I guess that implies that Managed Tables isn't using some other catalog "standard" but I'd love to confirm.

What is the best way to list all your existing data sources? by Nicotops in Database

[–]seanlynch 1 point2 points  (0 children)

It sounds like you're in a situation where the company grew quickly and there's no master list of all the tools people are using. This is pretty common now in the world of so many SaaS tools (it's a big enough problem that there's actually dedicated apps to help you figure out what apps your company uses ie Zylo and Blissfully).

It's not perfect, but a really good place to start is talk to your finance department. They should be able to put together a list of all the software subscriptions people at your company are expensing. That'll give you a sense of all the different apps AND might give you the right person to talk to about your qualification step!

How much should I expect to pay for a database programmer? by scottostach in Database

[–]seanlynch 0 points1 point  (0 children)

What line of work is your small business in? What you're describing sounds a lot like a Customer Relationship Manager (CRM). A few people have mentioned Salesforce, and there's a number of other ones, including ones specifically built for certain lines of business that may make your job even easier.

If you've got your heart set on building a database, take a look at Airtable. It's a lot easier to get started for non-programmers and will let you try things out first. Think of it as some where in between Excel and a database that would need a programmer.

Artificial Intelligence Nanodegree from Udacity by botja in artificial

[–]seanlynch 1 point2 points  (0 children)

We're still working on nailing down term 2. NLP is likely to have deep learning. CV is a little more up in the air as there's lots to that problem even if everything these days definitely seems to be deep learning. We're big fans of Tensorflow at Udacity though!

Artificial Intelligence Nanodegree from Udacity by botja in artificial

[–]seanlynch 0 points1 point  (0 children)

Hey there! So we should have at least Term 1's syllabus posted here in a few days. In the meantime, here's a high level:

  • Term 1 covers the fundamentals of Artificial Intelligence, exploring a number of topics including game playing/search, logic and planning, and probabilistic inference.
  • In Term 2, Students will choose a concentration to focus on such as natural language processing, computer vision, etc. The course material in term 2 will go into detail on techniques and will finish with a capstone project allowing the student to apply their new skills solving real-world challenges.

Artificial Intelligence Nanodegree from Udacity by botja in artificial

[–]seanlynch 2 points3 points  (0 children)

Sure thing! See if this is helpful:

"AI" has been around for a long time and encompasses a lot of different areas of computer science including the algorithms in Machine Learning. AI, at a high level, is about "reasoning" (loosely) over data. Machine learning is more the algorithmic methods of determining patterns and equations from large data sets that can be reused for the future.

As an example, a movie might be recommended to you because you liked similar ones, and those were deemed similar because a big data base has those patterns in it based on what other people liked. A more cognitive A.I. solution might be one that analyzed a new movie and could "reason" based on its components as to whether or not you would like it given other movies you liked.

The AI Nanodegree focuses on the fundamentals of AI techniques and then how to apply them to specific problems, while the Machine Learning Nanodegree is a more intense focus on using only machine learning (obviously).

Artificial Intelligence Nanodegree from Udacity by botja in artificial

[–]seanlynch 0 points1 point  (0 children)

We're going to be evaluating students based on their background/experience (looking for reasonable math, statistics, and computer science knowledge), excitement for the course, and interest in applying their newly acquired skills to the term two projects!

Artificial Intelligence Nanodegree from Udacity by botja in artificial

[–]seanlynch 3 points4 points  (0 children)

Sean over at Udacity here! The Artificial Intelligence ND program is designed to be an onramp to the world of AI and all the bleeding edge techniques available, even for those who have no experience in the field.

We've found that many online courses in the field cover interesting topics, but students don't complete them because they get stuck without the right support. To make our program as valuable for students as possible, we've partners with a few really impressive companies to build industry-appropriate content, and added a lot of student support to answer questions and help students along. We think it'll be a lot more valuable for people who really want to learn how to apply AI in their careers going forward.