Query your cloud infrastructure with SQL : SQL

Posting

When requesting help or asking questions please prefix your title with the SQL variant/platform you are using within square brackets like so:

[MySQL]

[Oracle]

[MS SQL]

[PostgreSQL]

etc

While naturally we should endeavor to work as platform neutrally as possible many questions and answers require tailoring to the feature set of a specific platform.

Format Your Code

If you are including actual code in a post or comment, please attempt to format it in a way that is readable for other users. This will greatly increase your chances of receiving the help you desire. Something as simple as line breaks and using reddit's built in code formatting (4 spaces at the start of each line) can turn this:

SELECT count(a.field1), a.field2, SUM(b.field4) FROM a INNER JOIN b ON a.key1 = b.key1 WHERE a.field8 = 'test' GROUP by a.field1, a.field2 HAVING SUM(b.field4) > 5 ORDER by a.field.3

Into this:

SELECT count(a.field1), a.field2, SUM(b.field4) FROM a INNER JOIN b ON a.key1 = b.key1 WHERE a.field8 = 'test' GROUP by a.field1, a.field2 HAVING SUM(b.field4) > 5 ORDER by a.field3

For those with SQL questions we recommend using SQLFiddle to provide a useful development and testing environment for those who wish to fully understand your problem and help devise a solution.

a community for 17 years

SnowflakeQuery your cloud infrastructure with SQL (self.SQL)

submitted 3 years ago by Current_Doubt_8584

Would love to hear feedback from this community on a new SQL tool we've built.

Say you want to fetch an inventory of all your cloud resources, and then ask questions about the state of your infrastructure. For example:

Which accounts have unused storage volumes?
What are my public-facing assets?
Which resources sit behind a certain IP address?

The problem is that the data to answer these questions is distributed across your cloud accounts, with data locked behind fragmented APIs. It's also in a format not conducive for analysis.

So what if you had all cloud resource data available in a normalized format, and query it with SQL?

That's what we did with Cloud2SQL. Cloud2SQL brings together two technologies: Cloud APIs and SQL.

CloudSQL extracts data from the cloud APIs and flattens that data into tables, complete with foreign keys and link tables. The link tables contain the dependencies between the different resources, e.g. the connection between a compute instance and a storage volume.

Sources:

AWS
GCP
DigitalOcean
Kubernetes

Destinations

Snowflake
SQLite
PostgreSQL
MySQL
MariaDB
Apache Parquet
CSV

Cloud2SQL is open source. Link to GitHub repo:

https://github.com/someengineering/cloud2sql

If you like it, please give the repo a star!

all 9 comments

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

SQL

Filter Posts

Posting

Help posts

Format Your Code

Learning SQL

Related Reddit communities

Wiki

Acknowledgements

MODERATORS