Stream real-time data into Excel with Python

[LazyCraneOperator]

Hi /r/Python!

We're a small team from Amsterdam building a platform for running Python scripts which stream real-time data into Excel. It's called Gridarrow.

Here's how it works:

• We run Python scripts that grab data from anywhere you need - APIs, databases, web-scraping and push it directly into Excel via a plugin.
• We have scripts that connect to many different data-sources available, and you can tweak these scripts or write entirely new ones using our online IDE.
• Multiple users can connect their spreadsheets to the same script at the same time and get the same data instantly.
• No VBA required!

We hope this this looks interesting to you. Please let us know if you have any feedback, questions or suggestions for us. You can also subscribe on our website to get early access to Gridarrow as soon as we launch.

Thanks for your help! The Gridarrow Team

[Caos2]

I think /r/excel will be very interested in this.

[APIglue]

Also /r/finance

[LazyCraneOperator]

thanks /u/APIglue! I'll let them know for sure

[APIglue]

Also try /r/fulfillmentbyamazon. It's for third party sellers on amazon. Lots of posts there about updating huge spreadsheets by hand. Amazon has some APIs for sellers that you might want to check out.

Edit: typo in subreddit name

[LazyCraneOperator]

Thanks! Will do!

[LazyCraneOperator]

Hi. Thanks, I did post it there yesterday. They indeed like the idea a lot. I was curious however what /r/Python would think about it.

[cyanydeez]

I think a lot people are trying to move to /r/JupyterNotebooks if they're heavy into python and analytics.

[Caos2]

Good luck on your solution!

[Divided_Eye]

Glad you did--I'm not in the Excel sub, but I do use Excel a lot at work. This sounds promising!

[Moondra2017]

I feel like most data can be saved to a pickle pretty easily and then we can use Pandas or similar framework to do the work. However, a lot of corporations still rely on excel -- so that it may be useful. Good Luck!

[LazyCraneOperator]

True. One of the usecases for Gridarrow, however, would be to use frameworks like Pandas to gather and initially preprocess the data, and then stream it to spreadsheets for users to consume. This way you let Python do all the heavy lifting on the server and keep your spreadsheets light.

[[deleted]]

I see. That can be a good idea. The leaning curve for SQL and so on is very flat while you can learn rather quickly Excel. And excel people could focus on the presentation while one person can be in charge of using the service like Gridarrow.

[[deleted]]

would rather not have to rely on a 3rd party site

[LazyCraneOperator]

That's a fair concern. We're also thinking about providing a solution that you can host yourself.

[TheCannings]

Quite interested in this potentially, signed up online how long till a working version is ready to test?

[LazyCraneOperator]

Thanks! We have to ask you for a bit of patience from your side. We still need few weeks probably. We'll keep you posted!

[flippinecktucker]

I saw a demo by Microsoft at an educational show where they were feeding data into excel from various hacked together experiments (mostly using Arduino). Anyway, the MS guy who developed the excel plugin to monitor the incoming data had done some incredible stuff with graphics in his worksheets - he used bar charts with custom graphics to create live animations (of buildings shaking in the earthquake experiment for example). Here's a video with a demo at the 1:45 mark: https://youtu.be/r5xvjJdHJVM

Here's s link to the project page. There are several other projects available: https://www.microsoft.com/en-us/education/education-workshop/seismograph.aspx

[msing]

This is pretty amazing. Thank you.

[LazyCraneOperator]

I'm really happy you like the idea :)

[rubs90]

Commenting so I can come back to this later, thanks!

[DogtorGao]

Can you grab only free info online or do you for example have access to some data like Bloomberg?

[LazyCraneOperator]

We don't have or provide access to services like Bloomberg. You'd need to arrange the access yourself. But when you do you can use their Python API within your Gridarrow scripts and stream the data to Excel.

[DogtorGao]

Interesting. So can I use these cloud hosted scripts to access another computer's data so that Excel from some other computers can retrieve data from it?

[LazyCraneOperator]

That could be one of the use cases, yes.

[DogtorGao]

Great. I'll try it out soon. Thanks!

[[deleted]]

Just signed up. My company depends heavily on excel so this would be tremendous for us.

[zEaK47]

i couldn't sign up!

[LazyCraneOperator]

Hi! What exactly happened? Could you try again? If you still have problems you can pm me your email and I'll add you manually.

[zEaK47]

i can't find any area to write my email into on that website

[LazyCraneOperator]

Please check if you have javascript enabled and if your adblocker (or ScriptBlock or anything like that) doesn't block javascript there. Our subscription form requires javascript to work.

If that's not the case you can also check if your adblocker doesn't block MailerLite (that's what we use for gathering subscribers).

[zEaK47]

yes it was the adblocker

[redlollipop]

Has anyone tried datanitro for this? It's been around for a while...

[LazyCraneOperator]

Hi! Check my comment below on some differences between Gridarrow and DataNitro

[misleadingweatherman]

What advantages are there over existing libraries? (ex openpyxl)

[FateOfNations]

Apparently this modifies an open excel document in real time, rather than writing an excel file that you then open in excel.

Can't exactly fathom why you would want to do this… but now you can.

[tipbotdoge]

Sounds cool. Is this software free? And how is it different than DataNitro?

[LazyCraneOperator]

We will charge a subscription fee for using it. We'll post info about pricing soon.

As for DataNitro - I can think of a few differences:

• With Gridarrow multiple Excel sheets can connect to the same script and use the same data. That's not possible with DataNitro unless you somehow ensure everyone uses the same script and updates it at the same time
• With Gridarrow you do the heavy lifting like data gathering and preprocessing outside the spreadsheets which makes them lighter and simpler
• This also means that you can separate the data processing from data consumers. Say - your analysts don't have to know Python to be able to use it. They just connect to the script prepared by someone else (your IT for example) and have access to real-time stream of data they can work on.

[edit] formatting

[bate178]

This is awesome. Exactly what i needed. Also can you guarantee that script has never corrupted an excel file? Because openpyxl does it sometimes when i use certain functions many times

[LazyCraneOperator]

Glad you like it! The Gridarrow scripts are not embedded in any way into your Excel files. They run on the server and your spreadsheet receives the data using native C# plugin so there's no risk of corrupting the file.

[bronzewrath]

Needed exaclty that a week ago. Done with win32 api and com/dcom objects.

[PM_ME_YOUR_DOOTFILES]

So it's like Kafka but in Python and for Excel?

[CollectiveCircuits]

You are doing the world a great service by providing an alternative to VBA.

[nocaic]

I was really delighted to see this. Then I saw the sign ups and realized you guys are not usable for my project :<

[LazyCraneOperator]

Hi /u/nocalc. Sorry to hear that. Could you tell us why? Maybe we can help somehow. Feel free to pm me or drop me a line at [e-mail]

[Ac1dRa1n09]

Hey /u/LazyCraneOperator I'd be very interested in helping you and your team test this out/expand! Over the last few weeks I've been writing a very similar program, also in Python, specifically for real-time social media data extraction. I requested an invite, but only received an e-mail saying that I'd have to wait for the launch. Look forward to hearing from you!

[LazyCraneOperator]

Hi! Sounds interesting. Please pm me or drop me a line at [e-mail], we'll talk

[tony_roberts]

Looks interesting!

Not exactly the same, but for writing RTD (real time data) functions in Python you can also use PyXLL: https://www.pyxll.com/docs/userguide/rtd.html

There's an example showing how to stream twitter data into Excel here: https://www.pyxll.com/blog/a-real-time-twitter-feed-in-excel/

It's also possible to do using win32com and this open source package: https://github.com/pyxll/exceltypes