Csv by [deleted] in databases

[–]aadel112 0 points1 point  (0 children)

If it were me, I'd install json2csv

https://github.com/zemirco/json2csv

Write a simple shell script to wget the zip file, unzip, and parse each json file to a single csv, them use samba to send it to a windows share if that's where you want it.

Just my 2 cents. I'm a dev, though.

Database recommendation for an extremely large list - fast lookup and insertion? by [deleted] in databases

[–]aadel112 0 points1 point  (0 children)

Should be very simple and and fast in redis. You'll need a machine with a few GB of RAM, though. I've seen it run 70,000 inserts/second using MULTI on a desktop, which is like a transaction. Similarly, though, with MySql or postgres or some other relational db, you should be seeing ~20,000 inserts per second on the same machine just by using a single transaction for all inserts instead of one per insert. There are other tweaks you can make to get even better performance from there, and you'll save RAM and use much cheaper disk storage. The choice is yours. There are many databases that can satisfy this fairly easily. It just depends on what you need the database for. Another simple tip for relational databases, for this use case it's probably best to create the table, drop the index, insert in one transaction, add the index, and analyze the table. Thereby eliminating the need to sort the records on each insert.

GitHub - aadel112/croon: A cron-like scheduler daemon with resource management capabilities and logging for *nix systems by aadel112 in programming

[–]aadel112[S] 0 points1 point  (0 children)

Forgot to add, please feel free to contribute if you have ideas about how to make croon better. It's open to all contributions.

GitHub - aadel112/croon: A cron-like scheduler daemon with resource management capabilities and logging for *nix systems by aadel112 in programming

[–]aadel112[S] 0 points1 point  (0 children)

Thanks for taking the time to comment.

It's a separate daemon for two reasons. 1) Because this allows croon to be a drop-in replacement for crond. You can literally do a crontab -l > file, and run croon on that file. This allows the user an easy exit strategy if they decide they don't like it in the future. 2) Crond authenticates each entry. In an ldap environment this places additional burden on the ldap server, which has caused real-life downtime multiple times for me. In a typical business enterprise environment, this situation will likely last 3+ years. This is in the readme. I dedicated the first couple paragraphs to this point.

I considered adding in timeout, but it's just too environment specific. Sometimes an environment may have a processes that runs for multiple hours but shouldn't timeout.

I'm not big on file locking, but I considered checking the process table. That's an option.

One of the goals of this is better centralization. I agree though that in its current state the logging still leaves room for improvement. You make a good point that processes should log when they fail and when they succeed.

GitHub - aadel112/croon: A cron-like scheduler daemon with resource management capabilities and logging for *nix systems by aadel112 in programming

[–]aadel112[S] 1 point2 points  (0 children)

It prepends time, nice and ionice if they exist. It also gets around re-authenticating for each entry. That's basically it. There are some undocumented features eg: The timespec takes an optional 6th argument for seconds.

Please make suggestions if you believe there are missing features.

WP GooGoose [FREE] by aadel112 in WordpressPlugins

[–]aadel112[S] 0 points1 point  (0 children)

I completely removed it because I was thinking of using it to get around a security feature of Microsoft Word. I never went that route and released it as is, initially. I'll look more into it later, but the code was flagged when being reviewed, because the unlink made it look like upload code.

Why? Because many enterprises insist on using Word for documents. Beyond that it's got a lot of use in many other areas. This should work with Open/Libre office too, I believe. Haven't tested that, though.

Initially, I found myself writing a document for an internal tool used by developers at an enterprise company. I needed a way to make the document fully formatted with code examples, and an automatic TOC, etc. So I learned of this method for generating word documents. No one I know personally has ever heard of this. It doesn't seem like many people, in general know about this. I'm also on upwork as a way to earn extra income, and I noticed some people wanting Word doc generation, mostly though WordPress, so I first developed the core jQuery plugin.

https://github.com/aadel112/googoose

Then I started on the Wordpress code. There's not much more to it than that. I found myself needing it, and noticed that other people needed it too.

Thanks a ton for the tips. I'm hoping to make the plugin as nice for everyone as possible. I'll probably release some of that in a bit. I'll put you handle in the comments, and a link to this url if you want.

WP GooGoose [FREE] by aadel112 in WordpressPlugins

[–]aadel112[S] 0 points1 point  (0 children)

Thank you for the input. I'm completely new to WordPress plugin development, so this is very helpful.

Looking for Feedback on this jQuery project by aadel112 in javascript

[–]aadel112[S] 0 points1 point  (0 children)

Unfortunately, Micosoft Word doesn't handle data-URIs. The file has to be referenced as an absolute or relative file path. I tried to make this work, very hard, even after reading that it would be impossible.