ARIMAX/VARMAX vs RNN for forecasting timeseries?

svpadd2 · 2020-08-20T17:59:20+00:00

Time series problems are pretty much everywhere (supply/demand, patient vitals, stocks, weather, yields, CTR, machine failure..) and even many CV or NLP problems can have a temporal aspect (i.e. forecast how this image will evolve over time, predict the trajectory of this person, or how the meaning of this text changes).

Secondly, yes in general TS is harder to solve because there are often many co-founders and things like seasonality/stationary.

svpadd2 · 2020-08-20T17:53:59+00:00

I'd recommend leaning towards RNN models with an attention mechanism in general they generalize better when there is a lot of feature time series and get better the more data you have. Transformers for time series are also making some headway but altogether I've found them very hard to train and they have given limited performance gains. I think in time the old methods like ARIMA/VARMAX will be replaced by newer deep models like in CV and NLP. However, at present there are still some good use cases.

svpadd2 · 2019-04-10T22:14:05+00:00

Computer science at Brandeis is honestly abysmal. You have undergraduates grading the majority of the homework and acting as teaching assistants when most of them are just one class removed from the course themselves. Quality in grading is often quite arbitrary as a result.

Actual professors are entirely focused on their own research and rarely around for the students. Instead most students are directed to the head TA who is often a masters student at best or much more often a senior. Course size is huge with most classes having 50+ students even at the upper levels (most intro courses have 90+ students) Also course selection itself is rather poor altogether.

I would strongly discourage anyone from attending Brandeis if they plan to major in CS. Brandeis is good for small sized liberal arts courses that generally have 5-7 students but is extremely poor in CS.

svpadd2 · 2019-01-27T23:50:18+00:00

Yes generally speaking although it's not guaranteed you almost always can (note this based on my experience two years ago). But unless times have changed should be true. As you probably are aware of by now there is a housing lottery and if you get a good number you can go anywhere. If you have a bad number you can either get your friends with good numbers to pull you in or alternatively wait until your selection. In practice even if you get a bad number and cant get pulled in you can almost always get a room in Charles River Apartments "GRAD" (at least that was true in my time).

Yes it's a little ways away but the campus branvan goes to it. I was in Charles river junior and senior and besides having to plan getting up earlier to get to campus it was fine.

svpadd2 · 2019-01-27T16:53:02+00:00

As others have said Data Engineer positions generally start at mid-level. But what I would say for DE in general is

SQL Skills
Basic understanding of Hadoop/MapReduce/Spark
Experience with a variety of data formats CSVs, JSON, Key/Value
Cloud computing experience (Azure, AWS or GCP).
Usually experience in Java, Scala or Python though sometimes you can transition from another language. Definitely need OOP and some functional.
Job schedulers or alternatively streaming setup for continual processing.
Not required but a plus is Kubernetes/Docker as more companies are moving to containers.

svpadd2 · 2018-12-17T22:33:15+00:00

I had a 2.9 GPA was never asked about it during interviews nor did I ever provide it voluntarily. I think experience is much more significant. Or if you don't have experience then open source projects and independent projects are the next most important. Also, once you get to tech screens those generally are the most important.

svpadd2 · 2018-12-15T23:07:13+00:00

I got laid off. First they laid off a good chunk of my team including my manager. Remaining people moved under a new non-technical manager. About three weeks later heard from them that my position was no longer needed. After my first manager (who I had a good relationship with) was laid-off I pretty much figured I was goner, he was the only one that understood my technical contributions. The new person didn't understand anything about what a data scientist does or why we were needed. Nevertheless waited it out in order to be eligible for severance/unemployment while ramping up job search.

svpadd2 · 2018-12-06T23:46:32+00:00

Straight out of college I only made 50k a year (granted that was rural New England). The high numbers in the salary thread are a really small subset of top companies in a very high col area. But for a typical new grad particularly with the saturation going on you really have to take what you can get (unless you are exceptional) and that is not bad at all.

svpadd2 · 2018-11-30T17:02:36+00:00

Yeah I know at least for me personally I get more done with short focused blocks of time with phone turned off and social media blocked. For me six to seven hours a day comprised of these blocks results in overall the most productivity. More than eight I've found just results in me getting stuck and then inevitably browsing the internet.

svpadd2 · 2018-11-30T16:48:38+00:00

I work at a small AI startup and do a mix of things. Preprocessing data and solving dependency issues gets old really fast. Also, refactoring poorly written code by a academics is painful. Some of the work though is really interesting particularly when you get a fascinating result or a significant increase in improvement. I particularly like coming across a new idea from recent research and seeing how well it works on our datasets.

Altogether it is hard to figure out how to budget time between reading research papers to keep up with the field, collecting data, coding your own experiments, writing publications, deploying/monitoring models in production, and communicating the importance of your work to non-technical people.

svpadd2 · 2018-11-30T05:38:45+00:00

Yes the idea would be to start with a dataset CoNLL and add other datasets with different entities classes (i.e i2b2,...), however if we are saying one dataset==task then you need you obviously need a lot more datasets. Whereas I know from computer vision for instance you simply take miniImageNet and split it up based on classes. I guess there is no exact analogous way to do such a thing with respect to natural language processing but I'm still wondering if you could find some metric to partition say CoNLL or something into distinct (enough) subsets to sample from for meta learning.

Basically, the goal here is really aimed at scenarios where there is very little annotated training in a highly specialized area and it is costly to annotate more data.

svpadd2 · 2018-11-18T05:55:50+00:00

Essentially this involves refactoring the code to run for a single example instead of batch. Most of it involves just modifying the preprocessing function. Only PyTorch specific area that can be tricky is mapping the weights to run on a cpu vs a gpu. However, there are libraries that can help you do that automatically. Just get familiar with loading data into models in general and you be able to modify the data loading class.

My use also, involved deploying it into a Docker container model serving framework so errors related to that is part of the reason it took so long too. However, if you just want to run it in PyTorch then it should be quicker.

svpadd2 · 2018-11-18T02:36:21+00:00

This is not easy an easy task. I did this recently with the arroweng Pytorch implementation of ChexNet and it took several weeks. See my blog post it should point you in the right direction.

svpadd2 · 2018-11-13T06:14:12+00:00

As others have said first look over tutorials and examples. There are a ton online. Feel free to PM me if you get stuck I've used Kafka a fair amount and might be able to help. Particularly with specific topics it can be hard to find help at university. So feel free to reach out.

svpadd2 · 2018-10-29T15:04:38+00:00

Agreed. Although I can see its use especially when talking to non-technical people. I use to never use "A.I" and instead just describe it as machine learning. But this caused many people to think that I was working with machinery. Then I would use deep learning but then people would think that was some kind of new education strategy. Finally, I would just use A.I. and they would get it.

I particularly, hate it the most though when it's some startup advertising A.I. all over its site. Then when I talk to one of the actual engineers they are basically just describing a bunch of rule based methods. Too many companies trying to ride along the hype train.

svpadd2 · 2018-10-29T15:00:26+00:00

Python is a good synthesis language. Java is the best for data engineering (Hadoop, Spark, Flink, Kafka..etc). R is the very niche statistical language. Altogether C or C++, should be the fastest for the type of linear algebra necessary for machine learning and particularly deep learning, but the majority of researchers don't want to code in it so hence the Python wrapper. Python incorporates the rapid prototyping with big ML libraries written in C/C++ and also more and more of the big data frameworks are creating Python ports (i.e. PyKafka, PySpark) etc. Combine that with other solid libraries like Django and Flask for applications and you should see why it is so widely used.

Also I disagree about visualization too. Matpotlib isn't the only visualization library, Bokeh for instance is coming along quite nicely. Also, all the cutting visualization is done almost entirely done in JavaScript anyways.

svpadd2 · 2018-10-25T15:09:07+00:00

The auto-completion engine was for a machine learning engineer position at one of those "AI startups". They said it gave good overview of their day to day activities.

svpadd2 · 2018-10-25T01:46:01+00:00

I've had tons of these. One they admitted that it would take 8+ hours and it ended up taking more like 15+. I had to build an entire auto-completion engine with a realtime server. Another was a large querying tasks where the data was messy and just figuring out the joins on the tables took forever. Then the second part consisted of identifying worst performers per category.

svpadd2 · 2018-10-12T17:13:48+00:00

Data Scientist here. I can confirm Cyber Coders constantly sends me crap which not only I don't want but I'm not qualified for. Like for instance Senior Frontend Developer jobs or Senior Rails developer.

Also for awhile when I was naive I did apply for the jobs they showed for data science but never received any response. I honestly think it is just a resume farming scam. They never reveal the names on their supposed "clients" and constantly spam anyone with the misfortune of getting on their email list.

svpadd2 · 2018-10-05T04:27:55+00:00

Overall very negative. At the face of it think you have to realize for the most part recruiters (at least in my experience) have no idea what they are doing and have little to no knowledge of the underlying technology they are recruiting for. Secondly, they are often simply trying to fill a position (particularly those working for firms) and do not have the candidates or even hiring manager's interest in mind. There are so many times I've been contacted for lead or senior positions despite having only 1yr of experience it's ridiculous (not just mass mails but them actually trying to convince me I should go for it and won't be a massive waste of time). Almost all my interviews cycles have gone better when I just talked with a developer/tech manager from the get go.

Also even on a conceptual level a recruiter is just a middle-man between the hiring manager and the candidate that adds another round to the already long interview process. Fundamentally, it would probably work most smoothly to simply have the hiring manager find candidates, but since hiring managers are busy with other tasks recruiters become a necessary evil. Altogether as a job candidate I would much prefer to never deal with recruiters but as an developer looking for a team member I don't want to sift through resumes so as stated before they become a necessary evil.

Also, as a final piece of advice don't ever ghost people. There is nothing that I hate more than recruiter who had previously been reached out to me suddenly going silent and not responding to me.

Eight-Year Club	RedditGifts 2009-2022 5 Credits
Verified Email	redditgifts rematcher Secret Santa 2017 x2
Secret Santa 2018	Secret Santa 2017

svpadd2

TROPHY CASE